Thank you very much for posting! This is exactly what I was looking for.
On one hand, I understand what he's saying, and that's why I have been frustrated in the past when I've heard people say "it's just fancy autocomplete" without emphasizing the awesome capabilities that can give you. While I haven't seen this video by Sutskever before, I have seen a very similar argument by Hinton: in order to get really good at next token prediction, the model needs to "discover" the underlying rules that make that prediction possible.
All that said, I find his argument wholly unconvincing (and again, I may be waaaaay stupider than Sutskever, but there are other people much smarter than I who agree). And the reason for this is because every now and then I'll see a particular type of hallucination where it's pretty obvious that the LLM is confusing similar token strings even when their underlying meaning is very different. That is, the underlying "pattern matching" of LLMs becomes apparent in these situations.
As I said originally, I'm really glad VCs are pouring money into this, but I'd easily make a bet that in 5 years that LLMs will be nowhere near human-level intelligence on some tasks, especially where novel discovery is required.
Watching that video actually makes me completely unconvinced that SSI will succeed if they are hinging it on LLM...
He puts a lot of emphasis on the fact that 'to generate the next token you must understand how', when thats precisely the parlor trick that is making people lose their minds (myself included) with how effective current LLMs are. The fact that it can simulate some low-fidelity reality with _no higher-level understanding of the world_, using purely linguistic/statistical analysis, is mind-blowing. To say "all you have to do is then extrapolate" is the ultimate "draw the rest of the owl" argument.
I actually echo your exact sentiments. I don't have the street cred but watching him talk for the first few minutes I immediately felt like there is just no way we are going to get AGI with what we know today.
Without some raw reasoning (maybe Neuro-symbolic is the answer maybe not) capacity, LLM won't be enough. Reasoning is super tough because its not as easy as predicting the next most likely token.
>All that said, I find his argument wholly unconvincing (and again, I may be waaaaay stupider than Sutskever, but there are other people much smarter than I who agree). And the reason for this is because every now and then I'll see a particular type of hallucination where it's pretty obvious that the LLM is confusing similar token strings even when their underlying meaning is very different. That is, the underlying "pattern matching" of LLMs becomes apparent in these situations.
So? One of the most frustrating parts of these discussions is that for some bizzare reason, a lot of people have a standard of reasoning (for machines) that only exists in fiction or their own imaginations.
Humans have a long list of cognitive shortcomings. We find them interesting and give them all sorts of names like cognitive dissonance or optical illusions. But we don't currently make silly conclusions like humans don't reason.
The general reasoning engine that makes neither mistake nor contradiction or confusion in output or process does not exist in real life whether you believe Humans are the only intelligent species on the planet or are gracious enough to extend the capability to some of our animal friends.
So the LLM confuses tokens every now and then. So what ?
> Humans have a long list of cognitive shortcomings. We find them interesting and give them all sorts of names like cognitive dissonance or optical illusions. But we don't currently make silly conclusions like humans don't reason.
Exactly! In fact, things like illusions are actually excellent windows into how the mind really works. Most visual illusions are a fundamental artifact of how the brain needs to turn a 2D image into a 3D, real-world model, and illusions give clues into how it does that, and how the contours of the natural world guided the evolution of the visual system (I think Steven Pinker's "How the Mind Works" gives excellent examples of this).
So I am not at all saying that what LLMs do isn't extremely interesting, or useful. What I am saying is that the types of errors you get give a window into how an LLM works, and these hint at some fundamental limitations at what an LLM is capable of, particularly around novel discovery and development of new ideas and theories that aren't just "rearrangements" of existing ideas.
>So I am not at all saying that what LLMs do isn't extremely interesting, or useful. What I am saying is that the types of errors you get give a window into how an LLM works, and these hint at some fundamental limitations at what an LLM is capable of, particularly around novel discovery and development of new ideas and theories that aren't just "rearrangements" of existing ideas.
ANN architectures are not like brains. They don't come pre-baked with all sorts of evolutionary steps and tweaking. They're far more blank slate and the transformer is one of the most blank slate there is.
Mostly at best, maybe some failure mode in GPT-N gives insight to how some concept is understood by GPT-N. It rarely will say anything about language modelling or Transformers.
GPT-2 had some wildly different failure modes than 3, which itself has some wildly different failure modes to 4.
All a transformer's training objective asks it to do is spit out a token. How it should do so is left for transformer to figure along the way and everything is fair game.
And confusing words with wildly different meanings but with some similarity in some other way is something that happens to humans as well. Transformers don't see words or letters(but tokens). So just because it doesn't seem to you like two tokens should be confused doesn't mean there isn't a valid point of confusion there.
They might never work for novel discovery but that probably can be handled by outside loop or online (in-context) learning. The thing is that 100k or 1M context is a marketing scam for now.
To clarify this, I think it's reasonable that token prediction as a training objective could lead to AGI given the underlying model has the correct architecture. The question really is if the underlying architecture is good enough to capitalize on the training objective so as to result in superhuman intelligence.
For example, you'll have little luck achieving AGI with decision trees no matter what's their training objective.
He doesn't address the real question of how an LLM predicting the next token could exceed what humans have done. They mostly interpolate, so if the answer isn't to be found in an interpolation, the LLM can't generate something new.
Arizona is a great case where agrovoltaics could really work. Solar reduces the sun load to something reasonable and minimizes evaporation, plants cool the air under the panels. Human / nature infrastructure symbiosis.
If you ask the questions without providing the limerick first, it never gets the right answer. When the LLM gets the wrong answer, it is usually because it reverts to its training data and gives a generic answer that doesn't apply to the limerick.
Why are you ruling out the possibility that training on the material may confer an advantage when the data is presented, even if the advantage may not be strong enough to pass the test without the data present in the context window?
ESL is indeed widely recommended for ML but perhaps, I am looking for something more data-exploration focused. In any case, did you find Kevin Murphy's MLPP to be helpful? I read the book in detail and found the book as an introductory mathematical book for discussing ML algorithms, rather than doing any real world data-modeling or prediction error analysis -- it may help with ML, but not sure if it will help me with DS. And the book (at least the edition I used) was full of non-trivial errors.
so basically what TFA says is if you build a new model on Nate's model outputs and use that to compute conditional probabilites of various improbable scenarios you see weird shit. State level polling data according to Nate isn't all that great (read: high uncertainty, moves around when a new poll comes out) unless it's a swing state. But hey when you go looking into noise you are going to see.... weird shit.
