What François Chollet thinks about LLMs
Creator of the Keras deep-learning library and the ARC-AGI benchmark.
François Chollet did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter LLMs · Newest first
Only François Chollet on LLMs. See everyone on LLMs
6 dated positions, 2024 to 2026, in their own words. Our reading of what François Chollet has said — not written or endorsed by them.
-
Their wordsThe only way to avoid extreme power concentration is to ensure we have multiple independent providers of frontier AI models, including open-source options.
-
Our readingCurrent models are unsafe because they are not smart enough, not because they are too smart.
Their wordsCurrent models are unsafe not because they're too smart, but because they take goals too literally or take nonsensical shortcuts to achieve these goals, i.e. they're RL-fried. They lack common sense. They don't do the right thing in the face of ambiguity. Basically, they're not smart enough.
- 5 weeks earlier
-
Their wordsHowever, looking ahead, I still do not believe that future AI (say, in 15 years) will be based on the LLM stack. I believe it will necessarily have to move closer to its optimal, final form -- symbolic learning. Obviously this is a risky and contrarian belief -- the safe bet would be LRMs. But let's see.
-
Our reading · no longer heldThe LLM line of research will reach a capability plateau.
measuring intelligencescaling laws
Recalls holdingIn the era of base LLM scaling (2022-2024), I believed the LLM line of research would reach a capability plateau (as later seen with base LLMs). In late 2024, after the o3 test-time compute demo, I changed my views: the new models were showing genuine fluid intelligence, and with this new line of work, the LLM line of research could achieve unbounded capability scaling. "There will be no wall."
SaidIn the era of base LLM scaling (2022-2024), I believed the LLM line of research would reach a capability plateau (as later seen with base LLMs). In late 2024, after the o3 test-time compute demo, I changed my views: the new models were showing genuine fluid intelligence, and with this new line of work, the LLM line of research could achieve unbounded capability scaling. "There will be no wall."
- 20 months earlier
-
Their wordsThis "memorize, fetch, apply" paradigm can achieve arbitrary levels of skills at arbitrary tasks given appropriate training data, but it cannot adapt to novelty or pick up new skills on the fly (which is to say that there is no fluid intelligence at play here.)
↗OpenAI o3 Breakthrough High Score on ARC-AGI-Pubarcprize.org
-
Our reading · no longer heldThe memorize-fetch-apply paradigm behind LLMs can reach arbitrary skill given training data, but it cannot adapt to novelty or acquire new skills on the fly.
No longer holdsOpenAI's new o3 model represents a significant leap forward in AI's ability to adapt to novel tasks. This is not merely incremental improvement, but a genuine breakthrough, marking a qualitative shift in AI capabilities compared to the prior limitations of LLMs.
SaidOpenAI's new o3 model represents a significant leap forward in AI's ability to adapt to novel tasks. This is not merely incremental improvement, but a genuine breakthrough, marking a qualitative shift in AI capabilities compared to the prior limitations of LLMs.
↗OpenAI o3 Breakthrough High Score on ARC-AGI-Pubarcprize.org