LLMs don’t do formal reasoning - and that is a HUGE problem

Gary Marcus · 11 Oct 2024

Others here said this too

Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas.

Their wordsNeurosymbolic AI — combining such machinery with neural networks – is likely a necessary condition for going forward.

Also said by François Chollet

Read the piece on garymarcus.substack.com ↗

2 korrents from this piece ↓

Gary Marcus did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

Our reading
Large language models do not reason formally: their performance collapses as a problem is made bigger, in the way a calculator's never does.

Their wordsAnother manifestation of the lack of sufficiently abstract, formal reasoning in LLMs is the way in which performance often fall apart as problems are made bigger.
Our reading
Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas.

Their wordsNeurosymbolic AI — combining such machinery with neural networks – is likely a necessary condition for going forward.

Keep reading

The other pieces are the ones whose claims are worded most like this one's. That measures language, not agreement: they may argue the opposite.