korrents

korrents · piece

LLMs don’t do formal reasoning - and that is a HUGE problem

Gary Marcus · 11 Oct 2024 · garymarcus.substack.com ↗

2 korrents from this piece

Gary Marcus did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

  1. Our reading

    Large language models do not reason formally: their performance collapses as a problem is made bigger, in the way a calculator's never does.

    Their wordsAnother manifestation of the lack of sufficiently abstract, formal reasoning in LLMs is the way in which performance often fall apart as problems are made bigger.
    Interesting

  2. Our reading

    Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas.

    Their wordsNeurosymbolic AI — combining such machinery with neural networks – is likely a necessary condition for going forward.
    Interesting