korrents

Our readingWhat is a korrent?

Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas.

Drawn from what François Chollet and Gary Marcus said

What these subjects mean

scaling laws Whether more compute keeps buying more capability, and where the curve now bends -- pre-training, post-training, test-time.

neural networks The architecture almost all modern machine learning is built on, and what changes when it gets deeper.

What they actually said

  1. François Chollet Creator of the Keras deep-learning library and the ARC-AGI benchmark o3's improvement over the GPT series proves that architecture is everything. You couldn't throw more compute at GPT-4 and get these results. Simply scaling up the things we were doing from 2019 to 2023 -- take the same architecture, train a bigger version on more data -- is not enough. OpenAI o3 Breakthrough High Score on ARC-AGI-Pubarcprize.org · 20 Dec 2024
    o3's improvement over the GPT series proves that architecture is everything. You couldn't throw more compute at GPT-4 and get these results. Simply scaling up the things we were doing from 2019 to 2023 -- take the same architecture, train a bigger version on more data -- is not enough.
  2. Gary Marcus Cognitive scientist, professor emeritus of psychology… Neurosymbolic AI — combining such machinery with neural networks – is likely a necessary condition for going forward. LLMs don’t do formal reasoning - and that is a HUGE problemgarymarcus.substack.com · 11 Oct 2024All 2 korrents from this piece
    Neurosymbolic AI — combining such machinery with neural networks – is likely a necessary condition for going forward.

Added to korrents 29 Aug 2026 · How quotes work · Something wrong? Tell us

Holders over time
Holders over time
012 Oct 2024 Oct 2026

Do you hold this korrent?Do you also believe this?

I hold this too

Where this collides

Related korrents

Shared subject first, then similar wording. They may agree or disagree.

On the map

Loading the map… or open it on its own page

Open the map on its own page