Tap a claim on the ring to put it at the centre.
← Mixture-of-experts models can scale stably but begin to show fundamental limits as they are scaled further.
17 connected korrents · 17 moments from 11 Oct 2018 to 9 Sept 2026.
Everything filed under scaling laws
scaling laws
Everything filed under LLMs
LLMs
Everything filed under AI alignment
AI alignment
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Mixture-of-experts models can scale stably but begin to show fundamental limits as they are scaled further.
Mixture-of-experts models can scale stably but begin to show fundamental limits as they are scaled further.
Last stated 6 years ago
4 Feb 2021
GW
Gwern — holds since 2021-02-04 — tap for who they are
Same subject: The scaling law for world models was a conviction held in advance, not a discovery; the devils were always in the architecture choices and the data mixture. — tap to centre the map on it
The scaling law for world models was a conviction held in advance, not a discovery; the devils were always in the architecture choices and the data mixture.
Last stated 3 months ago
4 Sept 2026
FL
Fei-Fei Li — holds since 2026-09-04 — tap for who they are
Same subject: The scaling curve really is S-shaped, but there are at least two cycles left in it, which means models sixteen times smarter and all knowledge work subsumed. — tap to centre the map on it
The scaling curve really is S-shaped, but there are at least two cycles left in it, which means models sixteen times smarter and all knowledge work subsumed.
Last stated 8 months ago
11 Mar 2026
SY
Steve Yegge — holds since 2026-03-11 — tap for who they are
Same subject: Alignment and values are not guaranteed to emerge as models scale. — tap to centre the map on it
Alignment and values are not guaranteed to emerge as models scale.
Last stated 3 years ago
8 Aug 2023
DA
Dario Amodei — holds since 2023-08-08 — tap for who they are
Same subject: In verifiable domains model capability scaling should remain unbounded, because the space of enumerable patterns is infinite by construction. — tap to centre the map on it
In verifiable domains model capability scaling should remain unbounded, because the space of enumerable patterns is infinite by construction.
Last stated 3 months ago
27 Aug 2026
FC
François Chollet — holds since 2026-08-27 — tap for who they are
Same subject: With continued scaling, models will keep improving across the board, with no area where they are not starting to make progress. — tap to centre the map on it
With continued scaling, models will keep improving across the board, with no area where they are not starting to make progress.
Last stated 3 years ago
8 Aug 2023
DA
Dario Amodei — holds since 2023-08-08 — tap for who they are
Same subject: Scaling laws are still working, but the models people actually use run a few months behind the maximum capability Google can deliver, because the biggest model is too slow and expensive to serve. — tap to centre the map on it
Scaling laws are still working, but the models people actually use run a few months behind the maximum capability Google can deliver, because the biggest model is too slow and expensive to serve.
Last stated a year ago
5 Jun 2025
SP
Sundar Pichai — holds since 2025-06-05 — tap for who they are
Same subject: How far AI systems are scaled must be constrained by how confident we are in their safety. — tap to centre the map on it
How far AI systems are scaled must be constrained by how confident we are in their safety.
Last stated 3 months ago
6 Sept 2026
JP
Jakub Pachocki — holds since 2026-09-06 — tap for who they are
Same subject: The arguments for scalable end-to-end interpretability seem reasonable in theory, because the bitter lesson is powerful. — tap to centre the map on it
The arguments for scalable end-to-end interpretability seem reasonable in theory, because the bitter lesson is powerful.
Last stated 11 months ago
20 Dec 2025
NN
Neel Nanda — holds since 2025-12-20 — tap for who they are
Same subject: A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. — tap to centre the map on it
A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models.
Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: A model may internally represent that it is uncertain, but that signal is not wired up to it saying in words that it does not know. — tap to centre the map on it
A model may internally represent that it is uncertain, but that signal is not wired up to it saying in words that it does not know.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: A neural network's latent space is closer to an uncopyrightable syntax than to copyrightable expression. — tap to centre the map on it
A neural network's latent space is closer to an uncopyrightable syntax than to copyrightable expression.
Last stated 2 months ago
7 Sept 2026
KK
Kevin Kelly — holds since 2026-09-07 — tap for who they are
Same subject: Large language models are grown rather than built, and nobody really knows how the resulting working AI functions. — tap to centre the map on it
Large language models are grown rather than built, and nobody really knows how the resulting working AI functions.
Last stated 2 months ago
8 Sept 2026
SA
Scott Alexander — holds since 2026-09-08 — tap for who they are
Same subject: Longer test-time thinking improves an AI model's robustness to adversarial or unusual inputs. — tap to centre the map on it
Longer test-time thinking improves an AI model's robustness to adversarial or unusual inputs.
Last stated 2 years ago
1 May 2025
LW
Lilian Weng — holds since 2025-05-01 — tap for who they are
Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it
Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.
Last stated 2 months ago
9 Sept 2026
SR
Sebastian Raschka — holds since 2026-09-09 — tap for who they are
Same subject: Work on large language models is not progress toward artificial general intelligence, because the models do not understand what they read. — tap to centre the map on it
Work on large language models is not progress toward artificial general intelligence, because the models do not understand what they read.
Last stated a year ago
30 Jul 2025
AG
Alexey Guzey — holds since 2024-08-09 — tap for who they are
AG
Alexey Guzey — no longer holds since 2025-07-30 — tap for who they are
Same subject: A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data. — tap to centre the map on it
A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact. — tap to centre the map on it
A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact.
Last stated 9 months ago
12 Feb 2026
AK
Andrej Karpathy — holds since 2026-02-12 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are faded, dashed ring: they no longer hold it — they changed their mind
At the centre
Mixture-of-experts models can scale stably but begin to show fundamental limits as they are scaled further.
Last stated 4 Feb 2021 · 6 years ago
Holds Gwern
Read this korrent →
Similar wording
The scaling law for world models was a conviction held in advance, not a discovery; the devils were always in the architecture choices and the data mixture.
Last stated 4 Sept 2026 · 3 months ago
Holds Fei-Fei Li
Similar wording
The scaling curve really is S-shaped, but there are at least two cycles left in it, which means models sixteen times smarter and all knowledge work subsumed.
Last stated 11 Mar 2026 · 8 months ago
Holds Steve Yegge
Similar wording
Alignment and values are not guaranteed to emerge as models scale.
Last stated 8 Aug 2023 · 3 years ago
Holds Dario Amodei
Similar wording
In verifiable domains model capability scaling should remain unbounded, because the space of enumerable patterns is infinite by construction.
Last stated 27 Aug 2026 · 3 months ago
Holds François Chollet
Similar wording
With continued scaling, models will keep improving across the board, with no area where they are not starting to make progress.
Last stated 8 Aug 2023 · 3 years ago
Holds Dario Amodei
Similar wording
Scaling laws are still working, but the models people actually use run a few months behind the maximum capability Google can deliver, because the biggest model is too slow and expensive to serve.
Last stated 5 Jun 2025 · a year ago
Holds Sundar Pichai
Similar wording
How far AI systems are scaled must be constrained by how confident we are in their safety.
Last stated 6 Sept 2026 · 3 months ago
Holds Jakub Pachocki
Similar wording
The arguments for scalable end-to-end interpretability seem reasonable in theory, because the bitter lesson is powerful.
Last stated 20 Dec 2025 · 11 months ago
Holds Neel Nanda
Same subject: LLMs
A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models.
Last stated 11 Oct 2018 · 8 years ago
Holds Kristina Toutanova Ming-Wei Chang Kenton Lee Jacob Devlin
Same subject: LLMs
A model may internally represent that it is uncertain, but that signal is not wired up to it saying in words that it does not know.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Same subject: LLMs
A neural network's latent space is closer to an uncopyrightable syntax than to copyrightable expression.
Last stated 7 Sept 2026 · 2 months ago
Holds Kevin Kelly
Same subject: neural networks
Large language models are grown rather than built, and nobody really knows how the resulting working AI functions.
Last stated 8 Sept 2026 · 2 months ago
Holds Scott Alexander
Same subject: neural networks
Longer test-time thinking improves an AI model's robustness to adversarial or unusual inputs.
Last stated 1 May 2025 · 2 years ago
Holds Lilian Weng
Same subject: neural networks
Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.
Last stated 9 Sept 2026 · 2 months ago
Holds Sebastian Raschka
Same subject: OpenAI
Work on large language models is not progress toward artificial general intelligence, because the models do not understand what they read.
Last stated 30 Jul 2025 · a year ago
No longer holds Alexey Guzey
Same subject: OpenAI
A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Same subject: OpenAI
A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact.
Last stated 12 Feb 2026 · 9 months ago
Holds Andrej Karpathy