korrents

On the map

Tap a claim on the ring to put it at the centre.

← Mixture-of-experts models can scale stably but begin to show fundamental limits as they are scaled further.

17 connected korrents · 17 moments from 11 Oct 2018 to 9 Sept 2026.

Everything filed under scaling laws scaling laws Everything filed under LLMs LLMs Everything filed under AI alignment AI alignment Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Mixture-of-experts models can scale stably but begin to show fundamental limits as they are scaled further. Mixture-of-experts models can scale stablybut begin to show fundamental limits asthey are scaled further. Last stated 6 years ago 4 Feb 2021 GW Gwern — holds since 2021-02-04 — tap for who they are Same subject: The scaling law for world models was a conviction held in advance, not a discovery; the devils were always in the architecture choices and the data mixture. — tap to centre the map on it The scaling law for world models wasa conviction held in advance, not adiscovery; the devils were always inthe architecture choices and thedata mixture. Last stated 3 months ago 4 Sept 2026 FL Fei-Fei Li — holds since 2026-09-04 — tap for who they are Same subject: The scaling curve really is S-shaped, but there are at least two cycles left in it, which means models sixteen times smarter and all knowledge work subsumed. — tap to centre the map on it The scaling curve really isS-shaped, but there are at least twocycles left in it, which meansmodels sixteen times smarter and allknowledge work subsumed. Last stated 8 months ago 11 Mar 2026 SY Steve Yegge — holds since 2026-03-11 — tap for who they are Same subject: Alignment and values are not guaranteed to emerge as models scale. — tap to centre the map on it Alignment and values are notguaranteed to emerge as modelsscale. Last stated 3 years ago 8 Aug 2023 DA Dario Amodei — holds since 2023-08-08 — tap for who they are Same subject: In verifiable domains model capability scaling should remain unbounded, because the space of enumerable patterns is infinite by construction. — tap to centre the map on it In verifiable domains modelcapability scaling should remainunbounded, because the space ofenumerable patterns is infinite byconstruction. Last stated 3 months ago 27 Aug 2026 FC François Chollet — holds since 2026-08-27 — tap for who they are Same subject: With continued scaling, models will keep improving across the board, with no area where they are not starting to make progress. — tap to centre the map on it With continued scaling, models willkeep improving across the board,with no area where they are notstarting to make progress. Last stated 3 years ago 8 Aug 2023 DA Dario Amodei — holds since 2023-08-08 — tap for who they are Same subject: Scaling laws are still working, but the models people actually use run a few months behind the maximum capability Google can deliver, because the biggest model is too slow and expensive to serve. — tap to centre the map on it Scaling laws are still working, butthe models people actually use run afew months behind the maximumcapability Google can deliver,because the biggest model is tooslow and expensive to serve. Last stated a year ago 5 Jun 2025 SP Sundar Pichai — holds since 2025-06-05 — tap for who they are Same subject: How far AI systems are scaled must be constrained by how confident we are in their safety. — tap to centre the map on it How far AI systems are scaled mustbe constrained by how confident weare in their safety. Last stated 3 months ago 6 Sept 2026 JP Jakub Pachocki — holds since 2026-09-06 — tap for who they are Same subject: The arguments for scalable end-to-end interpretability seem reasonable in theory, because the bitter lesson is powerful. — tap to centre the map on it The arguments for scalableend-to-end interpretability seemreasonable in theory, because thebitter lesson is powerful. Last stated 11 months ago 20 Dec 2025 NN Neel Nanda — holds since 2025-12-20 — tap for who they are Same subject: A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. — tap to centre the map on it A deep bidirectional model isstrictly more powerful than aleft-to-right model or a shallowconcatenation of unidirectionalmodels. Last stated 8 years ago 11 Oct 2018 KT Kristina Toutanova — holds since 2018-10-11 — tap for who they are MC Ming-Wei Chang — holds since 2018-10-11 — tap for who they are KL Kenton Lee — holds since 2018-10-11 — tap for who they are JD Jacob Devlin — holds since 2018-10-11 — tap for who they are Same subject: A model may internally represent that it is uncertain, but that signal is not wired up to it saying in words that it does not know. — tap to centre the map on it A model may internally representthat it is uncertain, but thatsignal is not wired up to it sayingin words that it does not know. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: A neural network's latent space is closer to an uncopyrightable syntax than to copyrightable expression. — tap to centre the map on it A neural network's latent space iscloser to an uncopyrightable syntaxthan to copyrightable expression. Last stated 2 months ago 7 Sept 2026 KK Kevin Kelly — holds since 2026-09-07 — tap for who they are Same subject: Large language models are grown rather than built, and nobody really knows how the resulting working AI functions. — tap to centre the map on it Large language models are grownrather than built, and nobody reallyknows how the resulting working AIfunctions. Last stated 2 months ago 8 Sept 2026 SA Scott Alexander — holds since 2026-09-08 — tap for who they are Same subject: Longer test-time thinking improves an AI model's robustness to adversarial or unusual inputs. — tap to centre the map on it Longer test-time thinking improvesan AI model's robustness toadversarial or unusual inputs. Last stated 2 years ago 1 May 2025 LW Lilian Weng — holds since 2025-05-01 — tap for who they are Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it Looped/recurrent-depth transformerarchitectures can improve modelquality at a fixed compute budgetwhen the model is large enough. Last stated 2 months ago 9 Sept 2026 SR Sebastian Raschka — holds since 2026-09-09 — tap for who they are Same subject: Work on large language models is not progress toward artificial general intelligence, because the models do not understand what they read. — tap to centre the map on it Work on large language models is notprogress toward artificial generalintelligence, because the models donot understand what they read. Last stated a year ago 30 Jul 2025 AG Alexey Guzey — holds since 2024-08-09 — tap for who they are AG Alexey Guzey — no longer holds since 2025-07-30 — tap for who they are Same subject: A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data. — tap to centre the map on it A model claiming to be built byOpenAI is not necessarily evidencethat it was trained on OpenAI data. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact. — tap to centre the map on it A tiny language model inventing aplausible-sounding name is the samephenomenon as a large oneconfidently stating a false fact. Last stated 9 months ago 12 Feb 2026 AK Andrej Karpathy — holds since 2026-02-12 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they arefaded, dashed ring: they no longer hold it — they changed their mind

At the centre Mixture-of-experts models can scale stably but begin to show fundamental limits as they are scaled further. Last stated 4 Feb 2021 · 6 years ago Holds Gwern Read this korrent →