Tap a claim on the ring to put it at the centre.
← Spending most of the compute memorising the world in pre-training is…
17 connected korrents · 17 moments from 28 Oct 2013 to 19 Sept 2026. Nearly all of them are about scaling laws .
Everything filed under LLMs
LLMs
Everything filed under mental models
mental models
Everything filed under benchmarks
benchmarks
Everything filed under AI agents
AI agents Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Spending most of the compute memorising the world in pre-training is brute force; what we want is a model that reasons the way a student sits an open-book exam. Spending most of the compute memorising the world in pre-training is brute force; what we want is a model that reasons the way a student sits an open-book exam. Last stated 2 years ago
19 Jun 2024
AS
Aravind Srinivas — holds since 2024-06-19 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 7 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
Same subject: After pre-training, post-training and test-time scaling, the fourth scaling law is agentic: multiplying AI by spawning agents, and the whole loop scales on one thing, compute. — tap to centre the map on it After pre-training, post-training and test-time scaling, the fourth scaling law is agentic: multiplying AI by spawning agents, and the whole loop scales on one thing, compute. Last stated 7 months ago
23 Mar 2026
JH
Jensen Huang — holds since 2026-03-23 — tap for who they are
Same subject: AI scaling laws require exponentially more compute and resources to produce only linear gains in model intelligence. — tap to centre the map on it AI scaling laws require exponentially more compute and resources to produce only linear gains in model intelligence. Last stated a month ago
19 Sept 2026
NL
Nathan Lambert — holds since 2026-09-19 — tap for who they are
Same subject: As models keep scaling up, modularity will become essential to how they are developed. — tap to centre the map on it As models keep scaling up, modularity will become essential to how they are developed. Last stated 4 years ago
23 Feb 2023
SR
Sebastian Ruder — holds since 2023-02-23 — tap for who they are
Same subject: Bet on a system that is maximally learned and minimally constrained, and add structure only where it improves the scaling laws. — tap to centre the map on it Bet on a system that is maximally learned and minimally constrained, and add structure only where it improves the scaling laws. Last stated 3 months ago
3 Aug 2026
DD
Dmitri Dolgov — holds since 2026-08-03 — tap for who they are
Same subject: Current AI techniques are four to six orders of magnitude away from optimal in data efficiency and test-time compute efficiency. — tap to centre the map on it Current AI techniques are four to six orders of magnitude away from optimal in data efficiency and test-time compute efficiency. Last stated 3 months ago
7 Aug 2026
FC
François Chollet — holds since 2026-08-07 — tap for who they are
Same subject: Current techniques restrict pre-trained representation power because standard language models are unidirectional. — tap to centre the map on it Current techniques restrict pre-trained representation power because standard language models are unidirectional. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: Rather than having AI just do the task, people should use it to help build the right mental models. — tap to centre the map on it Rather than having AI just do the task, people should use it to help build the right mental models. Last stated 7 months ago
25 Mar 2026
FG
Flora Guo — holds since 2026-03-25 — tap for who they are
Same subject: A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI. — tap to centre the map on it A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI. Last stated 10 months ago
1 Jan 2026
JL
Jason Lemkin — holds since 2026-01-01 — tap for who they are
Same subject: A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. — tap to centre the map on it A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. Last stated 3 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: A difficult problem is better thought through both forwards and backward than in one direction only. — tap to centre the map on it A difficult problem is better thought through both forwards and backward than in one direction only. Last stated 13 years ago
28 Oct 2013
SP
Shane Parrish — holds since 2013-10-28 — tap for who they are
Same subject: A few dozen mental models are enough to get a firm grasp of how the world works. — tap to centre the map on it A few dozen mental models are enough to get a firm grasp of how the world works. Last stated 9 years ago
by 12 Aug 2017
JC
James Clear — holds since 2017-08-12 — tap for who they are
Same subject: A map is not the territory it represents, but if it is correct it shares the territory's structure, which is what makes it useful. — tap to centre the map on it A map is not the territory it represents, but if it is correct it shares the territory's structure, which is what makes it useful.
AK
Alfred Korzybski — holds since c. 1933 — tap for who they are
Same subject: A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact. — tap to centre the map on it A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact. Last stated 8 months ago
12 Feb 2026
AK
Andrej Karpathy — holds since 2026-02-12 — tap for who they are
Same subject: AI should be released iteratively rather than built in secret, because AI and surprise do not go together and the world needs time to adapt before it is under the gun. — tap to centre the map on it AI should be released iteratively rather than built in secret, because AI and surprise do not go together and the world needs time to adapt before it is under the gun. Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: An AI can already earn an A on a university economics midterm, years earlier than informed forecasters expected. — tap to centre the map on it An AI can already earn an A on a university economics midterm, years earlier than informed forecasters expected. Last stated 4 years ago
21 Mar 2023
BC
Bryan Caplan — holds since 2023-03-21 — tap for who they are
At the centre
Spending most of the compute memorising the world in pre-training is brute force; what we want is a model that reasons the way a student sits an open-book exam.
Holds Read this korrent →
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds
After pre-training, post-training and test-time scaling, the fourth scaling law is agentic: multiplying AI by spawning agents, and the whole loop scales on one thing, compute.
Holds
AI scaling laws require exponentially more compute and resources to produce only linear gains in model intelligence.
Holds
As models keep scaling up, modularity will become essential to how they are developed.
Holds
Bet on a system that is maximally learned and minimally constrained, and add structure only where it improves the scaling laws.
Holds
Current AI techniques are four to six orders of magnitude away from optimal in data efficiency and test-time compute efficiency.
Holds
Current techniques restrict pre-trained representation power because standard language models are unidirectional.
Holds
Rather than having AI just do the task, people should use it to help build the right mental models.
Holds
A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI.
Holds
A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to.
Holds
A difficult problem is better thought through both forwards and backward than in one direction only.
Holds
A few dozen mental models are enough to get a firm grasp of how the world works.
Holds
A map is not the territory it represents, but if it is correct it shares the territory's structure, which is what makes it useful.
Last stated c. 1933
Holds
A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact.
Holds
AI should be released iteratively rather than built in secret, because AI and surprise do not go together and the world needs time to adapt before it is under the gun.
Holds
An AI can already earn an A on a university economics midterm, years earlier than informed forecasters expected.
Holds