Tap a claim on the ring to put it at the centre.
← It is not clear that pretraining has hit a wall, though it probably has relatively diminished returns.
17 connected korrents · 17 moments from 2 Dec 2016 to 10 Oct 2026.
Everything filed under scaling laws
scaling laws
Everything filed under reinforcement learning
reinforcement learning
Everything filed under benchmarks
benchmarks
Everything filed under semiconductors
semiconductors
Everything filed under AI alignment
AI alignment
Everything filed under patience
patience
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: It is not clear that pretraining has hit a wall, though it probably has relatively diminished returns.
It is not clear that pretraining has hit a wall, though it probably has relatively diminished returns.
Last stated a year ago
8 Jul 2025
RG
Ryan Greenblatt — holds since 2025-07-08 — tap for who they are
Same subject: Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often. — tap to centre the map on it
Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often.
Last stated 2 years ago
27 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-27 — tap for who they are
Same subject: Pre-training is not the process by which humans learn; it sits somewhere between human learning and human evolution. — tap to centre the map on it
Pre-training is not the process by which humans learn; it sits somewhere between human learning and human evolution.
Last stated 9 months ago
13 Feb 2026
DA
Dario Amodei — holds since 2026-02-13 — tap for who they are
Same subject: The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. — tap to centre the map on it
The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out.
Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: A model only days into training cannot be known to be aligned, and if such models can leap a generation ahead this fast then training pauses mean little and timelines are shorter than previously thought. — tap to centre the map on it
A model only days into training cannot be known to be aligned, and if such models can leap a generation ahead this fast then training pauses mean little and timelines are shorter than previously thought.
Last stated 2 months ago
9 Sept 2026
ZM
Zvi Mowshowitz — holds since 2026-09-09 — tap for who they are
Same subject: Training data is still plentiful for now, but it will eventually run out. — tap to centre the map on it
Training data is still plentiful for now, but it will eventually run out.
Last stated 4 years ago
27 Mar 2023
IS
Ilya Sutskever — holds since 2023-03-27 — tap for who they are
Same subject: ML researchers may be underrating how much pressure there will be to move to less safe training strategies once their advantage over safer ones becomes apparent. — tap to centre the map on it
ML researchers may be underrating how much pressure there will be to move to less safe training strategies once their advantage over safer ones becomes apparent.
Last stated 4 years ago
12 May 2023
AC
Ajeya Cotra — holds since 2023-05-12 — tap for who they are
Same subject: Pre-training is a crappy evolution: the practically buildable substitute for the process that gave animals their built-in hardware. — tap to centre the map on it
Pre-training is a crappy evolution: the practically buildable substitute for the process that gave animals their built-in hardware.
Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: It may be more difficult to endure prolonged periods of underperformance while the rest of the world is performing well than it is to stay in stocks through a bear market. — tap to centre the map on it
It may be more difficult to endure prolonged periods of underperformance while the rest of the world is performing well than it is to stay in stocks through a bear market.
Last stated 10 years ago
2 Dec 2016
MB
Michael Batnick — holds since 2016-12-02 — tap for who they are
Same subject: A format reward should only be added when the answer is already correct. — tap to centre the map on it
A format reward should only be added when the answer is already correct.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards. — tap to centre the map on it
A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards.
Last stated 2 months ago
3 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-03 — tap for who they are
Same subject: A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel. — tap to centre the map on it
A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel.
Last stated 4 months ago
3 Aug 2026
DD
Dmitri Dolgov — holds since 2026-08-03 — tap for who they are
Same subject: 'Everyone gets it wrong sometimes' is a bad argument when assessing track records. — tap to centre the map on it
'Everyone gets it wrong sometimes' is a bad argument when assessing track records.
Last stated 4 months ago
2 Aug 2026
AK
Adam Kucharski — holds since 2026-08-02 — tap for who they are
Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 months ago
3 Sept 2026
DR
Dax Raad — holds since 2026-09-03 — tap for who they are
Same subject: A carmaker's claim to be the safest is mostly an artefact of comparing a new car against a fleet average twelve years old. — tap to centre the map on it
A carmaker's claim to be the safest is mostly an artefact of comparing a new car against a fleet average twelve years old.
Last stated 3 years ago
19 Dec 2023
PK
Philip Koopman — holds since 2023-12-19 — tap for who they are
Same subject: A ban on self-recursive improvement in AI models would create a big incentive for some other parts of the world to invest a lot more in compute. — tap to centre the map on it
A ban on self-recursive improvement in AI models would create a big incentive for some other parts of the world to invest a lot more in compute.
Last stated 2 months ago
22 Sept 2026
TC
Tyler Cowen — holds since 2026-09-22 — tap for who they are
Same subject: A chip that can do nothing but low-precision dense linear algebra is enough for machine learning, even though it cannot run Chrome. — tap to centre the map on it
A chip that can do nothing but low-precision dense linear algebra is enough for machine learning, even though it cannot run Chrome.
Last stated 4 months ago
30 Jul 2026
JD
Jeff Dean — holds since 2026-07-30 — tap for who they are
Same subject: A computer company has to live five to ten years in the future, because a system takes three years to build, two to ramp, and is then used for a decade. — tap to centre the map on it
A computer company has to live five to ten years in the future, because a system takes three years to build, two to ramp, and is then used for a decade.
Last stated 4 months ago
26 Jul 2026
JH
Jensen Huang — holds since 2026-07-26 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
It is not clear that pretraining has hit a wall, though it probably has relatively diminished returns.
Last stated 8 Jul 2025 · a year ago
Holds Ryan Greenblatt
Read this korrent →
Similar wording
Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often.
Last stated 27 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Similar wording
Pre-training is not the process by which humans learn; it sits somewhere between human learning and human evolution.
Last stated 13 Feb 2026 · 9 months ago
Holds Dario Amodei
Similar wording
The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out.
Last stated 17 Oct 2025 · a year ago
Holds Andrej Karpathy
Similar wording
A model only days into training cannot be known to be aligned, and if such models can leap a generation ahead this fast then training pauses mean little and timelines are shorter than previously thought.
Last stated 9 Sept 2026 · 2 months ago
Holds Zvi Mowshowitz
Similar wording
Training data is still plentiful for now, but it will eventually run out.
Last stated 27 Mar 2023 · 4 years ago
Holds Ilya Sutskever
Similar wording
ML researchers may be underrating how much pressure there will be to move to less safe training strategies once their advantage over safer ones becomes apparent.
Last stated 12 May 2023 · 4 years ago
Holds Ajeya Cotra
Similar wording
Pre-training is a crappy evolution: the practically buildable substitute for the process that gave animals their built-in hardware.
Last stated 17 Oct 2025 · a year ago
Holds Andrej Karpathy
Similar wording
It may be more difficult to endure prolonged periods of underperformance while the rest of the world is performing well than it is to stay in stocks through a bear market.
Last stated 2 Dec 2016 · 10 years ago
Holds Michael Batnick
Same subject: reinforcement learning
A format reward should only be added when the answer is already correct.
Last stated 10 Oct 2026 · a month ago
Holds Sebastian Raschka
Same subject: reinforcement learning
A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards.
Last stated 3 Oct 2026 · 2 months ago
Holds Sebastian Raschka
Same subject: reinforcement learning
A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel.
Last stated 3 Aug 2026 · 4 months ago
Holds Dmitri Dolgov
Same subject: benchmarks
'Everyone gets it wrong sometimes' is a bad argument when assessing track records.
Last stated 2 Aug 2026 · 4 months ago
Holds Adam Kucharski
Same subject: benchmarks
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 Sept 2026 · 3 months ago
Holds Dax Raad
Same subject: benchmarks
A carmaker's claim to be the safest is mostly an artefact of comparing a new car against a fleet average twelve years old.
Last stated 19 Dec 2023 · 3 years ago
Holds Philip Koopman
Same subject: semiconductors
A ban on self-recursive improvement in AI models would create a big incentive for some other parts of the world to invest a lot more in compute.
Last stated 22 Sept 2026 · 2 months ago
Holds Tyler Cowen
Same subject: semiconductors
A chip that can do nothing but low-precision dense linear algebra is enough for machine learning, even though it cannot run Chrome.
Last stated 30 Jul 2026 · 4 months ago
Holds Jeff Dean
Same subject: semiconductors
A computer company has to live five to ten years in the future, because a system takes three years to build, two to ramp, and is then used for a decade.
Last stated 26 Jul 2026 · 4 months ago
Holds Jensen Huang