korrents

On the map

Tap a claim on the ring to put it at the centre.

← It is not clear that pretraining has hit a wall, though it probably has relatively diminished returns.

17 connected korrents · 17 moments from 2 Dec 2016 to 10 Oct 2026.

Everything filed under scaling laws scaling laws Everything filed under reinforcement learning reinforcement learning Everything filed under benchmarks benchmarks Everything filed under semiconductors semiconductors Everything filed under AI alignment AI alignment Everything filed under patience patience Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: It is not clear that pretraining has hit a wall, though it probably has relatively diminished returns. It is not clear that pretraining has hit awall, though it probably has relativelydiminished returns. Last stated a year ago 8 Jul 2025 RG Ryan Greenblatt — holds since 2025-07-08 — tap for who they are Same subject: Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often. — tap to centre the map on it Language models should be thought ofas somewhat out of date, becausepre-training is too expensive torepeat often. Last stated 2 years ago 27 Feb 2025 AK Andrej Karpathy — holds since 2025-02-27 — tap for who they are Same subject: Pre-training is not the process by which humans learn; it sits somewhere between human learning and human evolution. — tap to centre the map on it Pre-training is not the process bywhich humans learn; it sitssomewhere between human learning andhuman evolution. Last stated 9 months ago 13 Feb 2026 DA Dario Amodei — holds since 2026-02-13 — tap for who they are Same subject: The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. — tap to centre the map on it The knowledge a model soaks up inpre-training is holding it back;what we actually want is theintelligence with the knowledgestripped out. Last stated a year ago 17 Oct 2025 AK Andrej Karpathy — holds since 2025-10-17 — tap for who they are Same subject: A model only days into training cannot be known to be aligned, and if such models can leap a generation ahead this fast then training pauses mean little and timelines are shorter than previously thought. — tap to centre the map on it A model only days into trainingcannot be known to be aligned, andif such models can leap a generationahead this fast then training pausesmean little and timelines areshorter than previously thought. Last stated 2 months ago 9 Sept 2026 ZM Zvi Mowshowitz — holds since 2026-09-09 — tap for who they are Same subject: Training data is still plentiful for now, but it will eventually run out. — tap to centre the map on it Training data is still plentiful fornow, but it will eventually run out. Last stated 4 years ago 27 Mar 2023 IS Ilya Sutskever — holds since 2023-03-27 — tap for who they are Same subject: ML researchers may be underrating how much pressure there will be to move to less safe training strategies once their advantage over safer ones becomes apparent. — tap to centre the map on it ML researchers may be underratinghow much pressure there will be tomove to less safe trainingstrategies once their advantage oversafer ones becomes apparent. Last stated 4 years ago 12 May 2023 AC Ajeya Cotra — holds since 2023-05-12 — tap for who they are Same subject: Pre-training is a crappy evolution: the practically buildable substitute for the process that gave animals their built-in hardware. — tap to centre the map on it Pre-training is a crappy evolution:the practically buildable substitutefor the process that gave animalstheir built-in hardware. Last stated a year ago 17 Oct 2025 AK Andrej Karpathy — holds since 2025-10-17 — tap for who they are Same subject: It may be more difficult to endure prolonged periods of underperformance while the rest of the world is performing well than it is to stay in stocks through a bear market. — tap to centre the map on it It may be more difficult to endureprolonged periods ofunderperformance while the rest ofthe world is performing well than itis to stay in stocks through a bearmarket. Last stated 10 years ago 2 Dec 2016 MB Michael Batnick — holds since 2016-12-02 — tap for who they are Same subject: A format reward should only be added when the answer is already correct. — tap to centre the map on it A format reward should only be addedwhen the answer is already correct. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards. — tap to centre the map on it A model should not be called areasoning model unless it has beentrained with reinforcement learningwith verifiable rewards. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel. — tap to centre the map on it A physical AI company needs threeAIs, not one — the agent, thesimulator and the critic — turningdeployment into a flywheel. Last stated 4 months ago 3 Aug 2026 DD Dmitri Dolgov — holds since 2026-08-03 — tap for who they are Same subject: 'Everyone gets it wrong sometimes' is a bad argument when assessing track records. — tap to centre the map on it 'Everyone gets it wrong sometimes'is a bad argument when assessingtrack records. Last stated 4 months ago 2 Aug 2026 AK Adam Kucharski — holds since 2026-08-02 — tap for who they are Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it A benchmark that ranks Claude Codelast while it stays first in use ismeasuring the wrong thing, and hasbeen for a year. Last stated 3 months ago 3 Sept 2026 DR Dax Raad — holds since 2026-09-03 — tap for who they are Same subject: A carmaker's claim to be the safest is mostly an artefact of comparing a new car against a fleet average twelve years old. — tap to centre the map on it A carmaker's claim to be the safestis mostly an artefact of comparing anew car against a fleet averagetwelve years old. Last stated 3 years ago 19 Dec 2023 PK Philip Koopman — holds since 2023-12-19 — tap for who they are Same subject: A ban on self-recursive improvement in AI models would create a big incentive for some other parts of the world to invest a lot more in compute. — tap to centre the map on it A ban on self-recursive improvementin AI models would create a bigincentive for some other parts ofthe world to invest a lot more incompute. Last stated 2 months ago 22 Sept 2026 TC Tyler Cowen — holds since 2026-09-22 — tap for who they are Same subject: A chip that can do nothing but low-precision dense linear algebra is enough for machine learning, even though it cannot run Chrome. — tap to centre the map on it A chip that can do nothing butlow-precision dense linear algebrais enough for machine learning, eventhough it cannot run Chrome. Last stated 4 months ago 30 Jul 2026 JD Jeff Dean — holds since 2026-07-30 — tap for who they are Same subject: A computer company has to live five to ten years in the future, because a system takes three years to build, two to ramp, and is then used for a decade. — tap to centre the map on it A computer company has to live fiveto ten years in the future, becausea system takes three years to build,two to ramp, and is then used for adecade. Last stated 4 months ago 26 Jul 2026 JH Jensen Huang — holds since 2026-07-26 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre It is not clear that pretraining has hit a wall, though it probably has relatively diminished returns. Last stated 8 Jul 2025 · a year ago Holds Ryan Greenblatt Read this korrent →