korrents

On the map

Tap a claim on the ring to put it at the centre.

← Scaling laws are empirical regularities, not laws of the universe, and…

17 connected korrents · 17 moments from 10 Dec 2015 to 10 Oct 2026.

Everything filed under scaling laws scaling laws Everything filed under benchmarks benchmarks Everything filed under reinforcement learning reinforcement learning Everything filed under neural networks neural networks Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Scaling laws are empirical regularities, not laws of the universe, and they will probably keep holding, though that is not certain. Scaling laws are empirical regularities,not laws of the universe, and they willprobably keep holding, though that is notcertain. Last stated 2 years ago 11 Nov 2024 DA Dario Amodei — holds since 2024-11-11 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of itscompute on research rather than onbuilding the next model, becauseresearch is where the tenfold yearlyefficiency gains come from. Last stated 8 months ago 13 Mar 2026 DP Dylan Patel — holds since 2026-03-13 — tap for who they are Same subject: After pre-training, post-training and test-time scaling, the fourth scaling law is agentic: multiplying AI by spawning agents, and the whole loop scales on one thing, compute. — tap to centre the map on it After pre-training, post-trainingand test-time scaling, the fourthscaling law is agentic: multiplyingAI by spawning agents, and the wholeloop scales on one thing, compute. Last stated 8 months ago 23 Mar 2026 JH Jensen Huang — holds since 2026-03-23 — tap for who they are Same subject: AI scaling laws require exponentially more compute and resources to produce only linear gains in model intelligence. — tap to centre the map on it AI scaling laws requireexponentially more compute andresources to produce only lineargains in model intelligence. Last stated 2 months ago 19 Sept 2026 NL Nathan Lambert — holds since 2026-09-19 — tap for who they are Same subject: As models keep scaling up, modularity will become essential to how they are developed. — tap to centre the map on it As models keep scaling up,modularity will become essential tohow they are developed. Last stated 4 years ago 23 Feb 2023 SR Sebastian Ruder — holds since 2023-02-23 — tap for who they are Same subject: At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. — tap to centre the map on it At the level of loss functions,language-model pre-training isremarkably simple, though running itat scale is a complete engineeringfeat. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: Before training a base model with RL to emit think tokens, it is better to do some continued pre-training or supervised fine-tuning with those tokens present. — tap to centre the map on it Before training a base model with RLto emit think tokens, it is betterto do some continued pre-training orsupervised fine-tuning with thosetokens present. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: Bet on a system that is maximally learned and minimally constrained, and add structure only where it improves the scaling laws. — tap to centre the map on it Bet on a system that is maximallylearned and minimally constrained,and add structure only where itimproves the scaling laws. Last stated 4 months ago 3 Aug 2026 DD Dmitri Dolgov — holds since 2026-08-03 — tap for who they are Same subject: A feedback loop that reinforces a behavior pulls it toward whatever improves the loop's own score. — tap to centre the map on it A feedback loop that reinforces abehavior pulls it toward whateverimproves the loop's own score. Last stated 3 months ago 26 Aug 2026 HK Henrik Karlsson — holds since 2026-08-26 — tap for who they are Same subject: A format reward should only be added when the answer is already correct. — tap to centre the map on it A format reward should only be addedwhen the answer is already correct. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards. — tap to centre the map on it A model should not be called areasoning model unless it has beentrained with reinforcement learningwith verifiable rewards. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: A curve, which is what a deep learning model is, is a very bad substrate for discrete computation. — tap to centre the map on it A curve, which is what a deeplearning model is, is a very badsubstrate for discrete computation. Last stated 2 years ago 6 Nov 2024 FC François Chollet — holds since 2024-11-06 — tap for who they are Same subject: A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. — tap to centre the map on it A layer should learn a residual withreference to its own input ratherthan an unreferenced function, whichis what makes great depth trainable. Last stated 11 years ago 10 Dec 2015 JS Jian Sun — holds since 2015-12-10 — tap for who they are KH Kaiming He — holds since 2015-12-10 — tap for who they are Same subject: A project-level agent memory file works like a neural net, steering the agent much as a fine-tuned LoRA steers a model. — tap to centre the map on it A project-level agent memory fileworks like a neural net, steeringthe agent much as a fine-tuned LoRAsteers a model. Last stated 3 months ago 23 Aug 2026 KC Kun Chen — holds since 2026-08-23 — tap for who they are Same subject: 'Everyone gets it wrong sometimes' is a bad argument when assessing track records. — tap to centre the map on it 'Everyone gets it wrong sometimes'is a bad argument when assessingtrack records. Last stated 4 months ago 2 Aug 2026 AK Adam Kucharski — holds since 2026-08-02 — tap for who they are Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it A benchmark that ranks Claude Codelast while it stays first in use ismeasuring the wrong thing, and hasbeen for a year. Last stated 3 months ago 3 Sept 2026 DR Dax Raad — holds since 2026-09-03 — tap for who they are Same subject: A carmaker's claim to be the safest is mostly an artefact of comparing a new car against a fleet average twelve years old. — tap to centre the map on it A carmaker's claim to be the safestis mostly an artefact of comparing anew car against a fleet averagetwelve years old. Last stated 3 years ago 19 Dec 2023 PK Philip Koopman — holds since 2023-12-19 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Scaling laws are empirical regularities, not laws of the universe, and they will probably keep holding, though that is not certain. Last stated 11 Nov 2024 · 2 years ago Holds Dario Amodei Read this korrent →