korrents

On the map

Tap a claim on the ring to put it at the centre.

← At the level of loss functions, language-model pre-training is…

17 connected korrents · 18 moments from 29 Sept 2018 to 5 Nov 2026. Nearly all of them are about scaling laws.

Everything filed under LLMs LLMs Everything filed under simplicity simplicity Everything filed under benchmarks benchmarks Everything filed under AI agents AI agents Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. At the level of loss functions,language-model pre-training is remarkablysimple, though running it at scale is acomplete engineering feat. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of itscompute on research rather than onbuilding the next model, becauseresearch is where the tenfold yearlyefficiency gains come from. Last stated 8 months ago 13 Mar 2026 DP Dylan Patel — holds since 2026-03-13 — tap for who they are Same subject: After pre-training, post-training and test-time scaling, the fourth scaling law is agentic: multiplying AI by spawning agents, and the whole loop scales on one thing, compute. — tap to centre the map on it After pre-training, post-trainingand test-time scaling, the fourthscaling law is agentic: multiplyingAI by spawning agents, and the wholeloop scales on one thing, compute. Last stated 8 months ago 23 Mar 2026 JH Jensen Huang — holds since 2026-03-23 — tap for who they are Same subject: AI scaling laws require exponentially more compute and resources to produce only linear gains in model intelligence. — tap to centre the map on it AI scaling laws requireexponentially more compute andresources to produce only lineargains in model intelligence. Last stated 2 months ago 19 Sept 2026 NL Nathan Lambert — holds since 2026-09-19 — tap for who they are Same subject: As models keep scaling up, modularity will become essential to how they are developed. — tap to centre the map on it As models keep scaling up,modularity will become essential tohow they are developed. Last stated 4 years ago 23 Feb 2023 SR Sebastian Ruder — holds since 2023-02-23 — tap for who they are Same subject: Before training a base model with RL to emit think tokens, it is better to do some continued pre-training or supervised fine-tuning with those tokens present. — tap to centre the map on it Before training a base model with RLto emit think tokens, it is betterto do some continued pre-training orsupervised fine-tuning with thosetokens present. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: Bet on a system that is maximally learned and minimally constrained, and add structure only where it improves the scaling laws. — tap to centre the map on it Bet on a system that is maximallylearned and minimally constrained,and add structure only where itimproves the scaling laws. Last stated 4 months ago 3 Aug 2026 DD Dmitri Dolgov — holds since 2026-08-03 — tap for who they are Same subject: Current AI techniques are four to six orders of magnitude away from optimal in data efficiency and test-time compute efficiency. — tap to centre the map on it Current AI techniques are four tosix orders of magnitude away fromoptimal in data efficiency andtest-time compute efficiency. Last stated 4 months ago 7 Aug 2026 FC François Chollet — holds since 2026-08-07 — tap for who they are Same subject: In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset. — tap to centre the map on it In instruction tuning, a smaller setof high-quality training examplesoutperforms a larger, noisierdataset. Last stated 3 years ago 15 Nov 2023 SR Sebastian Ruder — holds since 2023-11-15 — tap for who they are Same subject: LLMs cannot be expected to undertake the significant task of managing complexity of their own volition because they do not operate under constraints of time or load. — tap to centre the map on it LLMs cannot be expected to undertakethe significant task of managingcomplexity of their own volitionbecause they do not operate underconstraints of time or load. Last stated 7 months ago 12 Apr 2026 BC Bryan Cantrill — holds since 2026-04-12 — tap for who they are Same subject: People who would replace books with AI summaries miss that a book is itself the optimal, highly condensed summary. — tap to centre the map on it People who would replace books withAI summaries miss that a book isitself the optimal, highly condensedsummary. Last stated 2 weeks ago 5 Nov 2026 CN Cal Newport — holds since 2026-11-05 — tap for who they are Same subject: There is unmet demand for a single, well-integrated local AI model experience, rather than a patchwork of local AI components. — tap to centre the map on it There is unmet demand for a single,well-integrated local AI modelexperience, rather than a patchworkof local AI components. Last stated 6 months ago 14 May 2026 SS Salvatore Sanfilippo — holds since 2026-05-14 — tap for who they are Same subject: A browser works better as a single window than as multiple separate windows. — tap to centre the map on it A browser works better as a singlewindow than as multiple separatewindows. Last stated 2 months ago 15 Sept 2026 CC Chris Coyier — holds since 2026-09-15 — tap for who they are Same subject: A business is more likely to fail from taking on too much complexity than from lacking resources. — tap to centre the map on it A business is more likely to failfrom taking on too much complexitythan from lacking resources. Last stated 8 years ago 29 Sept 2018 TG Tren Griffin — holds since 2018-09-29 — tap for who they are Same subject: A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data. — tap to centre the map on it A model claiming to be built byOpenAI is not necessarily evidencethat it was trained on OpenAI data. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact. — tap to centre the map on it A tiny language model inventing aplausible-sounding name is the samephenomenon as a large oneconfidently stating a false fact. Last stated 9 months ago 12 Feb 2026 AK Andrej Karpathy — holds since 2026-02-12 — tap for who they are Same subject: AI has lowered the barrier to understanding, since asking it is easier than doing the research yourself. — tap to centre the map on it AI has lowered the barrier tounderstanding, since asking it iseasier than doing the researchyourself. Last stated 2 years ago 27 Jan 2025 JH Jensen Huang — holds since 2025-01-27 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. Last stated 16 Jul 2026 · 4 months ago Holds Grant Sanderson Read this korrent →