korrents

On the map

Tap a claim on the ring to put it at the centre.

← Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often.

17 connected korrents · 16 moments from 11 Oct 2018 to 2 Nov 2026. Nearly all of them are about scaling laws.

Everything filed under LLMs LLMs Everything filed under Anthropic Anthropic Everything filed under open source open source Everything filed under OpenAI OpenAI Everything filed under benchmarks benchmarks Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often. Language models should be thought of assomewhat out of date, because pre-trainingis too expensive to repeat often. Last stated 2 years ago 27 Feb 2025 AK Andrej Karpathy — holds since 2025-02-27 — tap for who they are Same subject: Current techniques restrict pre-trained representation power because standard language models are unidirectional. — tap to centre the map on it Current techniques restrictpre-trained representation powerbecause standard language models areunidirectional. Last stated 8 years ago 11 Oct 2018 KT Kristina Toutanova — holds since 2018-10-11 — tap for who they are MC Ming-Wei Chang — holds since 2018-10-11 — tap for who they are KL Kenton Lee — holds since 2018-10-11 — tap for who they are JD Jacob Devlin — holds since 2018-10-11 — tap for who they are Same subject: In reinforcement learning for LLMs, the loss is not as informative as in classic deep learning or pre-training. — tap to centre the map on it In reinforcement learning for LLMs,the loss is not as informative as inclassic deep learning orpre-training. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: Language models perform better on benchmark problems released before their training data cutoff, indicating data contamination inflates scores. — tap to centre the map on it Language models perform better onbenchmark problems released beforetheir training data cutoff,indicating data contaminationinflates scores. Last stated 3 years ago 13 May 2024 SR Sebastian Ruder — holds since 2024-05-13 — tap for who they are Same subject: Large language models could still plateau, and that possibility should be held open even though no evidence of it has appeared. — tap to centre the map on it Large language models could stillplateau, and that possibility shouldbe held open even though no evidenceof it has appeared. Last stated 3 months ago 26 Aug 2026 DH David Heinemeier Hansson — holds since 2026-08-26 — tap for who they are Same subject: Reasoning models are an upgrade from conventional LLMs achieved by adding post-training via reinforcement learning and optionally inference scaling. — tap to centre the map on it Reasoning models are an upgrade fromconventional LLMs achieved by addingpost-training via reinforcementlearning and optionally inferencescaling. Last stated 4 weeks ago 24 Oct 2026 SR Sebastian Raschka — holds since 2026-10-24 — tap for who they are Same subject: The LLM line of research will reach a capability plateau. — tap to centre the map on it The LLM line of research will reacha capability plateau. Last stated 4 months ago 7 Aug 2026 FC François Chollet — no longer holds since 2026-08-07 — tap for who they are GM Gary Marcus — holds since 2025-06-07 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of itscompute on research rather than onbuilding the next model, becauseresearch is where the tenfold yearlyefficiency gains come from. Last stated 8 months ago 13 Mar 2026 DP Dylan Patel — holds since 2026-03-13 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as anonprofit and bolt a for-profit armon later, whatever OpenAI's ownhistory suggests. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phasefor the internet but a momentaryindustry, and an AI people pay foris better because they know theanswers are not influenced byadvertisers. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it A business making hundreds ofmillions from an open source projectowes it money or contribution, andcan be refused its trademarks if itgives neither. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it A codec does not really exist untileveryone can decode it. Last stated 6 months ago 31 May 2026 JK Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it A company that will not accept aproject’s terms is free to go anduse a more permissive projectinstead. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: "I don't know" becomes a kind of superpower when you can work with Claude Code to figure things out. — tap to centre the map on it "I don't know" becomes a kind ofsuperpower when you can work withClaude Code to figure things out. Last stated 8 months ago 19 Mar 2026 KP Kris Puckett — holds since 2026-03-19 — tap for who they are Same subject: A large share of Anthropic's revenue being consumption-based rather than subscriptions raises the question of how resilient that revenue is. — tap to centre the map on it A large share of Anthropic's revenuebeing consumption-based rather thansubscriptions raises the question ofhow resilient that revenue is. Last stated 3 weeks ago 2 Nov 2026 AA Azeem Azhar — holds since 2026-11-02 — tap for who they are Same subject: A skill is a set of repeatable, specific instructions that augments what Claude Code can do. — tap to centre the map on it A skill is a set of repeatable,specific instructions that augmentswhat Claude Code can do. Last stated 8 months ago 19 Mar 2026 KP Kris Puckett — holds since 2026-03-19 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they arefaded, dashed ring: they no longer hold it — they changed their mind

At the centre Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often. Last stated 27 Feb 2025 · 2 years ago Holds Andrej Karpathy Read this korrent →