korrents

On the map

Tap a claim on the ring to put it at the centre.

← Scaling up next-token prediction based on web text data is hitting a wall.

17 connected korrents · 15 moments from 18 Dec 2020 to 21 Nov 2026. Nearly all of them are about LLMs.

Everything filed under scaling laws scaling laws Everything filed under open source open source Everything filed under OpenAI OpenAI Everything filed under reinforcement learning reinforcement learning Everything filed under transformers transformers Everything filed under AI alignment AI alignment Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Scaling up next-token prediction based on web text data is hitting a wall. Scaling up next-token prediction based onweb text data is hitting a wall. Last stated 2 days ago 20 Nov 2026 FC François Chollet — holds since 2026-11-20 — tap for who they are Same subject: Available online text data is becoming a limiting factor for AI training, and repeating it yields diminishing returns. — tap to centre the map on it Available online text data isbecoming a limiting factor for AItraining, and repeating it yieldsdiminishing returns. Last stated 3 years ago 19 Dec 2023 SR Sebastian Ruder — holds since 2023-12-19 — tap for who they are Same subject: Until AI learns from something richer than text, its limited data will leave it struggling with the richness of language. — tap to centre the map on it Until AI learns from somethingricher than text, its limited datawill leave it struggling with therichness of language. Last stated 6 years ago 18 Dec 2020 MS Marcus du Sautoy — holds since 2020-12-18 — tap for who they are Same subject: The current AI stack is not among the top 50% most efficient approaches to learning. — tap to centre the map on it The current AI stack is not amongthe top 50% most efficientapproaches to learning. Last stated yesterday 21 Nov 2026 FC François Chollet — holds since 2026-11-21 — tap for who they are Same subject: Large language models could still plateau, and that possibility should be held open even though no evidence of it has appeared. — tap to centre the map on it Large language models could stillplateau, and that possibility shouldbe held open even though no evidenceof it has appeared. Last stated 3 months ago 26 Aug 2026 DH David Heinemeier Hansson — holds since 2026-08-26 — tap for who they are Same subject: As models keep growing, AI is running out of enough high-quality unique training tokens to keep up. — tap to centre the map on it As models keep growing, AI isrunning out of enough high-qualityunique training tokens to keep up. Last stated 5 months ago 24 Jun 2026 LW Lilian Weng — holds since 2026-06-24 — tap for who they are Same subject: The next-token language modeling objective is misaligned with following user instructions helpfully and safely. — tap to centre the map on it The next-token language modelingobjective is misaligned withfollowing user instructionshelpfully and safely. Last stated 5 years ago 4 Mar 2022 JL Jan Leike — holds since 2022-03-04 — tap for who they are Same subject: At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. — tap to centre the map on it At the level of loss functions,language-model pre-training isremarkably simple, though running itat scale is a complete engineeringfeat. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: By this time next year there will be significantly more tokens flowing, making token efficiency very important. — tap to centre the map on it By this time next year there will besignificantly more tokens flowing,making token efficiency veryimportant. Last stated a month ago 11 Oct 2026 LR Lee Robinson — holds since 2026-10-11 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as anonprofit and bolt a for-profit armon later, whatever OpenAI's ownhistory suggests. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phasefor the internet but a momentaryindustry, and an AI people pay foris better because they know theanswers are not influenced byadvertisers. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it A business making hundreds ofmillions from an open source projectowes it money or contribution, andcan be refused its trademarks if itgives neither. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it A codec does not really exist untileveryone can decode it. Last stated 6 months ago 31 May 2026 JK Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it A company that will not accept aproject’s terms is free to go anduse a more permissive projectinstead. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A feedback loop that reinforces a behavior pulls it toward whatever improves the loop's own score. — tap to centre the map on it A feedback loop that reinforces abehavior pulls it toward whateverimproves the loop's own score. Last stated 3 months ago 26 Aug 2026 HK Henrik Karlsson — holds since 2026-08-26 — tap for who they are Same subject: A format reward should only be added when the answer is already correct. — tap to centre the map on it A format reward should only be addedwhen the answer is already correct. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards. — tap to centre the map on it A model should not be called areasoning model unless it has beentrained with reinforcement learningwith verifiable rewards. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Scaling up next-token prediction based on web text data is hitting a wall. Last stated 20 Nov 2026 · 2 days ago Holds François Chollet Read this korrent →