korrents

On the map

Tap a claim on the ring to put it at the centre.

← In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset.

17 connected korrents · 15 moments from 29 Sept 2018 to 5 Nov 2026. Nearly all of them are about LLMs.

Everything filed under simplicity simplicity Everything filed under OpenAI OpenAI Everything filed under scaling laws scaling laws Everything filed under reinforcement learning reinforcement learning Everything filed under AI alignment AI alignment Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset. In instruction tuning, a smaller set ofhigh-quality training examples outperformsa larger, noisier dataset. Last stated 3 years ago 15 Nov 2023 SR Sebastian Ruder — holds since 2023-11-15 — tap for who they are Same subject: The argument that RAG and long context make fine-tuning unnecessary holds when information is missing, but not when the model struggles with the task itself. — tap to centre the map on it The argument that RAG and longcontext make fine-tuning unnecessaryholds when information is missing,but not when the model struggleswith the task itself. Last stated 2 weeks ago 5 Nov 2026 AR Aishwarya Naresh Reganti — holds since 2026-11-05 — tap for who they are Same subject: When pre-training data is limited, it is better to train a smaller model for multiple epochs than to train a larger model on unique data once. — tap to centre the map on it When pre-training data is limited,it is better to train a smallermodel for multiple epochs than totrain a larger model on unique dataonce. Last stated 3 years ago 1 Dec 2023 SR Sebastian Ruder — holds since 2023-12-01 — tap for who they are Same subject: Reasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own. — tap to centre the map on it Reasoning will generalize the wayinstruction tuning did: add enoughverifiable domains and, at somepoint nobody can yet locate, therest start working on their own. Last stated 2 years ago 3 Feb 2025 NL Nathan Lambert — holds since 2025-02-03 — tap for who they are Same subject: Given good data, room to operate and sound numerical conditioning, models will learn on their own. — tap to centre the map on it Given good data, room to operate andsound numerical conditioning, modelswill learn on their own. Last stated 3 years ago 8 Aug 2023 DA Dario Amodei — holds since 2023-08-08 — tap for who they are Same subject: Instability in small-scale RL training does not necessarily mean the algorithm is wrong; the scale may be too small to see its benefits. — tap to centre the map on it Instability in small-scale RLtraining does not necessarily meanthe algorithm is wrong; the scalemay be too small to see itsbenefits. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: A large context window does not replace fine-tuning, because fine-tuning serves the separate purpose of instilling preferences and demonstrating a task. — tap to centre the map on it A large context window does notreplace fine-tuning, becausefine-tuning serves the separatepurpose of instilling preferencesand demonstrating a task. Last stated 3 years ago 22 Mar 2024 EG Elad Gil — holds since 2024-03-22 — tap for who they are Same subject: Fine-tuning-based alignment does not remove a model's worrying knowledge and abilities; it only teaches the model not to output them. — tap to centre the map on it Fine-tuning-based alignment does notremove a model's worrying knowledgeand abilities; it only teaches themodel not to output them. Last stated 3 years ago 8 Aug 2023 DA Dario Amodei — holds since 2023-08-08 — tap for who they are Same subject: Models fine-tuned to imitate another model's outputs copy its style without acquiring its factual accuracy. — tap to centre the map on it Models fine-tuned to imitate anothermodel's outputs copy its stylewithout acquiring its factualaccuracy. Last stated 3 years ago 15 Nov 2023 SR Sebastian Ruder — holds since 2023-11-15 — tap for who they are Same subject: A browser works better as a single window than as multiple separate windows. — tap to centre the map on it A browser works better as a singlewindow than as multiple separatewindows. Last stated 2 months ago 15 Sept 2026 CC Chris Coyier — holds since 2026-09-15 — tap for who they are Same subject: A business is more likely to fail from taking on too much complexity than from lacking resources. — tap to centre the map on it A business is more likely to failfrom taking on too much complexitythan from lacking resources. Last stated 8 years ago 29 Sept 2018 TG Tren Griffin — holds since 2018-09-29 — tap for who they are Same subject: A business succeeds through a small number of highly effective distribution channels, not many mediocre ones. — tap to centre the map on it A business succeeds through a smallnumber of highly effectivedistribution channels, not manymediocre ones. Last stated 3 years ago 17 Feb 2024 AW Adam Wiggins — holds since 2024-02-17 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as anonprofit and bolt a for-profit armon later, whatever OpenAI's ownhistory suggests. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phasefor the internet but a momentaryindustry, and an AI people pay foris better because they know theanswers are not influenced byadvertisers. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because itwas one word: naming a researchdirection is what tells a wholefield what to do next. Last stated 12 months ago 25 Nov 2025 IS Ilya Sutskever — holds since 2025-11-25 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A CPU and memory crunch is coming. — tap to centre the map on it A CPU and memory crunch is coming. Last stated 4 weeks ago 26 Oct 2026 DM Dillon Mulroy — holds since 2026-10-26 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset. Last stated 15 Nov 2023 · 3 years ago Holds Sebastian Ruder Read this korrent →