korrents

On the map

Tap a claim on the ring to put it at the centre.

← The best distilled models are not as good as state-of-the-art…

17 connected korrents · 10 moments from 27 Mar 2023 to 26 Oct 2026.

Everything filed under LLMs LLMs Everything filed under open source open source Everything filed under OpenAI OpenAI Everything filed under reinforcement learning reinforcement learning Everything filed under scaling laws scaling laws Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: The best distilled models are not as good as state-of-the-art reasoning models but are maybe 80 to 90 percent of the way there. The best distilled models are not as goodas state-of-the-art reasoning models butare maybe 80 to 90 percent of the waythere. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: Distilling from a model's reasoning traces gets there quicker and cheaper than reinforcement learning, which is why labs that invest in RL complain about it. — tap to centre the map on it Distilling from a model's reasoningtraces gets there quicker andcheaper than reinforcement learning,which is why labs that invest in RLcomplain about it. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: If reinforcement learning is too expensive, model distillation can achieve quite good results more cheaply, though maybe not quite as good. — tap to centre the map on it If reinforcement learning is tooexpensive, model distillation canachieve quite good results morecheaply, though maybe not quite asgood. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: Language models are bad at multi-step reasoning in their heads but quite good at it when allowed to think out loud. — tap to centre the map on it Language models are bad atmulti-step reasoning in their headsbut quite good at it when allowed tothink out loud. Last stated 4 years ago 27 Mar 2023 IS Ilya Sutskever — holds since 2023-03-27 — tap for who they are Same subject: Use RLVR to train large state-of-the-art models, but for a smaller model distilling from a bigger one can give better results than RLVR from scratch. — tap to centre the map on it Use RLVR to train largestate-of-the-art models, but for asmaller model distilling from abigger one can give better resultsthan RLVR from scratch. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: Choosing which model to use for a task matters, and high reasoning effort may be too much for some work. — tap to centre the map on it Choosing which model to use for atask matters, and high reasoningeffort may be too much for somework. Last stated a year ago 25 Aug 2025 LR Lee Robinson — holds since 2025-08-25 — tap for who they are Same subject: On-policy distillation is probably a bit more accurate than off-policy, because it always uses the most up-to-date student model. — tap to centre the map on it On-policy distillation is probably abit more accurate than off-policy,because it always uses the mostup-to-date student model. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: Most reasoning models are still trained on code and math, and it would be interesting to go beyond those domains. — tap to centre the map on it Most reasoning models are stilltrained on code and math, and itwould be interesting to go beyondthose domains. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: Whether soft or hard distillation works better is an empirical question, but both are fine approaches. — tap to centre the map on it Whether soft or hard distillationworks better is an empiricalquestion, but both are fineapproaches. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as anonprofit and bolt a for-profit armon later, whatever OpenAI's ownhistory suggests. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phasefor the internet but a momentaryindustry, and an AI people pay foris better because they know theanswers are not influenced byadvertisers. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because itwas one word: naming a researchdirection is what tells a wholefield what to do next. Last stated 12 months ago 25 Nov 2025 IS Ilya Sutskever — holds since 2025-11-25 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A CPU and memory crunch is coming. — tap to centre the map on it A CPU and memory crunch is coming. Last stated 4 weeks ago 26 Oct 2026 DM Dillon Mulroy — holds since 2026-10-26 — tap for who they are Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it A business making hundreds ofmillions from an open source projectowes it money or contribution, andcan be refused its trademarks if itgives neither. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it A codec does not really exist untileveryone can decode it. Last stated 6 months ago 31 May 2026 JK Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it A company that will not accept aproject’s terms is free to go anduse a more permissive projectinstead. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre The best distilled models are not as good as state-of-the-art reasoning models but are maybe 80 to 90 percent of the way there. Last stated 17 Oct 2026 · a month ago Holds Sebastian Raschka Read this korrent →