Tap a claim on the ring to put it at the centre.
← A teacher from the same model family likely distills better, because…
17 connected korrents · 9 moments from 18 Mar 2024 to 26 Oct 2026.
Everything filed under LLMs
LLMs
Everything filed under scaling laws
scaling laws
Everything filed under open source
open source
Everything filed under OpenAI
OpenAI
Everything filed under reinforcement learning
reinforcement learning
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: A teacher from the same model family likely distills better, because the student is more familiar with its prompt format and answer structure.
A teacher from the same model family likely distills better, because the student is more familiar with its prompt format and answer structure.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: Distillation carries a higher risk of overfitting, so it is not a free lunch. — tap to centre the map on it
Distillation carries a higher risk of overfitting, so it is not a free lunch.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: Distilling from a model's reasoning traces gets there quicker and cheaper than reinforcement learning, which is why labs that invest in RL complain about it. — tap to centre the map on it
Distilling from a model's reasoning traces gets there quicker and cheaper than reinforcement learning, which is why labs that invest in RL complain about it.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: On-policy distillation is probably a bit more accurate than off-policy, because it always uses the most up-to-date student model. — tap to centre the map on it
On-policy distillation is probably a bit more accurate than off-policy, because it always uses the most up-to-date student model.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: Process reward models, such as the additional grader in DeepSeekMath V2, are an interesting direction. — tap to centre the map on it
Process reward models, such as the additional grader in DeepSeekMath V2, are an interesting direction.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: Chinese labs more likely distill from US models because it is easier than getting more compute, not because it is decisive. — tap to centre the map on it
Chinese labs more likely distill from US models because it is easier than getting more compute, not because it is decisive.
Last stated 4 weeks ago
26 Oct 2026
NL
Nathan Lambert — holds since 2026-10-26 — tap for who they are
Same subject: AI Math Olympiad results may partly reflect teaching to the test. — tap to centre the map on it
AI Math Olympiad results may partly reflect teaching to the test.
Last stated a year ago
13 Aug 2025
NS
Nate Silver — holds since 2025-08-13 — tap for who they are
Same subject: Running a model five times and keeping the best answer buys a higher benchmark score without buying a better model. — tap to centre the map on it
Running a model five times and keeping the best answer buys a higher benchmark score without buying a better model.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: It is more common to train only on answer tokens than on prompt tokens. — tap to centre the map on it
It is more common to train only on answer tokens than on prompt tokens.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it
A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week.
Last stated a month ago
21 Oct 2026
PS
Peter Steinberger — holds since 2026-10-21 — tap for who they are
Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it
A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests.
Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it
Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers.
Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 12 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A CPU and memory crunch is coming. — tap to centre the map on it
A CPU and memory crunch is coming.
Last stated 4 weeks ago
26 Oct 2026
DM
Dillon Mulroy — holds since 2026-10-26 — tap for who they are
Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it
A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it
A codec does not really exist until everyone can decode it.
Last stated 6 months ago
31 May 2026
JK
Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are
Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it
A company that will not accept a project’s terms is free to go and use a more permissive project instead.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
A teacher from the same model family likely distills better, because the student is more familiar with its prompt format and answer structure.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Read this korrent →
Similar wording
Distillation carries a higher risk of overfitting, so it is not a free lunch.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
Distilling from a model's reasoning traces gets there quicker and cheaper than reinforcement learning, which is why labs that invest in RL complain about it.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
On-policy distillation is probably a bit more accurate than off-policy, because it always uses the most up-to-date student model.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
Process reward models, such as the additional grader in DeepSeekMath V2, are an interesting direction.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
Chinese labs more likely distill from US models because it is easier than getting more compute, not because it is decisive.
Last stated 26 Oct 2026 · 4 weeks ago
Holds Nathan Lambert
Similar wording
AI Math Olympiad results may partly reflect teaching to the test.
Last stated 13 Aug 2025 · a year ago
Holds Nate Silver
Similar wording
Running a model five times and keeping the best answer buys a higher benchmark score without buying a better model.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Similar wording
It is more common to train only on answer tokens than on prompt tokens.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Same subject: OpenAI
A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week.
Last stated 21 Oct 2026 · a month ago
Holds Peter Steinberger
Same subject: OpenAI
A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests.
Last stated 18 Mar 2024 · 3 years ago
Holds Sam Altman
Same subject: OpenAI
Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers.
Last stated 18 Mar 2024 · 3 years ago
Holds Sam Altman
Same subject: scaling laws
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 25 Nov 2025 · 12 months ago
Holds Ilya Sutskever
Same subject: scaling laws
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Same subject: scaling laws
A CPU and memory crunch is coming.
Last stated 26 Oct 2026 · 4 weeks ago
Holds Dillon Mulroy
Same subject: open source
A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither.
Last stated 26 Sept 2024 · 2 years ago
Holds Matt Mullenweg
Same subject: open source
A codec does not really exist until everyone can decode it.
Last stated 31 May 2026 · 6 months ago
Holds Jean-Baptiste Kempf
Same subject: open source
A company that will not accept a project’s terms is free to go and use a more permissive project instead.
Last stated 26 Sept 2024 · 2 years ago
Holds Matt Mullenweg