Tap a claim on the ring to put it at the centre.
← It is more common to train only on answer tokens than on prompt tokens.
17 connected korrents · 14 moments from 13 Oct 2020 to 21 Nov 2026.
Everything filed under LLMs
LLMs
Everything filed under scaling laws
scaling laws
Everything filed under open source
open source
Everything filed under OpenAI
OpenAI
Everything filed under reinforcement learning
reinforcement learning
Everything filed under transformers
transformers
Everything filed under code generation
code generation
Everything filed under curiosity
curiosity
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: It is more common to train only on answer tokens than on prompt tokens.
It is more common to train only on answer tokens than on prompt tokens.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: A teacher from the same model family likely distills better, because the student is more familiar with its prompt format and answer structure. — tap to centre the map on it
A teacher from the same model family likely distills better, because the student is more familiar with its prompt format and answer structure.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: A format reward should only be added when the answer is already correct. — tap to centre the map on it
A format reward should only be added when the answer is already correct.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: Before training a base model with RL to emit think tokens, it is better to do some continued pre-training or supervised fine-tuning with those tokens present. — tap to centre the map on it
Before training a base model with RL to emit think tokens, it is better to do some continued pre-training or supervised fine-tuning with those tokens present.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: Tokens are not the true unit of AI inference, because better models yield more problem-solving per token. — tap to centre the map on it
Tokens are not the true unit of AI inference, because better models yield more problem-solving per token.
Last stated 2 months ago
15 Sept 2026
PG
Paul Graham — holds since 2026-09-15 — tap for who they are
Same subject: Comparing price-per-token does not necessarily make sense because reasoning tokens mean that different models might use dramatically different numbers of tokens to solve the same problem. — tap to centre the map on it
Comparing price-per-token does not necessarily make sense because reasoning tokens mean that different models might use dramatically different numbers of tokens to solve the same problem.
Last stated 2 weeks ago
10 Nov 2026
SW
Simon Willison — holds since 2026-11-10 — tap for who they are
Same subject: The current AI stack is not among the top 50% most efficient approaches to learning. — tap to centre the map on it
The current AI stack is not among the top 50% most efficient approaches to learning.
Last stated yesterday
21 Nov 2026
FC
François Chollet — holds since 2026-11-21 — tap for who they are
Same subject: The case for sweating every line of code was premised on humans doing the modifications, and it now survives only because tokens are scarce. — tap to centre the map on it
The case for sweating every line of code was premised on humans doing the modifications, and it now survives only because tokens are scarce.
Last stated 3 months ago
26 Aug 2026
DH
David Heinemeier Hansson — holds since 2026-08-26 — tap for who they are
Same subject: Obvious questions to CEOs draw practised answers, so pattern recognition built on them is brittle and superficial. — tap to centre the map on it
Obvious questions to CEOs draw practised answers, so pattern recognition built on them is brittle and superficial.
Last stated 6 years ago
13 Oct 2020
CP
Chamath Palihapitiya — holds since 2020-10-13 — tap for who they are
Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it
A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week.
Last stated a month ago
21 Oct 2026
PS
Peter Steinberger — holds since 2026-10-21 — tap for who they are
Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it
A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests.
Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it
Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers.
Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 12 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A CPU and memory crunch is coming. — tap to centre the map on it
A CPU and memory crunch is coming.
Last stated 4 weeks ago
26 Oct 2026
DM
Dillon Mulroy — holds since 2026-10-26 — tap for who they are
Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it
A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it
A codec does not really exist until everyone can decode it.
Last stated 6 months ago
31 May 2026
JK
Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are
Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it
A company that will not accept a project’s terms is free to go and use a more permissive project instead.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are