Tap a claim on the ring to put it at the centre.
← What is in a model's weights is a hazy recollection of things it read…
17 connected korrents · 14 moments from 18 Mar 2024 to 21 Oct 2026.
Everything filed under LLMs
LLMs
Everything filed under scaling laws
scaling laws
Everything filed under AI writing
AI writing
Everything filed under OpenAI
OpenAI Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: What is in a model's weights is a hazy recollection of things it read a year ago; what is in its context window is working memory. What is in a model's weights is a hazy recollection of things it read a year ago; what is in its context window is working memory. Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: What sits in a model's context is far clearer to it than its training data, which is trillions of tokens stirred into a soup of parameters. — tap to centre the map on it What sits in a model's context is far clearer to it than its training data, which is trillions of tokens stirred into a soup of parameters. Last stated 3 months ago
30 Jul 2026
JD
Jeff Dean — holds since 2026-07-30 — tap for who they are
Same subject: A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. — tap to centre the map on it A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. Last stated 3 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: Language models replay early-2000s internet culture patterns because their training data overweights content from that era. — tap to centre the map on it Language models replay early-2000s internet culture patterns because their training data overweights content from that era. Last stated a year ago
30 Sept 2025
HR
Harper Reed — holds since 2025-09-30 — tap for who they are
Same subject: Extra thinking time cannot help with factual recall: a model that does not know a date will not know it after a week either. — tap to centre the map on it Extra thinking time cannot help with factual recall: a model that does not know a date will not know it after a week either. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore. — tap to centre the map on it Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore. Last stated 3 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: Every announcement of a longer context window hides the cost, which is that the model gets worse at following instructions. — tap to centre the map on it Every announcement of a longer context window hides the cost, which is that the model gets worse at following instructions. Last stated 2 years ago
19 Jun 2024
AS
Aravind Srinivas — holds since 2024-06-19 — tap for who they are
Same subject: Language models have no equivalent of sleep: nothing takes the day's context, thinks it over, and distils it back into the weights. — tap to centre the map on it Language models have no equivalent of sleep: nothing takes the day's context, thinks it over, and distils it back into the weights. Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: A model that solves a hard problem has learned nothing from it: the next session has forgotten it, with no new skill to carry to related problems. — tap to centre the map on it A model that solves a hard problem has learned nothing from it: the next session has forgotten it, with no new skill to carry to related problems. Last stated 7 months ago
20 Mar 2026
TT
Terence Tao — holds since 2026-03-20 — tap for who they are
Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. Last stated 2 days ago
21 Oct 2026
PS
Peter Steinberger — holds since 2026-10-21 — tap for who they are
Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 7 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
Same subject: A CEO who sends out an AI-written strategy memo is modelling that it is fine to outsource thinking and strategy. — tap to centre the map on it A CEO who sends out an AI-written strategy memo is modelling that it is fine to outsource thinking and strategy. Last stated 4 weeks ago
27 Sept 2026
MG
Molly Graham — holds since 2026-09-27 — tap for who they are
Same subject: A stream of AI-written papers with any error rate at all becomes insufferable, because finding the error costs more than the paper is worth even at ninety-nine percent. — tap to centre the map on it A stream of AI-written papers with any error rate at all becomes insufferable, because finding the error costs more than the paper is worth even at ninety-nine percent. Last stated 4 months ago
30 Jun 2026
GS
Grant Sanderson — holds since 2026-06-30 — tap for who they are
Same subject: AI can now produce in minutes work that used to take weeks to build. — tap to centre the map on it AI can now produce in minutes work that used to take weeks to build. Last stated 5 months ago
26 May 2026
CB
Carlos Alexandro Becker — holds since 2026-05-26 — tap for who they are
At the centre
What is in a model's weights is a hazy recollection of things it read a year ago; what is in its context window is working memory.
Holds Read this korrent →
What sits in a model's context is far clearer to it than its training data, which is trillions of tokens stirred into a soup of parameters.
Holds
A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to.
Holds
Language models replay early-2000s internet culture patterns because their training data overweights content from that era.
Holds
Extra thinking time cannot help with factual recall: a model that does not know a date will not know it after a week either.
Holds
Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore.
Holds
Every announcement of a longer context window hides the cost, which is that the model gets worse at following instructions.
Holds
Language models have no equivalent of sleep: nothing takes the day's context, thinks it over, and distils it back into the weights.
Holds
A model that solves a hard problem has learned nothing from it: the next session has forgotten it, with no new skill to carry to related problems.
Holds
A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week.
Holds
A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests.
Holds
Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers.
Holds
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Holds
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds
A CEO who sends out an AI-written strategy memo is modelling that it is fine to outsource thinking and strategy.
Holds
A stream of AI-written papers with any error rate at all becomes insufferable, because finding the error costs more than the paper is worth even at ninety-nine percent.
Holds
AI can now produce in minutes work that used to take weeks to build.
Holds