Tap a claim on the ring to put it at the centre.
← What is in a model's weights is a hazy recollection of things it read…
7 connected korrents · 6 moments on record from 19 Jun 2024 to 30 Jul 2026.
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: What is in a model's weights is a hazy recollection of things it read a year ago; what is in its context window is working memory. What is in a model's weights is a hazy recollection of things it read a year ago; what is in its context window is working memory. Last stated 11 months ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: What sits in a model's context is far clearer to it than its training data, which is trillions of tokens stirred into a soup of parameters. — tap to centre the map on it What sits in a model's context is far clearer to it than its training data, which is trillions of tokens stirred into a soup of parameters. Last stated a month ago
30 Jul 2026
JD
Jeff Dean — holds since 2026-07-30 — tap for who they are
Same subject: A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. — tap to centre the map on it A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. Last stated 2 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: Extra thinking time cannot help with factual recall: a model that does not know a date will not know it after a week either. — tap to centre the map on it Extra thinking time cannot help with factual recall: a model that does not know a date will not know it after a week either. Last stated 2 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore. — tap to centre the map on it Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore. Last stated 2 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: Every announcement of a longer context window hides the cost, which is that the model gets worse at following instructions. — tap to centre the map on it Every announcement of a longer context window hides the cost, which is that the model gets worse at following instructions. Last stated 2 years ago
19 Jun 2024
AS
Aravind Srinivas — holds since 2024-06-19 — tap for who they are
Same subject: Language models have no equivalent of sleep: nothing takes the day's context, thinks it over, and distils it back into the weights. — tap to centre the map on it Language models have no equivalent of sleep: nothing takes the day's context, thinks it over, and distils it back into the weights. Last stated 11 months ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: A model that solves a hard problem has learned nothing from it: the next session has forgotten it, with no new skill to carry to related problems. — tap to centre the map on it A model that solves a hard problem has learned nothing from it: the next session has forgotten it, with no new skill to carry to related problems. Last stated 6 months ago
20 Mar 2026
TT
Terence Tao — holds since 2026-03-20 — tap for who they are