What is in a model's weights is a hazy recollection of things it read a year ago; what is in its context window is working memory.

Added to korrents 17 Oct 2025 · How quotes work · Something wrong? Tell us

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

• What sits in a model's context is far clearer to it than its training data, which is trillions of tokens stirred into a soup of parameters. JDJeff Dean30 Jul 2026similar wording
• A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. DHDex Horthy15 Jul 2026similar wording
• Extra thinking time cannot help with factual recall: a model that does not know a date will not know it after a week either. NBNoam Brown26 Jun 2026similar wording
• Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore. DHDex Horthy15 Jul 2026similar wording
• Every announcement of a longer context window hides the cost, which is that the model gets worse at following instructions. ASAravind Srinivas19 Jun 2024similar wording

On the map

Open the map on its own page →