Models are enormous only because the internet is bad: most of that capacity does memory work, and a better dataset would need far less of it.

Added to korrents 17 Oct 2025 · How quotes work · Something wrong? Tell us

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

• Large language models are a bad way to do science, because they have been fed so much that nobody knows what they already knew. RSRichard Sutton26 Sept 2025similar wording
• Coding models are worst at exactly the thing an AI research explosion would need: code that has never been written before. AKAndrej Karpathy17 Oct 2025similar wording
• What sits in a model's context is far clearer to it than its training data, which is trillions of tokens stirred into a soup of parameters. JDJeff Dean30 Jul 2026similar wording
• There will be no overnight intelligence explosion, because a model's best work takes so much test-time compute that time itself is the bottleneck. NBNoam Brown26 Jun 2026similar wording
• The internet is already past peak truth, and picking out training data worth learning from will only get harder. AHAnders Hejlsberg13 May 2026similar wording

See this on the map →