What Neel Nanda thinks about scaling laws
Runs the mechanistic interpretability team at Google DeepMind; worked on interpretability at Anthropic before that.
Neel Nanda did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter scaling laws · Newest first
Only Neel Nanda on scaling laws. See everyone on scaling laws
4 dated positions, 2025, in their own words. Our reading of what Neel Nanda has said — not written or endorsed by them.
-
Their wordsUm, more generally, I think that um, the easier version of this of just after a post- training run, go look at the RL transcripts and figure out problems with the various environments use and see if you can use this for debugging things. Seems like a pretty good thing to do in general.
↗How Aligned Is Claude? A Live Review of the Opus 4.5 System Cardyoutube.com 2nd of 27 in this recording
-
Our readingPrefill attacks work because pre-training deeply ingrains a reflex in models to continue sentences.
Their wordsAnd they have like a really strong prior for continuing sentences because you know if you don't continue a sentence properly you normally fail in pre-training which is a strong incentive to really deeply ingrain the reflex to continue sentences.
↗How Aligned Is Claude? A Live Review of the Opus 4.5 System Cardyoutube.com 6th of 27 in this recording
- 7 days earlier
-
Their wordsBeing smart is not the most important thing. You need to be scalable and you need to be good at flexibly using large amounts of compute to do the things that you care about doing.
↗Bitter Lesson-Pilled Interp: A Live Paper Review (Activation Oracles & PCD)youtube.com 1st of 21 in this recording
-
Their wordsAnd presumably if you train it a bunch longer, things might change in further weird qualitative ways. And yeah, this is like pretty interesting and suggests that it might be hard to form confident takes from cheap small scale experiments.
↗Bitter Lesson-Pilled Interp: A Live Paper Review (Activation Oracles & PCD)youtube.com 20th of 21 in this recording