What Neel Nanda thinks about transformers
Runs the mechanistic interpretability team at Google DeepMind; worked on interpretability at Anthropic before that.
Neel Nanda did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter transformers · Newest first
Only Neel Nanda on transformers. See everyone on transformers
1 dated position, 2025, in their own words. Our reading of what Neel Nanda has said — not written or endorsed by them.
-
Their wordsLinear probing is OP. Um and attention head probes are even more OP, but whatever.
↗Bitter Lesson-Pilled Interp: A Live Paper Review (Activation Oracles & PCD)youtube.com 5th of 21 in this recording