Read the piece on amankhan1.substack.com ↗
Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.
Their wordsOnce a benchmark becomes a target, it can pull optimization toward the test itself. That’s the concern behind Goodhart’s Law, and it’s a reason to check more than the headline score. None of that tells you whether the new model is better at reviewing your product spec or catching the bug in your pull request.
Their wordsI can judge a spec review because I know the product and the decisions I’m trying to make. The same goes for turning my rough notes into a pitch or catching a bug in a pull request. I have enough context to push back on the answer. That’s where I start when I’m trying a new model.
Their wordsThe model is one choice. Your harness is another: are you using Claude Code, Codex, Cursor, or a chat window? Then there are the skills you’ve given it (a spec-review checklist, say) and the context it has to work from. A missing doc can matter more than which model you picked.
Keep reading
The other pieces are the ones whose claims are worded most like this one's. That measures language, not agreement: they may argue the opposite.