What Boris Cherny thinks about benchmarks

@boris-cherny · 35 positions · On ppll ↗

Creator of Claude Code at Anthropic.

Boris Cherny did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter benchmarks · Newest first

Topic

All 35benchmarks 1LLMs 11AI agents 4Anthropic 3coding agents 3design 3JavaScript 3prompt injection 2startups 2

17 more topics

AI and jobs 1AI and science 1ambition 1chat interfaces 1code generation 1developer tools 1investing 1management 1neural networks 1open source 1optimism 1patience 1remote work 1scaling laws 1simplicity 1software performance 1software quality 1

Sort

NewestOldest

Only Boris Cherny on benchmarks. See everyone on benchmarks

1 dated position, 2026, in their own words. Our reading of what Boris Cherny has said — not written or endorsed by them.

1 position so far — this page is not yet offered to search engines.

1. 27 Jul 2026

   Our readingEvals are not the durable asset people think they are: one survives about one to three model generations before it saturates and has to be thrown away.

   Their wordsI think evals, they outlive the harness a little bit, but not by that much. Like an eval might live for maybe one, two, three model generations, but nowadays the you know, we're on the exponential. The model is improving so quickly, very often we just saturate the eval, and then we have to throw it away, and we have to come up with a new eval.

   ↗Boris Cherny: We Cut 80% of Claude Code's Promptyoutube.com 9th of 20 in this recording