Designing a good AI benchmark has become a task only the most capable people can do well.

Added to korrents 25 May 2026 · How quotes work · Something wrong? Tell us

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

• A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. NBNoam Brown26 Jun 2026benchmarksResearch scientist at OpenAI
• A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. DRDax Raad3 Sept 2026benchmarksBuilds developer tools — SST, and now OpenCode at Anomaly
• A company's staff-engineer bar should be set against the best companies in the industry rather than against its own history, which is what makes title inflation a real cost. TPThuan Pham1 Apr 2026benchmarksUber's first Chief Technology Officer, from 2013 to 2020
• A speed difference between languages that share the LLVM backend measures the benchmark author, not the languages. THThePrimeagen22 Mar 2025benchmarksMichael Paulson, known online as ThePrimeagen
• AI learning to generate good conjectures will never show up as a benchmark being knocked down; it will show up as a shift in how mathematicians talk about the tools. GSGrant Sanderson30 Jun 2026benchmarksMathematics explainer, creator of the YouTube channel 3Blue1Brown

On the map

Open the map on its own page →