The right bar for an LLM evaluator is human-level performance, not perfect accuracy.

Added to korrents 23 Nov 2025 · How quotes work · Something wrong? Tell us

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

• Language models perform better on benchmark problems released before their training data cutoff, indicating data contamination inflates scores. SRSebastian Ruder13 May 2024LLMsbenchmarksResearch scientist at Meta working on multilingual models and evaluation
• A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. NBNoam Brown26 Jun 2026benchmarksResearch scientist at OpenAI
• A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. DRDax Raad3 Sept 2026benchmarksBuilds developer tools — SST, and now OpenCode at Anomaly
• A company's staff-engineer bar should be set against the best companies in the industry rather than against its own history, which is what makes title inflation a real cost. TPThuan Pham1 Apr 2026benchmarksUber's first Chief Technology Officer, from 2013 to 2020
• A speed difference between languages that share the LLVM backend measures the benchmark author, not the languages. THThePrimeagen22 Mar 2025benchmarksMichael Paulson, known online as ThePrimeagen

On the map

Open the map on its own page →