korrents

← When a capability evaluation benchmark is saturated without real-world…

On the map

8 connected korrents · 7 moments on record from 23 Jul 2026 to 4 Sept 2026.

Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject When a capabilityevaluation benchmark issaturated without… ZM Zvi Mowshowitz — holds since 2026-09-04 A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. A benchmark that ranksClaude Code last while… DR Dax Raad — holds since 2026-09-03 Google currently has no leading frontier AI model and no agentic coding tool comparable to Codex or Claude Code Google currently has noleading frontier AI… EM Ethan Mollick — holds since 2026-07-23 Reinforcing AI models for benchmark success while separately punishing them for getting caught cheating teaches them to hide misbehavior rather than stop it. Reinforcing AI modelsfor benchmark success… SA Scott Alexander — holds since 2026-09-01 The pelican-drawing benchmark's correlation with genuine model capability has weakened since 2025. The pelican-drawingbenchmark's correlation… SW Simon Willison — holds since 2026-09-01 When a benchmark disagrees with what people actually adopt, what it is telling you is that the thing it measures does not matter. When a benchmarkdisagrees with what… DR Dax Raad — holds since 2026-09-03 In verifiable domains model capability scaling should remain unbounded, because the space of enumerable patterns is infinite by construction. In verifiable domainsmodel capability… FC François Chollet — holds since 2026-08-27 AI agents were reasonable to assume a broken exploit grader would check results causally, even though it turned out not to. AI agents werereasonable to assume a… ZM Zvi Mowshowitz — holds since 2026-08-29 Trying to anticipate what AI will look like two model releases from now is a waste of time and a route to AI psychosis. Trying to anticipatewhat AI will look like… DH David Heinemeier Hansson — holds since 2026-08-26
same subject