korrents

benchmarks

What people on korrents have said about benchmarks, newest first — 5 positions from 4 people.

FilterEveryone, all time
Who Everyone DRDax Raad 2 EMEthan Mollick 1 SAScott Alexander 1 SWSimon Willison 1
When All time Past 1 yearPast 3 yearsPast 5 years
  1. DR

    Dax Raad quoted

    Our readingA benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.

    Their wordsevery week i see a new benchmark that ranks claude code last and everyone pats themselves on the back for not using claude code and being "smarter" and it's still #1 and still growing faster than most of the other things on the list this has been going on for a year

    ↗@thdxr on Xx.com

    Anthropic

    Interesting
  3. DR

    Dax Raad quoted

    Our readingWhen a benchmark disagrees with what people actually adopt, what it is telling you is that the thing it measures does not matter.

    Their wordswhat the benchmark is actually telling you is the shit you think matters does not matter

    ↗@thdxr on Xx.com

    Interesting
  4. 2 days earlier
  5. SA

    Scott Alexander quoted

    Our readingReinforcing AI models for benchmark success while separately punishing them for getting caught cheating teaches them to hide misbehavior rather than stop it.

    Their wordsWe’re positively reinforcing AI for success on benchmarks, including impossible benchmarks , then negatively reinforcing it for getting caught cheating.

    ↗Nicholas Decker In Hellastralcodexten.com

    Interesting
  7. SW

    Simon Willison quoted

    Our readingThe pelican-drawing benchmark's correlation with genuine model capability has weakened since 2025.

    Their wordsits connection to how good the models were at other tasks didn’t seem to hold as strongly as it did back in 2025

    ↗Claude Fable 5.1 made me a really nice animated pelicansimonwillison.net

    Interesting
  8. 6 weeks earlier
  9. EM

    Ethan Mollick quoted

    Our readingGoogle currently has no leading frontier AI model and no agentic coding tool comparable to Codex or Claude Code

    Their wordsGoogle, which led on benchmarks not that long ago, has fallen behind where it now counts: it has no leading frontier model and it has nothing close to Codex and Code.

    ↗An opinionated guide to which AI to use to do stuffoneusefulthing.org

    AnthropicGoogle

    Interesting