What Gwern thinks about benchmarks
Pseudonymous essayist. Writes at gwern.net on AI scaling, statistics, genetics and long-running internet research, with heavy citation and a habit of revising old essays in place rather than leaving them wrong.
Gwern did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter benchmarks · Newest first
Only Gwern on benchmarks. See everyone on benchmarks
1 dated position, 2021, in their own words. Our reading of what Gwern has said — not written or endorsed by them.
-
Their wordsThe more people tweet about how MLP-Mixer is overhyped because is −X% worse than the ultra-hand-optimized baseline or requires Y× more FLOPS, the more they demonstrate precisely why this sort of research is so important!
↗May 2021 Gwern.net Newslettergwern.substack.com 1st of 3 in this piece