What Dario Amodei thinks about benchmarks

@dario-amodei · 43 positions · On ppll ↗

Co-founder and CEO of Anthropic; previously VP of research at OpenAI.

Dario Amodei did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter benchmarks · Newest first

Topic

All 43benchmarks 1LLMs 10AI and jobs 7AI alignment 5data centers 4scaling laws 4AGI 3measuring intelligence 3code generation 2

14 more topics

government 2Silicon Valley 2the cloud 2AI and science 1America 1Anthropic 1chat interfaces 1China 1coding agents 1democracy 1happiness 1JavaScript 1reinforcement learning 1robotics 1

Sort

NewestOldest

Only Dario Amodei on benchmarks. See everyone on benchmarks

1 dated position, 2020, in their own words. Our reading of what Dario Amodei has said — not written or endorsed by them.

1 position so far — this page is not yet offered to search engines.

1. 28 May 2020

   Our readingOne-shot or zero-shot settings are the fairest comparisons to human performance and important targets for future work.

   Their wordsUltimately, however, one-shot, or even sometimes zero-shot, seem like the fairest comparisons to human performance, and are important targets for future work.

   ↗Language models are few-shot learners (with 30 co-authors)arxiv.org All 3 from this paper