What Ilya Sutskever thinks about benchmarks

@ilya-sutskever · 28 positions · On ppll ↗

Co-founder and chief scientist of Safe Superintelligence Inc., and previously co-founder and chief scientist of OpenAI.

Ilya Sutskever did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter benchmarks · Newest first

Topic

All 28 benchmarks 1 AGI 6 AI alignment 6 scaling laws 6 reinforcement learning 4 LLMs 3 evolutionary biology 2 innovation 2 measuring intelligence 2

8 more topics

startups 2 Addiction 1 AI agents 1 AI and science 1 AI sentience 1 health and fitness 1 mathematics 1 self-driving cars 1

Sort

Newest Oldest

Only Ilya Sutskever on benchmarks. See everyone on benchmarks

1 dated position, 2025, in their own words. Our reading of what Ilya Sutskever has said — not written or endorsed by them.

1 position so far — this page is not yet offered to search engines.

25 Nov 2025

Our readingModels look far better on evals than they are in the world because researchers, inadvertently, take inspiration from the evals when they build RL environments.

reinforcement learning

Their wordsAnd one of the one thing you could do, and I think that's something that is done inadvertently, is that people take inspiration from the evals.

↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 1st of 28 in this recording