korrents

What Ilya Sutskever thinks about reinforcement learning

@ilya-sutskever · 28 positions · 0 changes of mind

Co-founder and chief scientist of Safe Superintelligence Inc., and previously co-founder and chief scientist of OpenAI.

Ilya Sutskever did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

7 dated positions, 2025, in their own words. Our reading of what Ilya Sutskever has said — not written or endorsed by them.

7 positions so far — this page is not yet offered to search engines.

  1. Our readingModels look far better on evals than they are in the world because researchers, inadvertently, take inspiration from the evals when they build RL environments.

    Their wordsAnd one of the one thing you could do, and I think that's something that is done inadvertently, is that people take inspiration from the evals.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 1st of 28 in this recording

  3. Our readingOur built-in emotions were tuned for a world that no longer exists, and hunger is one that now fails to guide us correctly.

    Their wordsI think for example our intuitive feeling of hunger is not succeeding in guiding us correctly in this world with an abundance of food.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 4th of 28 in this recording

  5. Our readingValue functions only make reinforcement learning faster: anything you can do with one you can also do without it, just more slowly.

    Their wordsI want to like emphasize that I think the value function is something like it's going to make RL more efficient and I think that makes a difference but I think that anything you can do with a value function you can do without just more slowly.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 7th of 28 in this recording

  7. Our readingThe age of scaling sucked the air out of the room and left the field with more companies than ideas, by quite a bit.

    Their wordsAnd so because scaling sucked out all the air in the room, everyone started to do the same thing. We got to the point where uh we are in a world where there are more companies than ideas by quite a bit.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 12th of 28 in this recording

  9. Our readingResearch needs some compute but it is far from obvious that it needs the largest amount of compute in the world.

    Their wordsSo there definitely for for research you need like definitely some amount of compute but it's far from obvious that you need the absolutely largest amount of compute ever for research.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 13th of 28 in this recording

  11. Our readingModels resemble each other because pre-training is the same everywhere; what differentiates labs now is RL and post-training.

    Their wordsSo the reason there has been no diversity I believe is because of pre-training. All the pre-trained models are the same pretty much because the pre-train on the same data.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 26th of 28 in this recording

    scaling laws

  13. Our readingSelf-play as it was done in the past is too narrow: it only ever develops a small set of skills like negotiation and strategising.

    Their wordsNow the the thing is that selfplay at least the way it was done in the past when you have agents which are somehow compete with each other it's only good for developing a certain set of skills it is too narrow.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 27th of 28 in this recording