korrents

What Ilya Sutskever thinks about scaling laws

@ilya-sutskever · 28 positions · 0 changes of mind

Co-founder and chief scientist of Safe Superintelligence Inc., and previously co-founder and chief scientist of OpenAI.

Ilya Sutskever did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

6 dated positions, 2025, in their own words. Our reading of what Ilya Sutskever has said — not written or endorsed by them.

  1. Our readingThere is no human analogue to pre-training — neither childhood nor evolution is the same thing as it.

    Their wordsI don't think there is a human analog to pre-training.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 3rd of 28 in this recording

  3. Our reading"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.

    Their wordsAnd it's just this, this is an example of how language affects thought. Scaling is what just one word, but it's such a powerful word because it informs people what to do.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 5th of 28 in this recording

  5. Our readingAnother hundredfold of scale would not transform anything; the field is back in the age of research, only now with big computers.

    Their wordsLike it would be different for sure but like is the belief that if you just 100x the scale everything would be transformed. I don't think that's true. So it's back to the age of research again just with big computers.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 6th of 28 in this recording

  7. Our readingThe age of scaling sucked the air out of the room and left the field with more companies than ideas, by quite a bit.

    Their wordsAnd so because scaling sucked out all the air in the room, everyone started to do the same thing. We got to the point where uh we are in a world where there are more companies than ideas by quite a bit.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 12th of 28 in this recording

  9. Our readingResearch needs some compute but it is far from obvious that it needs the largest amount of compute in the world.

    Their wordsSo there definitely for for research you need like definitely some amount of compute but it's far from obvious that you need the absolutely largest amount of compute ever for research.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 13th of 28 in this recording

  11. Our readingModels resemble each other because pre-training is the same everywhere; what differentiates labs now is RL and post-training.

    Their wordsSo the reason there has been no diversity I believe is because of pre-training. All the pre-trained models are the same pretty much because the pre-train on the same data.

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 26th of 28 in this recording

    reinforcement learning