korrents

Ilya Sutskever

@ilya-sutskever · 28 positions · 0 changes of mind

Co-founder and chief scientist of Safe Superintelligence Inc., and previously co-founder and chief scientist of OpenAI.

Ilya Sutskever did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

  1. Our reading

    Models look far better on evals than they are in the world because researchers, inadvertently, take inspiration from the evals when they build RL environments.

    Their wordsAnd one of the one thing you could do, and I think that's something that is done inadvertently, is that people take inspiration from the evals.
    spoken · machine transcript hear it at 0:04:08 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 1st of 28 in this recording

    reinforcement learning

    Interesting

  3. Our reading

    Today's models are the student who drilled 10,000 hours for the programming contest, which is exactly why what they learn does not carry anywhere else.

    Their wordsThe models are much more like the first student but even more because then we say okay so the model should be good at competitive programming so let's get every single competitive programming problem ever and then let's do some data augmentation so we have even more competitive programming problems
    spoken · machine transcript hear it at 0:06:58 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 2nd of 28 in this recording

    Interesting

  5. Our reading

    There is no human analogue to pre-training — neither childhood nor evolution is the same thing as it.

    Their wordsI don't think there is a human analog to pre-training.
    spoken · machine transcript hear it at 0:09:36 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 3rd of 28 in this recording

    scaling laws

    Interesting

  7. Our reading

    Our built-in emotions were tuned for a world that no longer exists, and hunger is one that now fails to guide us correctly.

    Their wordsI think for example our intuitive feeling of hunger is not succeeding in guiding us correctly in this world with an abundance of food.
    spoken · machine transcript hear it at 0:18:35 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 4th of 28 in this recording

    Interesting

  9. Our reading

    "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.

    Their wordsAnd it's just this, this is an example of how language affects thought. Scaling is what just one word, but it's such a powerful word because it informs people what to do.
    spoken · machine transcript hear it at 0:19:39 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 5th of 28 in this recording

    Interesting

  11. Our reading

    Another hundredfold of scale would not transform anything; the field is back in the age of research, only now with big computers.

    Their wordsLike it would be different for sure but like is the belief that if you just 100x the scale everything would be transformed. I don't think that's true. So it's back to the age of research again just with big computers.
    spoken · machine transcript hear it at 0:21:51 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 6th of 28 in this recording

    Interesting

  13. Our reading

    Value functions only make reinforcement learning faster: anything you can do with one you can also do without it, just more slowly.

    Their wordsI want to like emphasize that I think the value function is something like it's going to make RL more efficient and I think that makes a difference but I think that anything you can do with a value function you can do without just more slowly.
    spoken · machine transcript hear it at 0:24:40 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 7th of 28 in this recording

    reinforcement learning

    Interesting

  15. Our reading

    The most fundamental thing wrong with today's models is not scale or efficiency but that they generalize dramatically worse than people do.

    Their wordsThe thing which I think is the most fundamental is that these models somehow just generalize dramatically worse than people.
    spoken · machine transcript hear it at 0:25:00 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 8th of 28 in this recording

    Interesting

  17. Our reading

    A five-year-old's vision is already good enough to drive a car, on a tiny and strikingly undiverse amount of data.

    Their wordsAt least for me, when I remember myself being 5 years old, my I was I was very excited about cars back then, and I'm pretty sure my car recognition was more than adequate for self-driving already.
    spoken · machine transcript hear it at 0:27:32 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 9th of 28 in this recording

    self-driving cars

    Interesting

  19. Our reading

    That people are good at maths and coding, which evolution never selected for, shows human learning is not a complicated evolutionary prior but something more fundamental.

    Their wordsWhat I meant to say is that language math and coding and especially math and coding suggests that whatever it is that makes people good at learning is probably not so much a complicated prior but something more some fundamental thing.
    spoken · machine transcript hear it at 0:28:09 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 10th of 28 in this recording

    Interesting

  21. Our reading

    The human value function is extraordinarily robust — addiction aside — which is why a person can judge their own performance at a skill they have only just started.

    Their wordsThey have a general sense which is also by the way extremely robust in people like whatever it is the human value function whatever the human value function is with a few exceptions around addiction it's actually very very robust
    spoken · machine transcript hear it at 0:30:41 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 11th of 28 in this recording

    Addiction

    Interesting

  23. Our reading

    The age of scaling sucked the air out of the room and left the field with more companies than ideas, by quite a bit.

    Their wordsAnd so because scaling sucked out all the air in the room, everyone started to do the same thing. We got to the point where uh we are in a world where there are more companies than ideas by quite a bit.
    spoken · machine transcript hear it at 0:36:51 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 12th of 28 in this recording

    Interesting

  25. Our reading

    Research needs some compute but it is far from obvious that it needs the largest amount of compute in the world.

    Their wordsSo there definitely for for research you need like definitely some amount of compute but it's far from obvious that you need the absolutely largest amount of compute ever for research.
    spoken · machine transcript hear it at 0:39:06 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 13th of 28 in this recording

    Interesting

  27. Our reading

    Letting the public see a powerful AI does something no essay about AI can do, and that is a real reason not to straight-shot superintelligence.

    Their wordsLike basically I think I think that there is a big benefit from AI being in the public and that would be a reason for us to not be quite straight shot.
    spoken · machine transcript hear it at 0:45:20 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 14th of 28 in this recording

    AI alignment

    Interesting

  29. Our reading

    A human being is not an AGI: we lack a huge amount of knowledge and rely on continual learning instead, so continual learning is what superintelligence should mean.

    Their wordsA human being, a human being lacks a huge amount of knowledge. Instead, we rely on continual learning. We rely on continual learning.
    spoken · machine transcript hear it at 0:49:53 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 15th of 28 in this recording

    AGIAI alignment

    Interesting

  31. Our reading

    Broad deployment of AI that can learn on the job will very likely produce rapid economic growth for some period of time.

    Their wordsBut I think the idea of very rapid economic growth for some time, I think it's very possible from broad deployment.
    spoken · machine transcript hear it at 0:53:12 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 16th of 28 in this recording

    Interesting

  33. Our reading

    AI should be deployed incrementally and in advance, because a system that does not yet exist is one almost nobody can imagine.

    Their wordsone of the one of the ways in which my thinking has been changing is that I now place more importance on AI being deployed incrementally and in advance.
    spoken · machine transcript hear it at 0:56:10 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 17th of 28 in this recording

    Interesting

  35. Our reading

    The whole problem of AI and AGI is the power, and nothing else about it is the problem.

    Their wordsIndeed, the whole problem, what is the problem of AI and AGI? The whole problem is the power. The whole problem is the power. When the power is really big, what's going to happen?
    spoken · machine transcript hear it at 0:57:34 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 18th of 28 in this recording

    AGI

    Interesting

  37. Our reading

    Once AI starts to feel powerful rather than mistake-prone, every AI company will become far more paranoid about safety than it is today.

    Their wordsI do think that at some point the AI will start to feel powerful actually and I think when that happens we will see a big change in the way all AI companies approach safety.
    spoken · machine transcript hear it at 1:00:06 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 19th of 28 in this recording

    Interesting

  39. Our reading

    It will be easier to build an AI that cares about all sentient life than one that cares about humans alone, because the AI itself will be sentient.

    Their wordsI think in particular it will be there's a case to be made that it will be easier to build an AI that cares about sentient life than an AI that cares about human life alone because the AI itself will be sentient.
    spoken · machine transcript hear it at 1:01:29 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 20th of 28 in this recording

    AI sentience

    Interesting

  41. Our reading

    Capping the power of the most powerful superintelligence would be materially helpful, whatever the difficulty of doing it.

    Their wordsNumber three, I think it would be really materially helpful if the power of the most powerful super intelligence was somehow capped because it would address a lot of these concerns.
    spoken · machine transcript hear it at 1:03:10 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 21st of 28 in this recording

    AI alignment

    Interesting

  43. Our reading

    The long-run equilibrium with powerful AI may require people to merge with it, so that whatever the AI understands is transmitted wholesale to its human.

    Their wordsAnd the solution is if people become part AI with some kind of neural link++ because what will happen as a result is that now the AI understands something and we understand it too like because now the understanding is transmitted wholesale.
    spoken · machine transcript hear it at 1:10:18 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 22nd of 28 in this recording

    Interesting

  45. Our reading

    The AI companies' strategies will converge, because as AI grows more powerful it becomes clearer to everyone what the strategy should be.

    Their wordsI maintain that in the end there will be a convergence of strategies. So I think there will be a convergence of strategies where at some point as AI becomes more powerful it's going to become more or less clearer to everyone what the strategy should be.
    spoken · machine transcript hear it at 1:21:01 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 23rd of 28 in this recording

    Interesting

  47. Our reading

    Even with human-like learners, competition will split the AI market into specialised niches rather than hand it to one company.

    Their wordsI think what's going to happen is that the way competition like competition loves specialization and you see it in the market, you see it in evolution as well. So you're going to have lots of different niches and you're going to have lots of different companies who are occupying different niches
    spoken · machine transcript hear it at 1:26:42 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 24th of 28 in this recording

    Interesting

  49. Our reading

    A million copies of one researcher would hit diminishing returns quickly, because what research needs is people who think differently.

    Their wordsI think I think there'll definitely be there'll be diminishing returns because you want you want people who think differently rather than the same. I think that if they were literal copies of me, I'm not sure how much more incremental value you'd get.
    spoken · machine transcript hear it at 1:29:00 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 25th of 28 in this recording

    Interesting

  51. Our reading

    Models resemble each other because pre-training is the same everywhere; what differentiates labs now is RL and post-training.

    Their wordsSo the reason there has been no diversity I believe is because of pre-training. All the pre-trained models are the same pretty much because the pre-train on the same data.
    spoken · machine transcript hear it at 1:30:02 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 26th of 28 in this recording

    scaling lawsreinforcement learning

    Interesting

  53. Our reading

    Self-play as it was done in the past is too narrow: it only ever develops a small set of skills like negotiation and strategising.

    Their wordsNow the the thing is that selfplay at least the way it was done in the past when you have agents which are somehow compete with each other it's only good for developing a certain set of skills it is too narrow.
    spoken · machine transcript hear it at 1:31:07 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 27th of 28 in this recording

    Interesting

  55. Our reading

    A top-down aesthetic belief is what sustains a researcher when the experiments contradict them, because data alone cannot tell a wrong idea from a bug.

    Their wordsAnd then the top down belief is the thing that sustains you when the experiments contradict you. Because if you just trust the data all the time, well, sometimes you can be doing a correct thing, but there's a bug.
    spoken · machine transcript hear it at 1:34:54 · all korrents from this recording

    ↗Ilya Sutskever – We're moving from the age of scaling to the age of researchyoutube.com 28th of 28 in this recording

    Interesting