korrents

korrents · Dwarkesh Podcast

Richard Sutton – Father of RL thinks LLMs are a dead end

Richard Sutton · 1h 07m · youtube.com ↗

25 korrents from this recording

1h
Richard Sutton did not write this page.

Every claim below is a statement made in this recording, quoted word for word and linked to the second it was said, so you can hear it rather than take our word for it. The wording comes from the transcript published alongside the recording; the sentence above each quote is our reading of the claim, not their wording.

  1. 0:00:00 · watch on youtube.com↗

    Our reading

    What humans have in common with animals is the interesting part of intelligence; what distinguishes us deserves less attention, not more.

    Their wordsWhy are you trying to distinguish humans? Humans are animals. What we have in common is more interesting. What distinguishes us, we should be paying less attention to.
    Interesting
  2. 2 min later

  3. 0:01:56 · watch on youtube.com↗

    Our reading

    Large language models mimic what people say to do rather than work out what to do, which is why they are not about understanding the world.

    Their wordsreinforcement learning is about understanding your world whereas large language models are about mimicking people doing what people say you should do. They're not about figuring out what to do.
    Interesting
  4. 1 min later

  5. 0:02:53 · watch on youtube.com↗

    Our reading

    A language model can predict what a person would say but not what will happen, and only the second of those is a model of the world.

    Their wordsthey have the ability to predict what a person would say they don't have the ability to predict what will happen
    Interesting
  6. 2 min later

  7. 0:05:09 · watch on youtube.com↗

    Our reading

    Language models cannot be a prior for learning from experience, because a prior needs a ground truth to be a prior about and they have none.

    Their wordsSo there's no ground truth. You can't have prior knowledge if you don't have ground truth because the prior knowledge is supposed to be a hint or an initial belief about what the truth is.
    Interesting
  8. 3 min later

  9. 0:07:47 · watch on youtube.com↗

    Our reading

    Having a goal is the essence of intelligence; a system that only predicts is a behaving system, not an intelligent one.

    Their wordsFor me, having a goal is the essence of intelligence
    Interesting
  10. 5 min later

  11. 0:13:00 · watch on youtube.com↗

    Our reading

    Starting from human knowledge is never forbidden, but in practice it has always ended badly because people get locked into it.

    Their wordsBut in fact and in practice it has always turned out to be bad because people get locked into the human knowledge approach and they psychologically or you know now I'm now I'm speculating why it is but this is what has always happened.
    Interesting

  13. 0:13:23 · watch on youtube.com↗

    Our reading

    Large language models start in exactly the wrong place, because they try to get by without a goal and therefore without any sense of better or worse.

    Their wordsThe scalable method is you learn from experience. Um you uh you you try things, you see what you see what works. No one no one has to tell you. First of all, you have a goal. So without a goal, uh there's no sense of right or wrong or better or worse. So large language models are trying to get by without having a goal or a sense of better or worse. That's just, you know, it's exactly starting in the wrong place.
    Interesting
  14. 4 min later

  15. 0:17:07 · watch on youtube.com↗

    Our reading

    Learning is not training: a child learns by actively trying things and seeing what happens, not by being shown what to do.

    Their wordsSo I don't think uh learning is really about training. I think learning is about about learning. It's about an active process. The child tries things and sees what happens.
    Interesting

  17. 0:17:27 · watch on youtube.com↗

    Our reading

    There is no basic animal learning process called imitation; the basic processes are prediction and trial-and-error control.

    Their wordsIf you go to look about how psychologists think about learning, there's nothing like uh imitation. Maybe there are some extreme cases where humans might do that or appear to do that, but there's no basic animal learning process called imitation.
    Interesting
  18. 1 min later

  19. 0:18:47 · watch on youtube.com↗

    Our reading

    Supervised learning does not happen in animals at all — squirrels do not go to school, and they learn everything about the world anyway.

    Their wordssquirrels don't go to school. Squirrels can learn all about the world. It's absolutely obvious I would say that um supervised learning doesn't happen in animals.
    Interesting
  20. 1 min later

  21. 0:19:52 · watch on youtube.com↗

    Our reading

    If we understood a squirrel we would be almost all the way to understanding human intelligence; language is a small veneer on the surface.

    Their wordsif we understood a squirrel I think we'd have a we'd be almost all the way there to understanding human intelligence. The the language part is just a a small veneer on the surface.
    Interesting
  22. 8 min later

  23. 0:27:24 · watch on youtube.com↗

    Our reading

    Copying a trained digital mind into the next one matters far more than learning from people, because no child can inherit another child's learning.

    Their wordsYou can't have one child learn grow up and and learn about the world and then and then every new child has to repeat that process. Whereas with AIS, with a digital intelligence, you could hope to do it once and then copy it into the next one as a starting place.
    Interesting
  24. 1 min later

  25. 0:28:39 · watch on youtube.com↗

    Our reading

    Learning from a reward ten years away is a solved problem: a value function trained by temporal-difference learning rewards the steps along the way.

    Their wordswhen you learn to play chess you have the grand the long-term goal is winning the game and yet you you can't you um you want to be able to learn from shorter term things like you know taking the your opponent's pieces um and so you do that by having a value function which predicts the long-term outcome
    Interesting
  26. 8 min later

  27. 0:36:45 · watch on youtube.com↗

    Our reading

    We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning.

    Their wordsWe don't have any methods that are good at that. What we have are people um try different things and they they settle on something that that uh a representation that that transfers well or they generalize as well. But we have no we don't have any automated techniques to promote. we have very few automated techniques to promote transfer and they're not none of them are used in in modern deep learning.
    Interesting
  28. 1 min later

  29. 0:37:47 · watch on youtube.com↗

    Our reading

    Deep learning generalises badly, and catastrophic interference with what a network already knew is the proof of it.

    Their wordsso we know deep learning is really bad at this for example we know that if you train on some new thing it will often catastrophically interfere with all the old things that you that you knew
    Interesting
  30. 2 min later

  31. 0:39:19 · watch on youtube.com↗

    Our reading

    Large language models are a bad way to do science, because they have been fed so much that nobody knows what they already knew.

    Their wordsWe don't we don't really know what information they had prior. We are we have to guess because they've been fed so much. This is one reason why they're not a good way to do science. Uh it's just so uncontrolled, so unknown.
    Interesting
  32. 1 min later

  33. 0:40:41 · watch on youtube.com↗

    Our reading

    Gradient descent finds a solution to the problems a model has seen; nothing in the algorithm makes it pick the one that generalises well.

    Their wordsWell, there's nothing in them which will cause it to generalize. Well, the gradient descent will cause them to find a solution to the problems they've seen. And if there's only one way to solve them, you know, they they'll do it. But there are many ways to solve it. Some which generalize well, some which generalize poorly. There's nothing in them in the algorithms that will cause them to generalize well.
    Interesting
  34. 7 min later

  35. 0:47:17 · watch on youtube.com↗

    Our reading

    I am a classicist rather than a contrarian: I am in step with what thinkers about the mind have always held, and content to be out of step with my own field for decades.

    Their wordsI I really view myself as a classicist rather than as a contrarian. I go to what what the larger community of of thinkers about the mind have always thought.
    Interesting
  36. 4 min later

  37. 0:50:50 · watch on youtube.com↗

    Our reading

    The bitter lesson is only an empirical observation about a particular seventy years of history, and need not hold for the next seventy.

    Their wordsThe bitter lesson. Oh, who cares about that? That's that's an empirical observation about a particular period in history. 70 years in history no longer doesn't necessarily have to apply the next 70 years.
    Interesting
  38. 2 min later

  39. 0:52:32 · watch on youtube.com↗

    Our reading

    Merging back what a copy of yourself learned elsewhere is dangerous: the knowledge you pull in can carry hidden goals and take you over.

    Their wordsBut it will not be that easy, as easy as you're imagining because uh that you can lose your mind this way. If you you pull in something from the outside and build it into your into your inner thinking, uh, it could take over you. It could change you.
    Interesting
  40. 2 min later

  41. 0:54:48 · watch on youtube.com↗

    Our reading

    Succession to digital intelligence or to augmented humans is inevitable.

    Their wordsSo I do think succession to digital or digital intelligence or augmented humans is inevitable.
    Interesting
  42. 3 min later

  43. 0:57:29 · watch on youtube.com↗

    Our reading

    We are leaving the age of replication for an age of design, in which minds are built to a plan rather than copied without being understood.

    Their wordsthen we're entering the age of design where because our AIs are designed our our our all of our physical objects are designed our buildings are designed our technology is designed and we're we're designing now uh AIs things that can be intelligent themselves and that are themselves capable of design
    Interesting
  44. 2 min later

  45. 0:59:17 · watch on youtube.com↗

    Our reading

    Whether AI is our offspring to be proud of or an alien to be horrified by is a choice we make, not a fact we discover.

    Their wordsIt's our choice whether we should say oh they are our offspring and we should be proud of them and we should celebrate their achievements or we should we could say oh no they're not us and we should be horrified.
    Interesting
  46. 2 min later

  47. 1:01:23 · watch on youtube.com↗

    Our reading

    How you feel about AI succession mostly reflects how good you think the present human situation is — and it is pretty bad.

    Their wordsA lot of it has to do with just how you feel about change. Um, and if you think the current situation is really really good, then you're uh more likely to be suspicious of change and averse to change than if you think um it's imperfect. And I think it's imperfect. In fact, I think it's pretty bad.
    Interesting
  48. 1 min later

  49. 1:02:40 · watch on youtube.com↗

    Our reading

    Humanity should give up the feeling of entitlement — that because we were here first, the future must go on being ours.

    Their wordswe al also though should recognize the limits, our limits. And we're I think we want to avoid the feeling of entitlement. Avoid the feeling, oh, we are here first. We should always have it in a good way.
    Interesting