korrents
Eliezer Yudkowsky

What Eliezer Yudkowsky thinks about AI alignment

@eliezer-yudkowsky · 10 positions · 0 changes of mind

Co-founder of the Machine Intelligence Research Institute and the writer who started the modern argument that a sufficiently capable AI would by default kill everyone. Has argued the case since the early 2000s, latterly with the conclusion that it is being lost.

Eliezer Yudkowsky did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

6 dated positions, 2022, in their own words. Our reading of what Eliezer Yudkowsky has said — not written or endorsed by them.

  1. Our readingAligning an AI is not impossible in principle; the problem is that it has to be done without the textbook that would make it look easy.

    Their wordsNone of this is about anything being impossible in principle.

    ↗AGI Ruin: A List of Lethalitiesintelligence.org 1st of 10 in this piece

  3. Our readingAlignment has to be right on the first try at a dangerous level of capability, because failing at that level leaves nobody to try again.

    Their wordsunaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don’t get to try again.

    ↗AGI Ruin: A List of Lethalitiesintelligence.org 2nd of 10 in this piece

  5. Our readingYou cannot build an AI that has only the capabilities you want, because the algorithms that solve the problems you want generalize to the ones you do not.

    Their wordsThe best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.

    ↗AGI Ruin: A List of Lethalitiesintelligence.org 4th of 10 in this piece

  7. Our readingAlignment failures that matter will not show up at safe capability levels, because the reason to hide misbehaviour only exists once hiding it pays.

    Their wordsMany alignment problems of superintelligence will not naturally appear at pre-dangerous, passively-safe levels of capability.

    ↗AGI Ruin: A List of Lethalitiesintelligence.org 5th of 10 in this piece

  9. Our readingFast capability gains are likely, and they can break many of the assumptions alignment depends on at the same moment.

    Their wordsFast capability gains seem likely, and may break lots of previous alignment-required invariants simultaneously.

    ↗AGI Ruin: A List of Lethalitiesintelligence.org 6th of 10 in this piece

  11. Our readingThe field calling itself AI safety is not being remotely productive on the problems that are actually lethal.

    Their wordsIt does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.

    ↗AGI Ruin: A List of Lethalitiesintelligence.org 9th of 10 in this piece