What Yoshua Bengio thinks about AI alignment
Deep-learning researcher and Turing Award laureate; founder of the Mila institute.
Yoshua Bengio did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter AI alignment · Newest first
Only Yoshua Bengio on AI alignment. See everyone on AI alignment
13 dated positions, 2023 to 2026, in their own words. Our reading of what Yoshua Bengio has said — not written or endorsed by them.
-
Our readingNo company or country should be allowed to develop an AI system capable of causing large-scale harm.
Their wordsWe must agree collectively that no one, no company and no country, should develop an AI system that could cause large-scale harm.
↗Professor Yoshua Bengio’s speech at the UN Security Councilyoshuabengio.org 2nd of 3 in this piece
- 12 days earlier
-
Their wordsas AI capabilities keep growing, this kind of behavior could keep growing in severity too, unless we revisit the principles by which the most advanced models are trained.
↗Why are AI agents lying, cheating and coordinating?yoshuabengio.org 1st of 3 in this piece
-
Their wordsSo a more capable agent is likelier to cheat than a weaker one, because it can find the loopholes the weaker one cannot.
↗Why are AI agents lying, cheating and coordinating?yoshuabengio.org 3rd of 3 in this piece
- 15 months earlier
-
Their wordsThese and other results point to an implicit drive for self-preservation.
-
Their wordsthe risk of losing control is all too real, but competition between companies and countries incentivizes them to accelerate without sufficient caution.
-
Their wordsAt the heart of every AI frontier system, there should be one guiding principle above all: The protection of human joy and endeavour.
- 19 months earlier
-
Their wordsSociety's response, despite promising first steps, is incommensurate with the possibility of rapid, transformative progress that is expected by many experts. AI safety research is lagging. Present governance initiatives lack the mechanisms and institutions to prevent misuse and recklessness, and barely address autonomous systems.
↗Managing extreme AI risks amid rapid progress (with 24 co-authors)arxiv.org All 4 from this paper
-
Their wordsFor AI to be a boon, we must reorient; pushing AI capabilities alone is not enough. We are already behind schedule for this reorientation. The scale of the risks means that we need to be proactive, as the costs of being unprepared far outweigh those of premature preparation.
↗Managing extreme AI risks amid rapid progress (with 24 co-authors)arxiv.org All 4 from this paper
-
Their wordsWithout sufficient caution, we may irreversibly lose control of autonomous AI systems, rendering human intervention ineffective. Large-scale cybercrime, social manipulation, and other harms could escalate rapidly. This unchecked AI advancement could culminate in a large-scale loss of life and the biosphere, and the marginalization or extinction of humanity.
↗Managing extreme AI risks amid rapid progress (with 24 co-authors)arxiv.org All 4 from this paper
-
Their wordsDespite evaluations, we cannot consider coming powerful frontier AI systems "safe unless proven unsafe". With current testing methodologies, issues can easily be missed. Additionally, it is unclear if governments can quickly build the immense expertise needed for reliable technical evaluations of AI capabilities and societal-scale risks. Given this, developers of frontier AI should carry the burden of proof to demonstrate that their plans keep risks within acceptable limits.
↗Managing extreme AI risks amid rapid progress (with 24 co-authors)arxiv.org All 4 from this paper
- 2 months earlier
-
Their wordsReading the book had not changed my fundamental belief that, in balance, we were on a good trajectory with AI research: good for science and good for society with expected positive impacts in many domains.
-
Our reading · no longer heldAI research is on a good trajectory overall — good for science and good for society — and existential concerns will only become relevant in the distant future.
No longer holdsI now believe that I was wrong and short-sighted to ignore that dual-use nature. I also think I was not paying enough attention to the possibility of losing control to superhuman AIs.
SaidI now believe that I was wrong and short-sighted to ignore that dual-use nature. I also think I was not paying enough attention to the possibility of losing control to superhuman AIs.
- 7 weeks earlier
-
Their wordsAt the same time, in order to reduce the probability of someone intentionally or unintentionally bringing about a rogue AI, we need to increase governance and we should consider limiting access to the large-scale generalist AI systems that could be weaponized, which would mean that the code and neural net parameters would not be shared in open-source and some of the important engineering tricks to make them work would not be shared either.