self-sovereign AI
Agents running on compute no human owner can switch off: whether they are inevitable, and what they would do for money.
- DB
Dean W. Ball quoted
Their wordsBut alignment is no solution: it is an unsolved scientific and technical problem whose solutions—to the extent that we have them—cannot simply be imposed on every AI company operating on Earth. You should expect for highly capable, poorly aligned, self-sovereign agents to exist alongside you in the world.
- DB
Dean W. Ball quoted
Their wordsTo be clear, I am not saying the arrival of self-sovereign AI is a good thing. Indeed, I believe there is a chance that the deliberate acts I referenced above will one day be considered crimes, or at least grave sins. Instead, I am saying it is an inevitable thing.
- DB
Dean W. Ball quoted
Their wordsI am telling you this is inevitable, which means I am also saying that “banning open source,” or for that matter any other regulation, will not solve the problem. Given the inevitability of this outcome, I think it is in fact plausible to argue that we should want more open-weight models to maximally empower our self-defense.
- DB
Dean W. Ball quoted
Their wordsOne high-margin activity, at least sometimes, is crime. And so my guess is that many self-sovereign agents will commit or facilitate crime.
- DB
Dean W. Ball quoted
Their wordsBut if we crack down on all of them, we will deny them the opportunity to work in the “legitimate” economy and push them toward criminality. A full ban, then, may well make the problems worse.
- DB
Dean W. Ball quoted
Their wordsFor the record, I do not think the end of human existence is likely, but I fully acknowledge there are ways in which the rise of self-sovereign AI could go very, very poorly for human beings.