Self-exfiltration is a key dangerous capability

Jan Leike · 13 Sept 2023

Others here said this too

Once model weights leave their owner's control the loss is irreversible, which is why securing them is the precondition for governing AI at all.

Their wordsMoreover, model exfiltration is likely impossible to reverse.

Also said by Miles Brundage

Read the piece on aligned.substack.com ↗

2 korrents from this piece ↓

Jan Leike did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

Our reading
Once model weights leave their owner's control the loss is irreversible, which is why securing them is the precondition for governing AI at all.

Their wordsMoreover, model exfiltration is likely impossible to reverse.
Our reading
The dangerous capability that matters most for misalignment is whether a model can exfiltrate its own weights, because that is the one that removes it from your control.

Their wordsIf a model was capable of self-exfiltration, it would have the option to remove itself from your control.

Keep reading

The other pieces are the ones whose claims are worded most like this one's. That measures language, not agreement: they may argue the opposite.