korrents

korrents · piece

Self-exfiltration is a key dangerous capability

Jan Leike · 13 Sept 2023 · aligned.substack.com ↗

2 korrents from this piece

Jan Leike did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

  1. Our reading

    Once model weights leave their owner's control the loss is irreversible, which is why securing them is the precondition for governing AI at all.

    Their wordsMoreover, model exfiltration is likely impossible to reverse.
    Interesting

  2. Our reading

    The dangerous capability that matters most for misalignment is whether a model can exfiltrate its own weights, because that is the one that removes it from your control.

    Their wordsIf a model was capable of self-exfiltration, it would have the option to remove itself from your control.
    Interesting