Why Security Comes First

Miles Brundage · 10 Jan 2025 · milesbrundage.substack.com ↗

2 korrents from this piece

Miles Brundage did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

Our reading
AI companies will not fix their own security without outside pressure, because they bear the cost of investing and everyone else bears the cost of their not investing.

Their wordsAnd while the costs of investing in security are largely borne by the company, the costs of under-investing are borne by a much larger set of actors, e.g. perhaps the country or world as a whole.

Interesting
Our reading
Securing model weights against theft is a more urgent policy priority than safety research, because security is built out of atoms and takes calendar time that safety does not.

Their wordsIncreasing the incentive and technical ability of AI companies to have good security is a very high priority, and in my view even more urgent than safety (though that’s also very important).

Interesting