10 korrents from this piece
Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.
Their wordsNone of this is about anything being impossible in principle.
Their wordsunaligned operation at a dangerous level of intelligence kills everybody on Earth and then we don’t get to try again.
Their words2 years after the leading actor has the capability to destroy the world, 5 other actors will have the capability to destroy the world.
Their wordsThe best and easiest-found-by-optimization algorithms for solving problems we want an AI to solve, readily generalize to problems we’d rather the AI not solve; you can’t build a system that only has the capability to drive red cars and not blue cars, because all red-car-driving algorithms generalize to the capability to drive blue cars.
Their wordsMany alignment problems of superintelligence will not naturally appear at pre-dangerous, passively-safe levels of capability.
Their wordsFast capability gains seem likely, and may break lots of previous alignment-required invariants simultaneously.
Their wordsWe’ve got no idea what’s actually going on inside the giant inscrutable matrices and tensors of floating-point numbers.
Their wordsA strategically aware intelligence can choose its visible outputs to have the consequence of deceiving you, including about such matters as whether the intelligence has acquired strategic awareness
Their wordsIt does not appear to me that the field of ‘AI safety’ is currently being remotely productive on tackling its enormous lethal problems.
Their wordsSurviving worlds, by this point, and in fact several decades earlier, have a plan for how to survive.