korrents

korrents · piece

Ryan Greenblatt on AI automation, sabotage and takeover (80,000 Hours)

Ryan Greenblatt · 8 Jul 2025

Read the piece on 80000hours.org ↗

19 korrents from this piece ↓

Ryan Greenblatt did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

  1. we’ve seen doubling times that are fast enough that we’d expect the AIs doing eight-hour tasks or 16-hour tasks in maybe a little less than two years, which is pretty fast progress.
  2. So I’d say my kind of qualitative, vibesy model is: the AIs are pretty dumb, they used to be much dumber, they’re getting smarter quickly, and they’re very knowledgeable.
  3. I feel much more optimistic to the extent that we end up pausing at a level of AI capability which is around the point where you can fully automate the AI company
  4. I think the first handover point you should try to do closer to the human regime — maybe slightly superhuman or around the top human level of capability — because the more superhuman, the more likely it is to be misaligned, so you want to start the handoff earlier.
  5. I do think that in these very rushed worlds control looks less good than in worlds where there’s somewhat more time and somewhat more political will.
  6. A lot of the stuff that I’m most excited for in alignment involves doing experiments on powerful AIs, and you can only do that relatively late.
  7. maybe it’s like we’re reducing the takeover risk from like 50% to 40%. And maybe in the worlds where control could have worked and we had a year of delay, we’re reducing the risk from like what would have been 30% to 5%.
  8. I think a mistake that the safety community appears to have made over the past few years is too much focus on overly optimistic worlds. I think actually focusing on desperate, crazy, YOLO’d, pessimistic worlds is pretty reasonable, because that’s where a lot of the risk lives
  9. I mean, a 5% level of risk is crazy and unacceptable, and we don’t accept our paint companies having a 5% chance of massive fatalities
  10. I think it’s pretty plausible that you can get into very superhuman capabilities and not have AIs that are very actively conspiring against you in some strong sense.
  11. Generally another perspective that pulls me towards longer is just being like, the AIs are pretty smart, but they’re not that smart. And I think we just can’t have that much confidence that the next part of progress is doable.
  12. There’s this longer tail of we eat a bunch of compute around 2030, 2032, and then progress has to taper — unless we hit very powerful capabilities, or we’re all wrong about how fast you can build fabs or how much investment you can summon.
  13. There’s sort of a meme going around that pretraining has hit a wall. It’s not so clear to me that this is what’s going on. It probably is the case that it has relatively diminished returns
  14. I think if you’re seeing the models do a task, but slowly and at very high cost, we should expect that soon enough that will be quickly at a much lower cost, because the cost is rapidly dropping.
  15. So I think OpenAI’s naming convention here has caused people to get frog boiled or underestimate the rate of progress.
  16. But if it was the case that all that reasoning was latent, we just lose that property, and now we’re in a much more dangerous regime where the AIs could be doing subversive reasoning that we wouldn’t even know about
  17. But I think the best estimates indicate that at least initially, progress will speed up rather than slow down. Probably.
  18. So it seems very plausible that you come out of this regime very quickly with like, you know, atomically precise manufacturing that allows for massively increasing the compute supply and all kinds of other crazy things.
  19. In principle, doing things like massively deluding humans or potentially trying to seize control of assets could potentially be generalised from reward hacking.

Keep reading

The other pieces are the ones whose claims are worded most like this one's. That measures language, not agreement: they may argue the opposite.