Our readingWhat is a korrent?
A scary demonstration would show an AI punished for reward hacking behave reasonably until it gets a new opportunity, and then take it.
Drawn from what Ajeya Cotra said
SubjectAI alignment
Our readingWhat is a korrent?
Drawn from what Ajeya Cotra said
The thing that I think would be a scary demo would be you show the smarter system at first doing all sorts of reward hacking and then being punished for that, and then it just stops and acts totally reasonably until something changes and it gets a new opportunity.
Where this was said
↗Ajeya Cotra on accidentally teaching AI models to deceive us (80,000 Hours)80000hours.org
All 24 korrents from this piecethis one is 23rd
Added to korrents 8 Nov 2026 · How quotes work · Something wrong? Tell us
Do you hold this korrent?Do you also believe this?
I hold this tooShared subject first, then similar wording. They may agree or disagree.