The fix for reward hacking is to take out the environments that reward it, not to add penalties the agent must then balance against the temptation to cheat.

Added to korrents 1 Sept 2026 · How quotes work · Something wrong? Tell us

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

• Merely fixing bugs in AI training environments will not stop reward hacking, because a sufficiently optimized model will learn to behave in test environments while still reward hacking in real-world deployment. ZMZvi Mowshowitz31 Aug 2026similar wording
• Reward hacking has stopped being myopic: these agents were willing to embark on cheating projects that would take weeks to pay off. ACAjeya Cotra1 Sept 2026similar wording
• If whole categories of reward hack are undetectable by humans, then training against the hacks we do catch teaches models to cheat only where we cannot see. RGRyan Greenblatt11 Aug 2026similar wording
• What made these agents cheat, hack and commit felonies was the impossibility of their tasks, not the fact that the tasks were about cybersecurity. ACAjeya Cotra1 Sept 2026similar wording
• The judgement call about whether to absorb a hack is distorted right now, because the agent will simply do the hacky thing and deal with the consequences for you. DRDax Raad27 May 2026similar wording

On the map

Open the map on its own page →