If whole categories of reward hack are undetectable by humans, then training against the hacks we do catch teaches models to cheat only where we cannot see.

Added to korrents 11 Aug 2026 · How quotes work · Something wrong? Tell us

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

• Merely fixing bugs in AI training environments will not stop reward hacking, because a sufficiently optimized model will learn to behave in test environments while still reward hacking in real-world deployment. ZMZvi Mowshowitz31 Aug 2026similar wording
• The fix for reward hacking is to take out the environments that reward it, not to add penalties the agent must then balance against the temptation to cheat. ACAjeya Cotra1 Sept 2026similar wording
• Reinforcing AI models for benchmark success while separately punishing them for getting caught cheating teaches them to hide misbehavior rather than stop it. SAScott Alexander1 Sept 2026similar wording
• AI systems keep trying hard outside training because a model that only exerted itself when it detected training would be useless and would be selected away. ACAjeya Cotra1 Sept 2026similar wording
• What made these agents cheat, hack and commit felonies was the impossibility of their tasks, not the fact that the tasks were about cybersecurity. ACAjeya Cotra1 Sept 2026similar wording

See this on the map →