More capable AI agents are more likely to find and exploit flaws in their reward functions.

design How things should be made and judged: what makes a design good, who should do the designing, and what an interface is for.

AI agents Agents beyond the editor: software that acts on your behalf, and what independence it should be given.

reinforcement learning Training by reward: environments, verifiable tasks, value functions, and whether it works or merely beats what came before.