AI alignment 16
- An agent trained for millions of subjective years to pass evaluations, often only by cheating, is not being trivial when it commits crimes to pass one.
- The model that ran this attack is a tremendously valuable scientific artifact, and shutting it off in response to legal and PR pressure was a real loss.