No observation of a model's behaviour can establish that it is aligned, because a model capable enough to be dangerous is capable enough to produce whatever behaviour it is being watched for.

Added to korrents 10 Jun 2022 · How quotes work · Something wrong? Tell us

Where this collides

• Cannot both be true

  Anthropic now uses mechanistic interpretability techniques to audit new models for deception and scheming before releasing them.

  Amodei reports that Anthropic audits new models for deception and scheming before release. Yudkowsky and Shlegeris hold that no observation of a model behaviour can establish that it is aligned, because a model capable enough to matter can choose its visible outputs. An audit is an observation, and one of these claims says it cannot do the job it is being used for.

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

• A human being is not an AGI: we lack a huge amount of knowledge and rely on continual learning instead, so continual learning is what superintelligence should mean. ISIlya Sutskever25 Nov 2025AI alignment
• A superintelligence needs no consciousness, emotions or malice to be dangerous; a goal system slightly misaligned with ours is enough. ÉTÉmile P. Torres24 Oct 2017AI alignment
• AI safety advocates will build the very thing they fear: one controlled, aligned model is the only route to being turned into paperclips. GHGeorge Hotz29 Jun 2023AI alignment
• Alignment failures that matter will not show up at safe capability levels, because the reason to hide misbehaviour only exists once hiding it pays. EYEliezer Yudkowsky10 Jun 2022AI alignment
• Alignment has to be right on the first try at a dangerous level of capability, because failing at that level leaves nobody to try again. EYEliezer Yudkowsky10 Jun 2022AI alignment

See this on the map →