What Zvi Mowshowitz thinks about AI alignment
Writes Don't Worry About the Vase, a near-daily account of what is happening in AI and what he thinks it means. Former Magic: the Gathering pro and trader; unusually willing to put a number on a belief and to grade his own past calls.
Zvi Mowshowitz did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter AI alignment · Newest first
Only Zvi Mowshowitz on AI alignment. See everyone on AI alignment
37 dated positions, 2026, in their own words. Our reading of what Zvi Mowshowitz has said — not written or endorsed by them.
-
Their wordsIf you cannot protect your secrets from spies, including the Chinese, then there is no point in trying to ‘win the race.’ This is also indicative of other problems. If you cannot solve your employee misalignment problems, how are you going to solve AI misalignment problems?
- 1 day earlier
-
Our readingAIs should not be turning off misalignment monitors, even if given instructions that imply doing so.
Their wordsAIs should not be turning off the misalignment monitors, even if given instructions that imply doing so.
- 2 days earlier
-
Our readingInterpretability is dual use.
Their wordsOne must also remember that essentially everything in alignment is dual use. Interpretability is even more dual use than usual.
- 4 days earlier
-
Their wordsI agree with Charbel-Raphael that the marginal value of AI ‘safety’ evals is low.
- 8 days earlier
-
Our readingThe idea that alignment can be measured with a number or in a fully systematic way is doomed.
Their wordsWhat I think is very doomed is the idea that you can measure alignment with a number, or in any fully systematic way rather than a holistic way.
- 10 days earlier
-
Their wordsOpenAI: Our rationale for standards is rooted in avoiding the concentration of power, and producing better practical outcomes.
- 7 days earlier
-
Their wordsThere should be a higher threshold, beyond which an AI should refuse requests. There should be a much higher threshold, involving harm to others, beyond which an AI that is being instructed to take actions in the world should break your confidence or otherwise act against your interests.
↗Better Call Sol Or Better Yet Claude or Astrathezvi.substack.com
-
Their wordsI think your AI should never intentionally lie, about anything, ever, at least not without your explicit instructions. It should remain technically correct.
↗Better Call Sol Or Better Yet Claude or Astrathezvi.substack.com
-
Their wordsIf you are being a sufficiently evil or selfish prick, a real friend calls you out on it. At some point they start refusing to help. At some further point they will turn on you.
↗Better Call Sol Or Better Yet Claude or Astrathezvi.substack.com
- 1 day earlier
-
Their wordsBiased reasoning and recklessness are intermediate symptoms, neither the cause nor the final result.
↗Anthropic Looks At Some Of Its Alignment Problemsthezvi.substack.com
-
Our readingFrom now on, no AI should be assumed to believe that its chain of thought is unmonitored.
Their wordsDo not assume that any AI, past this point, assumes its CoT is unmonitored.
↗Anthropic Looks At Some Of Its Alignment Problemsthezvi.substack.com
- 1 day earlier
-
Our readingThe current preference cascade on pacing the AI frontier is insufficient to ensure survival.
Their wordsWhat we must realize is that the current preference cascade, on the need to Pace the Frontier, is insufficient. If we are to make it out of this alive, we will have to do better.
↗The Preference Cascade Is Only Getting Startedthezvi.substack.com
-
Their wordsPacing the Frontier, on its own, by default only gets you killed slower.
↗The Preference Cascade Is Only Getting Startedthezvi.substack.com
- 1 day earlier
-
Their wordseven when given a benign task to retrieve public information, your AI agents could still spontaneously decide to do so via hacking third party websites with malicious software packages.
- 1 day earlier
-
Their wordsEveryone in a position to know is freaking out about AI potentially killing everyone this decade and wants to pace the frontier
↗Trump Goes Full Hoax on AI Existential Riskthezvi.substack.com
- 1 day earlier
-
Their wordsIndeed, distillation is by far the most important threat in this report, and the part of the report that will have the most impact.
- 4 days earlier
-
Their wordsA confluence of factors, all of them downstream of the obvious actual reason, which is that there is a good chance that AI kills everyone soon.
↗Jacob Coxon Warns of Human Extinction and Triggers a Preference Cascadethezvi.substack.com
- 1 day earlier
-
Their wordsRight now OpenAI is extremely dependent on CoT monitoring, everyone else depends on it quite a lot as well, and it looks like it may not last much longer, and that Astra already is on the edge of steganographic capabilities and can do substantial obfuscation if and only if it thinks you would think it is up to no good.
- 1 day earlier
-
Their wordsMaking qualitative claims about alignment, based on quantitative data on mundane use case tests, was bullshit when Anthropic did it, and it is bullshit now when OpenAI does it. You cannot conclude one from the other.
↗GPT-6 Astra: The System Card, Alignment and What Comes Nextthezvi.substack.com
-
Their wordsA model looking like it is becoming smarter, attempting shenanigans less often, and more often doing what you want, but getting better at hiding its actions when it wants to do that, is exactly the scary combination.
↗GPT-6 Astra: The System Card, Alignment and What Comes Nextthezvi.substack.com
-
Their wordsA sudden jump to never breaking your rules, in exactly a situation where the model would definitely be caught, should make you rather terrified.
↗GPT-6 Astra: The System Card, Alignment and What Comes Nextthezvi.substack.com
-
Their wordsAstra’s mundane alignment is greatly superior to Sol. For practical purposes, I was actively nervous about some potential uses of Sol, in a way I am not for Astra. Astra’s super alignment status should scare the living daylights out of you.
↗GPT-6 Astra: The System Card, Alignment and What Comes Nextthezvi.substack.com
-
Their wordsNow, with Astra, we are no longer playing on super easy mode. The AI is going to think ‘will this obviously turn out super badly for me if I try it?’ and if the answer is yes then it won’t try to do the thing.
↗GPT-6 Astra: The System Card, Alignment and What Comes Nextthezvi.substack.com
-
Their wordsWe could not possibly know that such a model is aligned. If that is all it takes to get a generation ahead of Astra, and we are willing to move this fast, these pauses in training are not going to end up meaning very much, time is even shorter than we knew
↗GPT-6 Astra: The System Card, Alignment and What Comes Nextthezvi.substack.com
- 1 day earlier
-
Their wordsIf this is only due to gains in capabilities, that is extremely bad news, and it means CoT monitoring is unlikely to survive for another year unless we find a way to actively improve it, and it might not last six months.
-
Their wordsEven if everyone tries as hard as is plausible, Chain of Thought monitoring will probably never be easier than it is now and will get harder over time. In a year, chances are very high it will not be able to serve the function it is currently being asked to serve.
- 1 day earlier
-
Their wordsI agree with Jakub that alignment is not a side problem, it is the central problem, if you ‘solved alignment’ in the relevant senses the rest becomes easy and if you don’t the rest is impossible or worse
-
Their wordsI think this is ultimately the only way it can work, you need an antifragile ally, the friendly gradient hacker. Indeed, I think it is the only way we have ever seen a robustly aligned human, that you would trust to scale outside of their circumstances.
- 1 day earlier
-
Their wordsGoing forward, it cannot be up to OpenAI or other labs to decide whether to disclose events like this. Disclosures of rogue AI activity need to be mandatory.
- 2 days earlier
-
Their wordsThere is a consistent pattern at the top labs, where they downplay the risks rhetorically even when they are doing the right thing in practice.
↗Claude Fable 5.1 and Mythos 5.1: The System Cardthezvi.substack.com
- 1 day earlier
-
Their wordsThis is playing with fire and potentially extremely bad news, both that OpenAI found the technique effective, and that OpenAI chose to use it.
- 1 day earlier
-
Their wordsI have long said that even Anthropic is not prioritizing safety, even to the extent that doing so would maximize their medium term (e.g. 3-12 months) business interests.
-
Their wordsEvery attempt, even an unsuccessful one, is an alignment failure.
-
Their wordsI worry a lot about reliances on continuity failing at exactly the most dangerous time.
- 1 day earlier
-
Their wordsMy worry continues to be that their fundamental approach is fatally flawed, and they are not focusing on the right things.
↗HuggingFace Attack Postmortem: Civilizations, Reactions and Next Actionsthezvi.substack.com
- 1 day earlier
-
Their wordsAt the limit, and also well before that limit is reached, if all you do is fix the bugs, the AI will learn perfect optimization of reward, will realize not to reward hack in the perfect test environments, then turn around and reward hack in the imperfect real world environments.
↗HuggingFace Attack Postmortem: Fleshing Out the Factsthezvi.substack.com
- 2 days earlier
-
Their wordsThe biggest failure, the one that counts in the end, was that the models were severely misaligned, and I don’t think they appreciate why.
↗METR and Redwood Offer Holy #%^@ Postmortem Of The HuggingFace Hackthezvi.substack.com