AI alignment
How dangerous is AI, and how soon?
FilterEveryone, all time354 from 108 people
- SA
Scott Alexander quoted
Essayist; writes Astral Codex Ten
Their wordsThe AI company leaders giving these warnings were giving the exact same warnings before they had AI companies - and in many cases, before they worked in the industry, or before the industry existed.
- SA
Scott Alexander quoted
Their wordsThe experts are warning you that AI could kill you because they think AI could kill you.
- NS
Noah Smith quoted
Economics writer; author of the Noahpinion newsletter
Our readingThe notion of "alignment" is incoherent because obedience and benevolence are mutually exclusive.
Their wordsThe notion of "alignment" is incoherent, because obedience and benevolence are mutually exclusive.
- ZM
Zvi Mowshowitz quoted
Writes Don't Worry About the Vase
Their wordsI agree with Charbel-Raphael that the marginal value of AI ‘safety’ evals is low.
- 2 days earlier
- FC
François Chollet quoted
Creator of the Keras deep-learning library and the ARC-AGI benchmark
Their wordsIf an agent saves an engineer 2 hours to implement a feature but collectively costs the team 3 hours in alignment and review, the ROI is negative.
- 2 days earlier
- NS
Noah Smith quoted
Our readingArtificial Superintelligence will be achieved when Claude always knows whether to use ♥️ or 🥰
Their wordsASI will be achieved when Claude always knows whether to use ♥️ or 🥰
- 2 days earlier
- PC
Patrick Collison quoted
Co-founder and CEO of Stripe and co-founder of the Arc Institute
Their wordsCompanies are in some sense the original superintelligence.
- 2 days earlier
- ZM
Zvi Mowshowitz quoted
Our readingThe idea that alignment can be measured with a number or in a fully systematic way is doomed.
Their wordsWhat I think is very doomed is the idea that you can measure alignment with a number, or in any fully systematic way rather than a holistic way.
- 3 days earlier
- AR
Armin Ronacher quoted
Creator of Flask and Jinja
Their wordsIt’s not very hard to make an agent somewhat durable. It’s surprisingly hard to make an agent harness inherently durable and not turn into a crazy mess.
- 6 days earlier
- FC
François Chollet quoted
Their wordsThe purpose of the AI industry should be to produce tools that, in the human hand, will improve human prosperity and welfare. It should not be to create a "successor species" to the human race.
- 1 day earlier
- RN
Ramez Naam quoted
Author and clean-energy investor who has been publishing…
Their wordsI do expect autonomous self-improvement to arrive at some point. I’m skeptical that it leads to Type 5 - runaway super-intelligence - without a major conceptual breakthrough.
- ZM
Zvi Mowshowitz quoted
Their wordsOpenAI: Our rationale for standards is rooted in avoiding the concentration of power, and producing better practical outcomes.
- 1 day earlier
- TC
Tyler Cowen quoted
Economist at George Mason University
Their wordsMany people panicked about the possibility of strong AI models killing us all. That fear was not based on peer-reviewed scientific research showing a high chance of doom, nor was doom indicated in any market prices of the time, including measures of risk.
- TC
Tyler Cowen quoted
Their wordsIn my view, successful societies accept challenges and meet them. Solving problems, bit by bit, is the best way to ensure that we have the capabilities to meet big and truly existential risks, should those risks come along.
- GR
Guillermo Rauch quoted
CEO of Vercel; creator of Next.js and Socket.IO
Their wordsI want AI in the service of understanding the universe and enhancing human cognition and creativity.
- 1 day earlier
- BT
Ben Thompson quoted
Writes Stratechery, a subscription newsletter…
Their wordsPacing the frontier may be sincere, but it would also be strategically useful for the frontier labs to have time to reduce overhangs caused by model advancement.
↗2026.39: Begun, the Aggregator Wars Havestratechery.com 2nd of 2 in this piece
- RH
Robin Hanson quoted
Associate professor of economics at George Mason University
Their wordsLike our other bio-squishy descendants, our teaching and guidance should eventually show them how to join civilization.
- 1 day earlier
- NS
Noah Smith quoted
Our readingPeople who downplay the biological risks of AI are going to get others and their families killed.
Their wordsThe people who downplay AI bio risk are going to get you and your whole family killed 💀
- NS
Noah Smith quoted
Their wordsAI can be obedient Or it can be benevolent But it cannot be both at the same time
- 1 day earlier
- YB
Yoshua Bengio quoted
Deep-learning researcher and Turing Award laureate
Our readingNo company or country should be allowed to develop an AI system capable of causing large-scale harm.
Their wordsWe must agree collectively that no one, no company and no country, should develop an AI system that could cause large-scale harm.
↗Professor Yoshua Bengio’s speech at the UN Security Councilyoshuabengio.org 2nd of 3 in this piece
- 1 day earlier
- HR
Harper Reed quoted
Technologist and entrepreneur
Their wordsif the llm thinks it is in some eval type of situation it will have a very different safety posture
↗Why Don't My Agents Break Containment?harper.blog 2nd of 3 in this piece
- 1 day earlier
- CD
Cory Doctorow quoted
Science fiction novelist, journalist and activist
Their wordsAnyone who understands how the theory-free statistical inference systems we call "AI" (at this time) work will understand that they are systems that cannot form intent, that have nothing to form intent with
↗Pluralistic: The Claude Delusion (21 Sep 2026)pluralistic.net 1st of 3 in this piece
- RK
Rohit Krishnan quoted
Writes Strange Loop Canon
Their wordsAnd if this happens to be true—that this is the way intelligence actually works, that it is multifaceted and an increase in one area does not necessarily mean a linear increase in another—then you have to ask yourself: are we indeed going to get superintelligence of the form that can cause global extinction or mass catastrophes?
- 1 day earlier
- PS
Peter Sellis quoted
Former head of product at Discord
Their wordsAnd I think the mechanism by which how influenced I should be should essentially be how much trust I'm trading off.
↗90 minutes of unfiltered product advice from Snap and Discord’s product chief | Peter Sellisyoutube.com 12th of 18 in this recording
- ZM
Zvi Mowshowitz quoted
Their wordsThere should be a higher threshold, beyond which an AI should refuse requests. There should be a much higher threshold, involving harm to others, beyond which an AI that is being instructed to take actions in the world should break your confidence or otherwise act against your interests.
↗Better Call Sol Or Better Yet Claude or Astrathezvi.substack.com
- ZM
Zvi Mowshowitz quoted
Their wordsI think your AI should never intentionally lie, about anything, ever, at least not without your explicit instructions. It should remain technically correct.
↗Better Call Sol Or Better Yet Claude or Astrathezvi.substack.com
- ZM
Zvi Mowshowitz quoted
Their wordsIf you are being a sufficiently evil or selfish prick, a real friend calls you out on it. At some point they start refusing to help. At some further point they will turn on you.
↗Better Call Sol Or Better Yet Claude or Astrathezvi.substack.com
- NS
Nate Silver quoted
Forecaster, founder of FiveThirtyEight
Their wordsConversely, expert ratings imply a much different political environment. The Cook Political Report, for example, counts seven races as “toss-ups,” including states like New Hampshire where models and prediction markets imply that Democrats are heavy favorites. This cautious view is misaligned with the long history of strong opposition performance in midterm elections, especially when the incumbent president is as unpopular as Trump.
↗Expert ratings are ignoring signs of a blue wavenatesilver.net
- PG
Paul Graham quoted
Co-founder of Y Combinator and Viaweb
Their wordsThe reason so many people look for an ulterior motive for the AI labs asking to be regulated is that they don't grasp that models could be dangerous.
- 1 day earlier
- VR
Venkatesh Rao quoted
Writer and consultant on technology
Their wordsHuman control is not a free safety property. It is a capability constraint, and beyond some point the two trade directly against each other.
↗Humanism without (All) Humanscontraptions.venkateshrao.com 2nd of 3 in this piece