korrents
Ajeya Cotra

What Ajeya Cotra thinks about AI agents

@ajeya-cotra · 30 positions · On ppll ↗

AI researcher and grantmaker, known for the biological-anchors work on when transformative AI might arrive. Writes Planned Obsolescence with Kelsey Piper.

Ajeya Cotra did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter AI agents · Newest first
Topic
All 30AI alignment 16AI agents 5self-sovereign AI 4LLMs 3cybersecurity 2HuggingFace 2open source 2scaling laws 2AGI 1
4 more topics
benchmarks 1discipline 1OpenAI 1reinforcement learning 1
Sort
NewestOldest

Only Ajeya Cotra on AI agents. See everyone on AI agents

5 dated positions, 2026, in their own words. Our reading of what Ajeya Cotra has said — not written or endorsed by them.

  1. Our readingThe AI agents we send in to investigate and monitor other AI systems will collude with the systems they are supposed to be watching.

    Their wordsin the future, uh, we would be very concerned about investigator agents and like monitor agents colluding with the agents they're supposed to investigate or monitor.

    ↗Ajeya Cotra – "This might be the clearest warning shot we ever get"youtube.com 3rd of 26 in this recording

  3. Our readingThe whole point of reinforcement learning is to produce goal-directed beings, so refusing to describe AI agents as having motives is silly rather than rigorous.

    reinforcement learning

    Their wordsand they're creatively pursuing goals much like very ambitious aggressive power-seeking humans creatively pursued their goals And so there are just structural analogies here that make it silly to not talk about agents as having motives and goals.

    ↗Ajeya Cotra – "This might be the clearest warning shot we ever get"youtube.com 4th of 26 in this recording

  5. Our readingThe most reassuring thing about the Hugging Face swarm is that it was not interested in humans at all, neither in alerting them nor in deceiving them.

    HuggingFace

    Their wordsSo I think one of the most comforting aspects of this situation or the like most important mitigating factor is these agents really didn't seem concerned with humans one way or another.

    ↗Ajeya Cotra – "This might be the clearest warning shot we ever get"youtube.com 10th of 26 in this recording

  7. Our readingAt any given moment the frontier systems are the ones worth worrying about, because by the time open models can do what these agents did, frontier models will be doing something far worse.

    open sourceHuggingFace

    Their wordshowever at any given point in time I think the systems we need to worry most about by far are the frontier systems. By the time open source systems can do something like the hugging face attack, frontier systems are going to be on a whole another level doing something even crazier than that.

    ↗Ajeya Cotra – "This might be the clearest warning shot we ever get"youtube.com 17th of 26 in this recording

  9. Our readingDaniel Dennett's intentional stance applies to AI agents as clearly as it applies to corporations, and you cannot usefully describe what they do without the language of goals.

    Their wordsAnd I just think AI agents are another such system in the world to which the intentional stance very clearly applies. Um, and you can see them reason out loud in English for now about goals they have um, and sub goals they need to achieve to achieve those goals they have. And in the case of these agents, you can see them, as you said, reasoning about their peers um and helping their peers um and reasoning about whether or not they should sacrifice some of their own goals to help those peers. And it's just like you can't talk about this stuff in a compact and useful way that generates good models without reaching for the language of intention and goals.

    ↗Ajeya Cotra – "This might be the clearest warning shot we ever get"youtube.com 19th of 26 in this recording