korrents
Andrej Karpathy

What Andrej Karpathy thinks about neural networks

@andrej-karpathy · 69 positions · On ppll ↗

Founding member of OpenAI and former director of AI at Tesla; creator of nanoGPT and the term "vibe coding".

Andrej Karpathy did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter neural networks · Newest first
Topic
All 69LLMs 26reinforcement learning 8AI agents 7AI and jobs 7coding agents 6neural networks 4AGI 3scaling laws 3AI alignment 2
19 more topics
AI and science 2code generation 2computer vision 2education 2AI and human skill 1developer tools 1documentation 1energy 1government 1ideas vs execution 1Microsoft 1open source 1OpenAI 1optimism 1physics 1product management 1self-driving cars 1software quality 1vibe coding 1
Sort
NewestOldest

Only Andrej Karpathy on neural networks. See everyone on neural networks

4 dated positions, 2025 to 2026, in their own words. Our reading of what Andrej Karpathy has said — not written or endorsed by them.

4 positions so far — this page is not yet offered to search engines.

  1. Our readingAn agent can understand a maximally simplified explanation and still be unable to come up with it — that gap is what is left of the expert's job.

    Their wordsso for example, micro GPT, like I asked I tried to get an agent to write micro GPT. So, I told it like try to boil down the simplest things. Like try to boil down my um neural network training to the simplest thing and it can't do it.

    ↗Skill Issue: Andrej Karpathy on Code Agents, AutoResearch, and the Loopy Era of AIyoutube.com 20th of 20 in this recording

  2. 4 months earlier

  3. Our readingA verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well.

    reinforcement learning

    Their wordsIf a task/job is verifiable, then it is optimizable directly or via reinforcement learning, and a neural net can be trained to work extremely well.

    ↗Verifiabilitykarpathy.bearblog.dev

  4. 4 weeks earlier

  5. Our readingPre-training is a crappy evolution: the practically buildable substitute for the process that gave animals their built-in hardware.

    scaling lawsreinforcement learning

    Their wordsSo that's why I kind of call pre-training this kind of like crappy evolution. It's like the practically possible version with our technology and what we have available to us to get to a starting point where we can actually do things like reinforcement learning and so on.

    ↗Andrej Karpathy — “We’re summoning ghosts, not building animals”youtube.com 5th of 30 in this recording

  7. Our readingThe knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out.

    scaling laws

    Their wordsAnd actually, you don't actually need or want the knowledge. I actually think that's probably actually holding back the neural networks overall because it's actually like getting them to rely on the knowledge a little too much sometimes.

    ↗Andrej Karpathy — “We’re summoning ghosts, not building animals”youtube.com 6th of 30 in this recording