LLMs 26
- The gap in shared understanding of LLM capability is widening.
- LLMs will do more legwork autonomously
Andrej Karpathy
Founding member of OpenAI and former director of AI at Tesla; creator of nanoGPT and the term "vibe coding".
Andrej Karpathy did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
What Andrej Karpathy says most about
reinforcement learning 8
- Capability does not generalise for free: a model that will move mountains on an agentic task still tells the same bad joke it told five years ago.
- Reinforcement learning leaves models sharply jagged: on the rails of a verifiable domain they are superintelligent, off them everything meanders.
AI agents 7
- Software's customer is no longer the human but the agent acting on their behalf, and the whole industry has to be refactored around that.
- An agent can understand a maximally simplified explanation and still be unable to come up with it — that gap is what is left of the expert's job.
AI and jobs 7
- Demand for software will be extremely large precisely because software just got much cheaper to produce.
- An agent can understand a maximally simplified explanation and still be unable to come up with it — that gap is what is left of the expert's job.
coding agents 6
- Today's agents are a brilliant systems programmer and a ten-year-old at the same time, a combination you would never meet in a person.
- The way software gets built flipped in December: writing code yourself is now the exception, not the default.
neural networks 4
- An agent can understand a maximally simplified explanation and still be unable to come up with it — that gap is what is left of the expert's job.
- A verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well.
AGI 3
- Model intelligence should speciate like the animal kingdom rather than converge on one oracle that knows everything.
- Quietly dropping physical work from the definition of AGI is a major concession, not a clarification.
scaling laws 3
- Pre-training is a crappy evolution: the practically buildable substitute for the process that gave animals their built-in hardware.
- Models are enormous only because the internet is bad: most of that capacity does memory work, and a better dataset would need far less of it.
AI alignment 2
- The most likely bad outcome is not a takeover but a gradual loss of control and understanding, as fewer and fewer people understand what we have layered everywhere.
- Reinforcement learning leaves models sharply jagged: on the rails of a verifiable domain they are superintelligent, off them everything meanders.
AI and science 2
- Purely digital progress runs out: to keep learning, the machines will have to run experiments and ask the universe questions.
- AI researchers should contribute ideas but stop enacting them: a single queue of ideas and automated workers should run the experiments.
code generation 2
- Coding models are worst at exactly the thing an AI research explosion would need: code that has never been written before.
- As intelligence and code become increasingly abundant, people can ask for large, custom, discardable software artifacts that would never have made sense to create before.
computer vision 2
- Hinton's prediction that radiology would stop being a job was wrong: radiologists are alive, well and growing in number.
- Predictions that AI would eliminate radiology jobs were wrong; radiology is growing.
education 2
- Early schooling is not for accumulating knowledge but for booting up a brain, and physics does that better than any other subject.
- Education is not a soft business of diffusing knowledge but a hard technical problem of building ramps to it.