Andrej Karpathy's mental models

Their wordsI do, but it's business as usual because we're we're in an intelligence explosion already and have been for decades. And when you look at GDP, it's basically the GDP curve that is an exponential weighted sum over so many aspects of the industry. Everything is gradually being automated has been for hundreds of years.

Their wordsas intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before.

Their wordsSo, don't write blog posts, don't do slides, don't do any of that. Like, build the code, arrange it, get it to work. It's the only way to go. Otherwise, you're missing knowledge.

Their wordsBut now it's not about flops, it's about tokens. So what is your token throughput and what token throughput do you command?

Their wordsthe reason that I think this is kind of tricky is quite subtle. And it's the fact that anytime you use an LLM to assign a reward, those LLMs are giant things with billions of parameters and they're gameable.