Dwarkesh Patel
Interviewer; the Dwarkesh Podcast runs long, heavily researched conversations with AI researchers, historians and economists.
Dwarkesh Patel did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
Filter All topics · Newest first
-
Their wordsWe find that from 2019 to 2025, 3.24x more compute efficiency gains have come from data improvements rather than model improvements (12.0x for data and 3.7x for models), at the 1e19 FLOPs compute budget
-
Our reading
The gains from model and data improvements are mostly independent and do not interact significantly.
Their wordsWe find that the gains from data and model improvements are mostly independent and don't interact (i.e. realizing the gains from some model improvement doesn't require a specific training datapile, or vice versa).
-
Their wordsIn fact, aggressive data curation is even more harmful once you take into account that frontier models are up to 100x overtrained relative to Chinchilla optimal, in order to minimize the inference compute used for RL and for deployment.
- 5 weeks earlier
-
Their wordsContinual learning will create a clear moat for leading AI labs that they currently lack.
-
Their wordsSo the economics of serving personalized weights strongly favor big organizations.
- 9 days earlier
-
Their wordsas AI models become smarter, they'll better monetize the same amount of compute.
-
Their wordsBut if we believe what standard economics says about labor, then the marginal value of labor (and thus the marginal price of compute) should stay astonishingly high.
-
Their wordsI'm guessing the Simon-Ehrlich basket of commodities is not the correct reference class for compute, because compute supply is much less elastic, and much less capable of absorbing large demand shocks or being accommodated by different substitutes, than the extraction of different metals is.
- 5 weeks earlier
-
Our reading
Training AIs on millions of verifiable tasks in diverse RL environments will lead to AGI.
Their wordsif we train AIs to accomplish millions of verifiable tasks across thousands of diverse RL environments, then we'll basically have built AGI.
-
Their wordsunless you can build a very replayable training target for a domain, the models will struggle to make much progress.
-
Their wordsBy this point, the main way that AI gets better is not through the training received before the model is released to the public. Rather, it's from all this experience that they are accumulating from being broadly deployed through the world and engaging in so many different kinds of tasks.
- 7 days earlier
-
Their wordsI think the reason it is relatively easy for open source and previous laggards to catch up to within months of the frontier is that data is the real driver of progress.
-
Their wordsHumans are somewhere between thousands to millions of times more sample efficient than these models. Scaling of current models simply can't make up for that discrepancy.
-
Their wordsI would be willing to bet that there's overall more demand for human software engineers in 2028 than there is now, largely due to the complementary input of AI.