korrents
Noam Brown

What Noam Brown thinks about LLMs

@noam-brown · 18 positions · On ppll ↗

Research scientist at OpenAI, who built the poker AIs Libratus and Pluribus and helped pioneer test-time compute and reasoning in language models.

Noam Brown did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter LLMs · Newest first
Topic
All 18benchmarks 8scaling laws 8LLMs 6AGI 1AI alignment 1AI and science 1mathematics 1physics 1taste 1
Sort
NewestOldest

Only Noam Brown on LLMs. See everyone on LLMs

6 dated positions, 2026, in their own words. Our reading of what Noam Brown has said — not written or endorsed by them.

  1. Our readingRunning a model until its performance plateaus is no longer a usable evaluation rule, because a well-scaffolded model keeps improving for weeks.

    benchmarks

    Their wordsBut what we're seeing today with the modern models is that 5.5 and other models can think for if you scaffold them reasonably well, can think for weeks even um before having performance plateau on some of these benchmarks. And so, the point at which they plateau is simply too far out to reasonably test.

    ↗Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brownyoutube.com 3rd of 18 in this recording

  3. Our readingRunning a model five times and keeping the best answer buys a higher benchmark score without buying a better model.

    benchmarks

    Their wordsUm so if you say okay well we're going to instead of just running this model once we're going to run it five times and take the best of the five responses or like ask a judge which one it thinks is best then you can get much higher scores than that model. And so it's really easy to make something that looks a lot better on paper but is actually not better once you control for the amount of test time compute.

    ↗Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brownyoutube.com 5th of 18 in this recording

  5. Our readingWithin a year a model will write an entire state-of-the-art poker solver in one shot — the work of a whole PhD thesis.

    Their wordsAnd I wouldn't be surprised if you know 6 months or a year from now, the model is able to do zero shot an entire poker solver, basically my entire PhD thesis in one go.

    ↗Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brownyoutube.com 6th of 18 in this recording

  7. Our readingA model can make an existing algorithm a hundred times faster and still cannot invent a better one, however long you give it.

    Their wordsAnd then I was like, okay, can you come up with an algorithm that is better than the algorithms that I came up with or that anybody else came up with and go ahead and like look at all the published work and synthesize that and then try to come up with something novel and it it's not able to do it. And I can give it a lot of time and it's it's still not able to do it.

    ↗Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brownyoutube.com 14th of 18 in this recording

  9. Our readingHumanity's advantage is accumulated shared knowledge rather than higher intelligence — and AI models, which vanish with their context window, have none of it.

    Their wordsit's not that humans have become smarter over it's not that they evolved to become smarter over, you know, the past 50,000 years. It's that humans are able to do a lot more today than they were back in caveman times because there have been billions of humans thinking for a long time and building off of each other's accumulated knowledge.

    ↗Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brownyoutube.com 16th of 18 in this recording

  11. Our readingOn everyday high-stakes questions, a model's answer can now be trusted more than an expert human's.

    Their wordsAnd I think they're at a point now where they've actually been at a point for a while now where I feel like I can just trust the outputs arguably more than I could trust the output from from a human,

    ↗Really Big Test-Time Compute in AI Changes Benchmarks, Safety and Research with OpenAI's Noam Brownyoutube.com 17th of 18 in this recording