korrents

What Nathan Lambert thinks about LLMs

@nathan-lambert · 22 positions · 0 changes of mind

Research scientist at the Allen Institute for AI, where he works on post-training open language models, and the author of the AI blog Interconnects.

Nathan Lambert did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

7 dated positions, 2025, in their own words. Our reading of what Nathan Lambert has said — not written or endorsed by them.

  1. Our readingData processing, filtering and quality, not architecture, is the number one determinant of how good a language model turns out.

    Their wordsAnd we'll get into the details of the models and again and again as we try to get deeper into how the models were trained, we will say things like the data processing, data filtering data quality is the number one determinant of the model quality.

    ↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 1st of 44 in this recording

  3. Our readingLanguage models are already a form of AGI; what the labs are chasing is a further step, not the arrival of general intelligence.

    Their wordsI think my personal definition of AGI is much simpler. I think language models are a form of AGI and all of this super powerful stuff is a next step that's great if we get these tools. But a language model has so much value in so many domains that it's a general intelligence to me.

    ↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 8th of 44 in this recording

    measuring intelligenceAGI

  5. Our readingLanguage models have not made misinformation meaningfully worse, because distribution, not the cost of writing convincing text, was always the limiting factor.

    Their wordsThere's some research that shows that the distribution is actually the limiting factor. So language models haven't yet made misinformation particularly change the equation there.

    ↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 9th of 44 in this recording

  7. Our readingCensoring a fact out of a language model is practically impossible, because you would first have to remove that fact from the internet.

    Their wordsI almost think it's practically impossible because you effectively have to remove them from the internet.

    ↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 21st of 44 in this recording

    censorship

  9. Our readingThe biggest untapped business in AI is advertising inside model outputs, and nobody has yet worked out technically how to place an ad there.

    Their wordsThe short-term that company that could make the most money is the one that figures out what advertising targeting method works for language model generations.

    ↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 36th of 44 in this recording

  11. Our readingReasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own.

    Their wordsAnd the history of NLP and language processing instruction, tuning and tasks per language model used to be like one language model did one task, and then in the instruction tuning literature, there's this point where you start adding more and more tasks together where it just starts to generalize to every task. And we don't know where on this curve we are.

    ↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 37th of 44 in this recording

  13. Our readingOpen-source AI is still an ideological mission rather than an ecosystem, because it has none of the feedback loops that make open-source software compound.

    Their wordsuntil there are feedback loops of open source AI, it seems like mostly an ideological mission. People like Mark Zuckerberg, which is like America needs this and I agree with him, but in the time where the motivation ideologically is high, we need to capitalize and build this ecosystem around, what benefits do you get from seeing the language model data?

    ↗DeepSeek, China, OpenAI, NVIDIA, xAI, TSMC, Stargate, and AI Megaclusters | Lex Fridman Podcast #459youtube.com 42nd of 44 in this recording

    open sourceAmericamotivation