korrents
Aravind Srinivas

What Aravind Srinivas thinks about scaling laws

@aravind-srinivas · 20 positions · 0 changes of mind

Co-founder and CEO of Perplexity, an AI answer engine; previously a research scientist at OpenAI and a PhD student at UC Berkeley.

Aravind Srinivas did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

4 dated positions, 2024, in their own words. Our reading of what Aravind Srinivas has said — not written or endorsed by them.

4 positions so far — this page is not yet offered to search engines.

  1. Our readingPost-training is what turns a model into a product, but it can only do that on top of a pre-trained model large enough to already have common sense.

    Their wordsAnd without good post-training, you’re not going to have a good product. But at the same time, without good pre-training, there’s not enough common sense to actually have the post-training have any effect.

    ↗Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet | Lex Fridman Podcast #434youtube.com 8th of 20 in this recording

  3. Our readingSpending most of the compute memorising the world in pre-training is brute force; what we want is a model that reasons the way a student sits an open-book exam.

    Their wordsI think there’s an interesting thought experiment here that, we’ve been spending a lot of compute in the pre-training to acquire general common sense, but that seems brute force and inefficient. What you want is a system that can learn like an open book exam.

    ↗Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet | Lex Fridman Podcast #434youtube.com 9th of 20 in this recording

  5. Our readingReal reasoning will have arrived when spending more compute at inference time reliably buys a dramatically better answer.

    Their wordsI think if we can achieve that amount of inference compute, where it leads to a dramatically better answer as you apply more inference compute, I think that will be the beginning of real reasoning breakthroughs.

    ↗Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet | Lex Fridman Podcast #434youtube.com 10th of 20 in this recording

  7. Our readingIf reasoning can be separated from facts, the frontier labs lose their advantage, because nobody needs a million-GPU cluster any more.

    Their wordsBut any breakthrough can disrupt that. If you can decouple reasoning and facts and end up with much smaller models that can reason really well, you don’t need a million H100 equivalent cluster.

    ↗Aravind Srinivas: Perplexity CEO on Future of AI, Search & the Internet | Lex Fridman Podcast #434youtube.com 17th of 20 in this recording