korrents
George Hotz

What George Hotz thinks about LLMs

@george-hotz · 30 positions · On ppll ↗

American programmer and hacker, founder of the self-driving company Comma.ai and of the tiny corp, which builds the tinygrad neural network framework.

George Hotz did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter LLMs · Newest first
Topic
All 30AI alignment 4LLMs 4developer tools 3coding agents 2Google 2Microsoft 2social media 2software quality 2AGI 1
16 more topics
AI agents 1AI and jobs 1America 1Apple 1code generation 1energy 1government 1housing 1measuring intelligence 1open source 1physics 1reinforcement learning 1religion 1self-driving cars 1semiconductors 1the cloud 1
Sort
NewestOldest

Only George Hotz on LLMs. See everyone on LLMs

4 dated positions, 2023, in their own words. Our reading of what George Hotz has said — not written or endorsed by them.

4 positions so far — this page is not yet offered to search engines.

  1. Our readingScaling up GPT will never produce AGI, because a model trained on cross-entropy loss cannot get there; reinforcement learning in rich environments is required.

    AGIreinforcement learning

    Their wordsWhen I talked about will GPT12 be AGI, my answer is no. Of course not. I mean, cross-entropy loss is never going to get you there. You need probably RL in fancy environments in order to get something that would be considered AGI-like.

    ↗George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387youtube.com 3rd of 30 in this recording

  3. Our readingAlphaGo and MuZero are far more impressive than large language models, because they play beyond the best humans while the models write middle-school essays.

    Their wordsI think things like MuZero and AlphaGo are so much more impressive because these things are playing beyond the highest human level. The language models are writing middle school level essays and people are like, wow, it’s a great essay.

    ↗George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387youtube.com 12th of 30 in this recording

  5. Our readingIf Copilot is meaningfully helping you write code, that is evidence your framework or library is bad and full of boilerplate.

    Microsoft

    Their wordsSo almost if Codex or copilot are helping you, that actually probably means that your framework or library is bad and there’s too much boilerplate in it.

    ↗George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387youtube.com 14th of 30 in this recording

  7. Our readingFuture language models will be smaller and loop on themselves with a retrieval system, which is what will finally let them cite their sources.

    Their wordsI think future LLMs are going to be smaller, but are going to run looping on themselves and are going to have retrieval systems. And the thing about using a retrieval system is you can cite sources, explicitly.

    ↗George Hotz: Tiny Corp, Twitter, AI Safety, Self-Driving, GPT, AGI & God | Lex Fridman Podcast #387youtube.com 15th of 30 in this recording