30 korrents from this recording
Every claim below is a statement made in this recording, quoted word for word and linked to the second it was said, so you can hear it rather than take our word for it. The wording comes from the transcript published alongside the recording; the sentence above each quote is our reading of the claim, not their wording.
0:01:47 · watch on youtube.com↗
Their wordsI think time is a useful model, whether it exists or not. Right. Does quantum physics exist? Well, it doesn’t matter. It’s about whether it’s a useful model to describe reality.
0:04:51 · watch on youtube.com↗
Their wordsIt is actually all their nuances and quirks and slight annoyances that make this relationship worthwhile. I don’t think we’re going to realize that until it’s too late.
0:09:11 · watch on youtube.com↗
Their wordsWhen I talked about will GPT12 be AGI, my answer is no. Of course not. I mean, cross-entropy loss is never going to get you there. You need probably RL in fancy environments in order to get something that would be considered AGI-like.
0:11:35 · watch on youtube.com↗
Their wordsWe’re actually going to build that, I think, but it’s not going to be one static tape. I think the human brain is too complex to be stuck in one static tape like that. If you look at ant brains, maybe they can be stuck on a static tape, but we’re going to build that using generative models. We’re going to build the TikTok that you actually can’t look away from.
0:18:30 · watch on youtube.com↗
Their wordsI think we’re going to build super intelligence before we build any sort of robustness in the AI. We cannot build an AI that is capable of going out into nature and surviving like a bird. A bird is an incredibly robust organism. We’ve built nothing like this. We haven’t built a machine that’s capable of reproducing.
0:21:26 · watch on youtube.com↗
Their wordsThere’s two stacks of life in the world. There’s the biological stack and the silicon stack. The biological stack starts with reproduction. Reproduction is at the absolute core. The first proto-RNA organisms were capable of reproducing. The silicon stack, despite, as far as it’s come, is nowhere near being able to reproduce.
0:52:48 · watch on youtube.com↗
Their wordsIt’s always harder to write software for more specialized hardware. A GPU is pretty generic. If you can’t write an in Nvidia stack, there’s no way you can write a stack for your chip. My approach with Tinygrad is first write a performant NVIDIA stack.
0:54:50 · watch on youtube.com↗
Their wordsmy central thesis about the world is there’s things that centralize power, and they’re bad. There’s things that decentralize power, and they’re good. Everything I can do to help decentralize power, I’d like to do.
0:55:23 · watch on youtube.com↗
Their wordsthe only ASIC that is remotely successful is Google’s TPU. The only reason that’s successful is because Google wrote a machine learning framework. I think that you have to write a competitive machine learning framework in order to be able to build an ASIC.
0:58:39 · watch on youtube.com↗
Their wordsJust dumping the code on GitHub is not open source. Open source is a culture. Open source means that your issues are not all one year old, stale issues. Open source means developing in public.
1:16:46 · watch on youtube.com↗
Their wordsThe problem is if you go back to 1960 and you tell them that you have a machine that can play amazing chess, of course someone in 1960 will tell you that machine is intelligent. Someone in 2010 won’t. What’s changed? Today, we think that these machines that have language are intelligent, but I think in 20 years we’re going to be like, yeah, but can it reproduce?
1:20:23 · watch on youtube.com↗
Their wordsI think things like MuZero and AlphaGo are so much more impressive because these things are playing beyond the highest human level. The language models are writing middle school level essays and people are like, wow, it’s a great essay.
1:21:26 · watch on youtube.com↗
Their wordsI think driving is not tool complete and programming is. Meaning you don’t use the best possible tools to drive. Cars have basically the same interface for the last 50 years. Computers have a radically different interface.
1:22:28 · watch on youtube.com↗
Their wordsSo almost if Codex or copilot are helping you, that actually probably means that your framework or library is bad and there’s too much boilerplate in it.
1:25:06 · watch on youtube.com↗
Their wordsI think future LLMs are going to be smaller, but are going to run looping on themselves and are going to have retrieval systems. And the thing about using a retrieval system is you can cite sources, explicitly.
1:28:03 · watch on youtube.com↗
Their wordsEveryone who talks about AI talks about the button, the button to turn it off, right? Do we have a button to turn off Google? Is anybody in the world capable of shutting Google down?
1:31:33 · watch on youtube.com↗
Their wordsWhat’s ironic about all these AI safety people is they’re going to build the exact thing they fear. We need to have one model that we control and align. This is the only way you end up paper clipped. There’s no way you end up paper clipped if everybody has an AI.
1:40:55 · watch on youtube.com↗
Their wordsAnd I will respect someone philosophically with that position, just like I will respect someone philosophically with the position that nobody should have guns. But I will not respect philosophically with, “Only the trusted authorities should have access to this.”
1:50:06 · watch on youtube.com↗
Their wordsI came into the business world with Comma, and I found the exact opposite. I found 5% of people good and 95% of people bad. I found a world that promotes psychopathy.
1:55:18 · watch on youtube.com↗
Their wordsAs soon as you charge people money, they’re no longer the product, they’re the customer. And then they can start building something that’s good for the customer, and not good for the other customer, which is the ad agencies.
1:57:27 · watch on youtube.com↗
Their wordsIt just predicts what you are likely to re-Tweet and like, and linger on. That’s what all these algorithms do. It’s what Tik-Tok does, it’s what all these recommendation engines do. And it turns out that the thing that you are most likely to interact with is outrage. And that’s a quirk of the human condition.
2:03:22 · watch on youtube.com↗
Their wordsSo, my technical recommendation to Elon, and I said this on the Twitter spaces afterward, I said this many times during my brief internship, was that you need re-factors before features.
2:14:03 · watch on youtube.com↗
Their wordsI mean, the first thing that I would do is build tests. The first thing I would do is get a CI to where people can trust to make changes. Before I touched any code, I would actually say, “No one touches any code. The first thing we do is we test this code base.”
2:22:33 · watch on youtube.com↗
Their wordsBut I mean, would I trade right now the current oligarchy which runs America for the monarchy? Yeah, I would. Sure. For the Elon monarchy, yeah.
2:28:37 · watch on youtube.com↗
Their wordsBut the real thing that I want is not something that like tab completes my code and gives me ideas. The real thing that I want is a very intelligent pair programmer that comes up with a little popup saying, “Hey, you wrote a bug on line 14 and here’s what it is.”
2:37:23 · watch on youtube.com↗
Their wordsOf course, when you talk to an AI that’s made by a big company in the cloud, the AI fundamentally is aligned to them, not to you. And that’s why you have to buy a tiny box. So you make sure the AI stays aligned to you.
2:40:03 · watch on youtube.com↗
Their wordsSo it struck me one day how just silly atheism is. Of course, we were created by God. It’s the most obvious thing.
2:47:25 · watch on youtube.com↗
Their wordsThe two companies that have endured through technology are Apple and Microsoft. And what do they make? Computers and business services, right. All the meme, social ads, they all come and go. But you want to endure, build hardware.
2:58:04 · watch on youtube.com↗
Their wordsUtilitarianism is an abhorrent ideology. Well, yeah, we’re going to kill those three people to save a thousand, of course, right.
3:00:08 · watch on youtube.com↗
Their wordsYou can make survival guaranteed without UBI. What you have to do, is make housing and food dirt cheap. Right? And that’s the good world. And actually, let’s go into what we should really be making dirt cheap, which is energy.