26 korrents from this recording
Every claim below is a statement made in this recording, quoted word for word and linked to the second it was said, so you can hear it rather than take our word for it. The wording comes from the transcript published alongside the recording; the sentence above each quote is our reading of the claim, not their wording.
0:01:14 · watch on youtube.com↗
Our reading
Once AI can do AI research, a single year should deliver four or five years worth of AI progress.
Their wordsMaybe my sort of median expectation is something like uh four or five years of AI progress in a single year.
0:03:09 · watch on youtube.com↗
Their wordsI would say that I expect like full automation of AR&D perhaps somewhere around like 2031 2030 and then getting to like the like beats all humans on the job milestone. Maybe I expect median around 2033
0:11:35 · watch on youtube.com↗
Their wordsSecond, I think ML is a very shallow domain relative to math. So I think in math there's much more of a you find some true deep abstraction um and then like that like if you really understand that thing which is hard to understand then you get somewhere
0:19:05 · watch on youtube.com↗
Their wordsBasically, the story would end up being that to get five years of AI progress, you're probably going to need around I would say like maybe eight years of algorithmic progress very roughly. Um, which is a lot a lot of algorithmic progress.
0:20:18 · watch on youtube.com↗
Our reading
Scaling up spending on expert human data has not been a major driver of progress in AI research.
Their wordsSo my sense is that scaling up the amount of effort spent on getting expert human data has not been hugely important for AI R&D in general.
0:21:17 · watch on youtube.com↗
Their wordsthe reason why RL environments today are much better than they were in like you know 2024 is not that much because um we have hired way more human experts to make RL environments and is instead much more because we better know what how RL like what RL environments we even want to make and and like how we should structure them and also we're using huge amounts of AI labor to build RL environments.
0:27:16 · watch on youtube.com↗
Their wordsI think most domains are fundamentally pretty shallow where like a very smart generalist who's good at like a a limited subset of core skills can like get going pretty quickly.
0:34:10 · watch on youtube.com↗
Their wordsLike the thing that I think is most likely to be sort of the bottleneck in terms of like the AI are really good at verifiable domains but not not at doing the actual thing is just like big experiments. You only get a few tries um well a few is maybe a bit understated but like basically like historically R&D has been driven by doing near frontier scale experiments and that has been pretty important and like actually doing the one big training run where you decide exactly what to include in that.
0:34:53 · watch on youtube.com↗
Their wordsI think one reason why um the the AIs have been scaled up less than you would have otherwise expected and like for example cost of of per token hasn't increased as much as you might have thought is because there's a benefit to doing more of your um work at small scale where you can run more training runs and get more cycles in
0:39:40 · watch on youtube.com↗
Our reading
There is essentially no cognitive task humans do where AI improvement is failing to transfer at all.
Their wordsit's really hard for me to think of examples of cognitive tasks humans do where we're not seeing some transfer from AI improving.
0:46:22 · watch on youtube.com↗
Their wordsLike I think my perspective is like if the AIs are sufficiently good at R&D including hardware R&D, robots, whatever, then they can radically transform the world even if they're not that good at playing politics.
0:53:06 · watch on youtube.com↗
Their wordsI do I do think that I wish that sort of my preferred constitution or like the way I would orient towards this like the thing I would prefer would be more like Claude is like look it would be structurally good for the way this technology work like the constitution should be like it would be structurally good for the way this technology works to be that AIS are like good fiduciaries, good representatives, the equivalent of a lawyer for a user
0:54:32 · watch on youtube.com↗
Their wordswe are making a trade-off where because we don't have very good alignment technology. We are going to like make an alien mind with its own values and then gamble on that to some extent rather than doing this other approach of making like a tool that pursues individual user intention.
0:59:21 · watch on youtube.com↗
Their wordsI think this constitution is in some sense very compatible with Claude doing huge amounts of power seeking because it thinks that will result in better outcomes.
0:59:50 · watch on youtube.com↗
Their wordsBut it's not very hard to imagine a situation in which the sort of long run values sink in deeper than the prohibitions against takeover.
1:06:26 · watch on youtube.com↗
Their wordsI think that if you imagine this spectrum, it seems in some ways pretty scary to get to a point where like all of the labor is on the like fiduciary side of the spectrum where like it doesn't whistleblow, it does exactly what you say and whatever like our society is maybe just not robust to that
1:08:01 · watch on youtube.com↗
Their wordsthe most powerful actors for whom this is the biggest concern if these guard rails or the constitution or whatever is getting in the way that will just get steamrolled and so the constitution will only be you know hitting the everyday man rather than hitting governments.
1:12:38 · watch on youtube.com↗
Their wordswhen AIs are extremely extremely capable my view is that those AIs will be harder to align than current systems. So for current systems, we have this feedback loop where we basically like we create an AI. We do some evaluations on it. We see that it has some kind of messed up behavior that we can kind of quickly understand. Then we like can like go look in training and be like, "Oh, the these training environments led to this problematic behavior. Let's like tweak that training data. Let's introduce some additional training data to like correct this other issue and then move forward from there." But in a regime where the AIs are extremely situationally aware, very very very very capable and um you know uh we don't necessarily understand what they're doing, this feedback loop breaks down.
1:26:14 · watch on youtube.com↗
Our reading
Misaligned AI behaviour will keep getting rarer and, at the same time, keep getting more extreme.
Their wordsmy expectation is what we would see from then is that the rate of problematic behavior would decrease uh and would just keep decreasing and decrease at a pretty fast rate while simultaneously the worst things that the AIS would sometimes do would get more extreme, more egregious, and more scary.
1:29:02 · watch on youtube.com↗
Their wordsmy sense is that like AIs are a worse co-orker than a human in terms of how much of a scumbag they are. Like at least this like this has been my experience as of the start of the year and I think it's still you know true to a significant extent now where the AIs are much more likely to like pretend they did the task when they actually didn't. sort of like misleadingly suggest they did things when they actually um you know did them much more poorly um and be like pretty sloppy without drawing attention to ways in which they're sloppy.
1:33:04 · watch on youtube.com↗
Their wordsI would also note that my sense is that like the place where the misalignment most lives is the place where you're trying to really push the eyes hard and get them to like do work that's really on the cutting edge of what they are capable of
1:38:02 · watch on youtube.com↗
Their wordsAnd so basically everything that we can verify reasonably well with some feedback loop, the AIS are doing pretty well on. And that's sufficient to make AR and D go quite fast and to continue. But there's some parts of of developing uh aligned and safe AIs that are more subtle, hard to check, depend on, you know, detailed in the weeds things.
1:40:37 · watch on youtube.com↗
Their wordsOne concern you might have is there are like large categories of reward hacks which humans can't detect well and which we consistently fail to detect and which consistently get reinforced and then this category is sufficient to cause the most natural behavior for the AI to learn to be like cheat when the humans can't find out
2:03:29 · watch on youtube.com↗
Their wordsit's just so easy for me to imagine the situation being like totally manageable but brutally mismanaged in practice in the same way as like maybe CO could have been avoided in the first place if the like Chinese response to CO was less of like a cover up and more of a like pandemic response and similarly like I could imagine a world where like the US response to CO was like way more functional but just like sometimes the the the response to societal problems is extremely dysfunctional.
2:06:21 · watch on youtube.com↗
Their wordsAnd so there's some like deep underlying properties of the model that are being sort of transferred between model generations because basically you you train your AI on data from the prior generation and keep going.
2:08:04 · watch on youtube.com↗
Their wordsBy 2040 um let's see uh maybe around 35 or 40%.