-
0:07:40 · watch on youtube.com↗
Their wordsI think that the like standard paradigm of evaluation where you just have some static benchmark um maybe a benchmark with like an LLM judge at the end with a specific rubric is kind of sus and not something that I would be fully happy with in the long term as methods of evaluating frontier systems
- 2 min later
-
0:09:55 · watch on youtube.com↗
Their wordsUm, more generally, I think that um, the easier version of this of just after a post- training run, go look at the RL transcripts and figure out problems with the various environments use and see if you can use this for debugging things. Seems like a pretty good thing to do in general.
- 1 min later
-
0:10:26 · watch on youtube.com↗
Their wordsit seems likely that a bunch of alignment relevant properties will be introduced during RL training. Um, not we should not assume this will be true for all such properties but like it seems plausible especially for the things like goal seeking, reward hacking, things like that.
- 1 min later
-
0:11:03 · watch on youtube.com↗
Their wordsEven if the model later learns to be secretive about it, it seems harder to learn while simultaneously being secretive about it without any prior signal in previous transcripts.
- 5 min later
-
0:16:16 · watch on youtube.com↗
Their wordsAnd yeah, it's great that they checked and reported this prominently because, you know, a really easy way to make a model incredibly safe is to replace it with a rock that always says, "As a large language model, I cannot help you with this."
- 1 min later
-
0:17:36 · watch on youtube.com↗
Their wordsAnd they have like a really strong prior for continuing sentences because you know if you don't continue a sentence properly you normally fail in pre-training which is a strong incentive to really deeply ingrain the reflex to continue sentences.
- 5 min later
-
0:22:54 · watch on youtube.com↗
Their wordsjust read the chain of thought is in my opinion a strong contender for one of our best current safety techniques. I also think it will at some point break and we should be paying really close attention to whether it is breaking
- 1 min later
-
0:24:18 · watch on youtube.com↗
Their wordstypically models currently seem not very good at hiding things in the chain of thought. So the hope is that they are just kind of bad at things like like if they're being deceptive, it will be apparent in the chain of thought.
- 2 min later
-
0:26:15 · watch on youtube.com↗
Their wordsSo, in my opinion, the current safety cases people give models basically boil down to um this model seems too dumb to be super dangerous.
- 5 min later
-
0:30:57 · watch on youtube.com↗
Their wordsUm the scenarios are often like pretty artificial, kind of deranged in the same way that like A human trying hard to get the model to act badly can write pretty wild and messy and unrealistic scenarios
- 2 min later
-
0:32:54 · watch on youtube.com↗
Their wordsI would generally say you should expect these numbers to be underestimates of the eval awareness, especially for more capable models which are going to be more capable of kind of holding things in their head and not saying them.
- 15 min later
-
0:48:22 · watch on youtube.com↗
Their wordsbecause if it's being trained to ignore prompt injections, one of the ways you could achieve this is by being a more deceptive model in general. And it would be good if we don't train our models in a way that incentivizes deception.
- 5 min later
-
0:53:05 · watch on youtube.com↗
Their wordsthe best kind of interpretability deep dives often involve a bunch of different kinds of evidence all pointing in the same direction. There just isn't really an interpretability technique that I trust enough to see it and then be like, great, I am completely satisfied and have no further questions.
- 1 min later
-
0:54:15 · watch on youtube.com↗
Their wordsAnd having two different teams both investigate the same thing and see if they reach consistent conclusions seems even better.
- 5 min later
-
0:59:15 · watch on youtube.com↗
Their wordslike we don't want to reinforce deception in models. That seems bad. That seems the kind of thing that could cause misaligned behavior in general. And thus, if we are training them to do role playing, this plausibly causes badness.
- 7 min later
-
1:06:44 · watch on youtube.com↗
Their wordsI am a big fan of not training the models based on what its thoughts look like.
- 1 min later
-
1:07:54 · watch on youtube.com↗
Their wordsbut if you're using data generated by other models and those models had certain flaws, this could easily get instilled into your model.
- 7 min later
-
1:15:13 · watch on youtube.com↗
Their wordsWe obviously need to seriously consider this hypothesis, but like h seems pretty unlikely that models develop the ability to do this well and to not verbalize it without sometimes screwing up.
- 2 min later
-
1:17:31 · watch on youtube.com↗
Their wordsUm, but I think evaluation awareness is a much bigger deal for like alignment audits because we're trying to measure will it do something, not can it do something.
- 2 min later
-
1:19:03 · watch on youtube.com↗
Their wordsthey removed data and environments rather than trying to like add in more to like monkey patch it out. Uh this seems like the right approach to me. I think that if you try to monkey patch something out, this might work, but this also might suppress it or like only suppress a particular facet of it.
- 11 min later
-
1:29:55 · watch on youtube.com↗
Their wordsthis is like actually quite an important baseline in my opinion because if you steer, especially with a high coefficient, it's easy to just make the model kind of deranged.
- 11 min later
-
1:41:17 · watch on youtube.com↗
Their wordsAI labs are like becoming pretty important places and I think it's plausible that in future they are much more important as these systems become like world changing
- 16 min later
-
1:57:38 · watch on youtube.com↗
Their wordsI generally think thinking of fine-tuning as introducing hard-coded prompts is helpful and underrated.
- 5 min later
-
2:02:44 · watch on youtube.com↗
Their wordsgenerally evaluating model welfare is like pretty cursed because these things are incredibly suggestible and really good at roleplaying and are often trying to do what they think you want them to be doing.
- 3 min later
-
2:06:06 · watch on youtube.com↗
Their wordsI don't really feel that afraid of Opus causing substantial damage, but I think the main loadbearing belief there is that probably it can't like reliably hide things from its chain of thoughts.
- 3 min later
-
2:09:25 · watch on youtube.com↗
Their wordsAnd it seems like we should expect to see like Yeah, we should expect to see like shittier versions before we see like the scary versions.
- 1 min later
-
2:09:58 · watch on youtube.com↗
Their wordsI kind of feel like a lot of what affects how aligned I think current models are are like things like their degree of reward hacking