korrents

What Buck Shlegeris thinks about AI alignment

@buck-shlegeris · 8 positions · 0 changes of mind

CEO of Redwood Research, where he works on AI control: making sure a powerful model cannot cause a catastrophe even if it is misaligned and trying to.

Buck Shlegeris did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

5 dated positions, 2024, in their own words. Our reading of what Buck Shlegeris has said — not written or endorsed by them.

  1. Our readingLabs should make powerful models safe to deploy even if those models are misaligned and actively trying to subvert the safeguards.

    Their wordsThat is, labs should make sure that the safety measures they apply to their powerful models prevent unacceptably bad outcomes, even if the AIs are misaligned and intentionally try to subvert those safety measures.

    ↗The case for ensuring that powerful AIs are controlledredwoodresearch.substack.com 1st of 8 in this piece

    Interesting

  3. Our readingWhether a model is controlled can be settled with capability evaluations, which makes control far easier to check than alignment.

    Their wordsBecause evaluating control just requires evaluating capabilities, it's far easier to robustly evaluate than alignment.

    ↗The case for ensuring that powerful AIs are controlledredwoodresearch.substack.com 4th of 8 in this piece

    Interesting

  5. Our readingRisk from a scheming model should be managed the way insider security risk is: assume some of them are compromised and stay safe anyway.

    Their wordsWe're advocating that companies handle risk from scheming models in a similar way–striving to ensure that they'll be safe even if their alignment efforts fail to prevent models from scheming.

    ↗The case for ensuring that powerful AIs are controlledredwoodresearch.substack.com 5th of 8 in this piece

    Interesting

  7. Our readingNo observation of a model's behaviour can establish that it is aligned, because a model capable enough to be dangerous is capable enough to produce whatever behaviour it is being watched for.

    Their wordsThe basic problem with evaluating alignment is that no matter what behaviors you observe, you have to worry that your model is just acting that way in order to make you think that it is aligned.

    ↗The case for ensuring that powerful AIs are controlledredwoodresearch.substack.com 7th of 8 in this piece

    Interesting

  9. Our readingControl is a fundamentally limited approach that stops working on sufficiently powerful models, so alignment is still needed in the end.

    Their wordsAI control (with only black-box techniques) seems like a fundamentally limited approach.

    ↗The case for ensuring that powerful AIs are controlledredwoodresearch.substack.com 8th of 8 in this piece

    Interesting