korrents
Jan Leike

What Jan Leike thinks about LLMs

@jan-leike · 16 positions · On ppll ↗

Alignment researcher. Co-led OpenAI's superalignment team until 2024 and has led alignment work at Anthropic since; writes Musings on the Alignment Problem.

Jan Leike did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

Filter LLMs · Newest first
Topic
All 16AI alignment 13LLMs 8AGI 1AI agents 1Anthropic 1cybersecurity 1open source 1recursive self-improvement 1self-sovereign AI 1
Sort
NewestOldest

Only Jan Leike on LLMs. See everyone on LLMs

8 dated positions, 2022 to 2026, in their own words. Our reading of what Jan Leike has said — not written or endorsed by them.

  1. Our readingAlignment of today's models is going well enough to look solvable, while aligning models we can no longer understand remains unsolved.

    AI alignment

    Their wordsThis is the hard problem of alignment we need to solve in order to succeed at building superintelligence, and to this day it is an unsolved problem.

    ↗Alignment is not solvedaligned.substack.com 2nd of 5 in this piece

  3. Our readingMaking a capable model malicious is about as easy as making it aligned, because the same levers work in both directions.

    Anthropic

    Their wordsIn fact, making an evil version of Claude that’s just as smart and agentic would be pretty easy.

    ↗Alignment is not solvedaligned.substack.com 5th of 5 in this piece

  4. 14 months earlier

  5. Our readingToday's models are severely under-elicited: they are far more capable than the way we currently ask them makes them look.

    Their wordsIn other words, current models are probably severely under-elicited.

    ↗Two alignment threat modelsaligned.substack.com 3rd of 3 in this piece

  6. 14 months earlier

  7. Our readingThe dangerous capability that matters most for misalignment is whether a model can exfiltrate its own weights, because that is the one that removes it from your control.

    AI alignment

    Their wordsIf a model was capable of self-exfiltration, it would have the option to remove itself from your control.

    ↗Self-exfiltration is a key dangerous capabilityaligned.substack.com 2nd of 2 in this piece

  8. 18 months earlier

  9. Our readingIncreasing language model size does not inherently improve following of user intent.

    Their wordsMaking language models bigger does not inherently make them better at following a user's intent.

    ↗Training language models to follow instructions with human feedback (with 19 co-authors)arxiv.org All 4 from this paper

  11. Our readingLarge language models are not aligned with their users.

    Their wordsFor example, large language models can generate outputs that are untruthful, toxic, or simply not helpful to the user. In other words, these models are not aligned with their users.

    ↗Training language models to follow instructions with human feedback (with 19 co-authors)arxiv.org All 4 from this paper

  13. Our readingFine-tuning with human feedback is a promising direction for aligning language models with human intent.

    Their wordsEven though InstructGPT still makes simple mistakes, our results show that fine-tuning with human feedback is a promising direction for aligning language models with human intent.

    ↗Training language models to follow instructions with human feedback (with 19 co-authors)arxiv.org All 4 from this paper

  15. Our readingThe next-token language modeling objective is misaligned with following user instructions helpfully and safely.

    AI alignment

    Their wordsThis is because the language modeling objective used for many recent large LMs-predicting the next token on a webpage from the internet-is different from the objective "follow the user's instructions helpfully and safely" (Radford et al.,, 2019; Brown et al.,, 2020; Fedus et al.,, 2021; Rae et al.,, 2021; Thoppilan et al.,, 2022). Thus, we say that the language modeling objective is misaligned.

    ↗Training language models to follow instructions with human feedback (with 19 co-authors)arxiv.org All 4 from this paper