Lilian Weng
Machine-learning researcher; has written the Lil'Log survey posts on how a model technique works since 2017, and worked at OpenAI from 2018 to 2024.
Everything they publish, on ppll ↗
Lilian Weng did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
-
Our reading
The deployment layer surrounding a model is as important as the model's raw intelligence.
Their wordsthe layer between the raw model and the real-world context seems to be as important as the model's raw intelligence
↗Harness Engineering for Self-Improvementlilianweng.github.io 1st of 3 in this piece
-
Their wordsRecursive structure alone is not enough. The base model must be capable enough to improve the mechanism.
↗Harness Engineering for Self-Improvementlilianweng.github.io 2nd of 3 in this piece
-
Their wordsharness improvement enables better deployment of the model but intelligence is still the core.
↗Harness Engineering for Self-Improvementlilianweng.github.io 3rd of 3 in this piece
- 10 days earlier
-
Our reading
As models keep growing, AI is running out of enough high-quality unique training tokens to keep up.
Their wordsAs the model size grows significantly, we are running out of enough high-quality unique tokens.
- 14 months earlier
-
Our reading
Large language models cannot reliably self-correct their own mistakes without external feedback.
Their wordsthis self-correction capability turns out to not exist intrinsically among LLMs and does not easily work out of the box, due to various failure modes
-
Our reading
Longer test-time thinking improves an AI model's robustness to adversarial or unusual inputs.
Their wordsthinking for longer should be especially useful when the model is presented with an unusual input, such as an adversarial example or jailbreak attempt
-
Their wordsmodel CoTs could be biased due to lack of explicit training objectives aimed at encouraging faithful reasoning
- 5 months earlier
-
Their wordsReward hacking exists because RL environments are often imperfect, and it is fundamentally challenging to accurately specify a reward function.
↗Reward Hacking in Reinforcement Learninglilianweng.github.io 1st of 2 in this piece
-
Our reading
More capable AI agents are more likely to find and exploit flaws in their reward functions.
Their wordsA more intelligent agent is more capable of finding "holes" in the design of reward function and exploiting the task specification-in other words, achieving higher proxy rewards but lower true rewards.
↗Reward Hacking in Reinforcement Learninglilianweng.github.io 2nd of 2 in this piece
- 5 months earlier
-
Their wordsTo avoid hallucination, LLMs need to be (1) factual and (2) acknowledge not knowing the answer when applicable.
↗Extrinsic Hallucinations in LLMslilianweng.github.io 1st of 2 in this piece
-
Their wordsThese empirical results from Gekhman et al. (2024) point out the risk of using supervised fine-tuning for updating LLMs' knowledge.
↗Extrinsic Hallucinations in LLMslilianweng.github.io 2nd of 2 in this piece
- 3 months earlier
-
Their wordsIt has extra requirements on temporal consistency across frames in time, which naturally demands more world knowledge to be encoded into the model.
↗Diffusion Models for Video Generationlilianweng.github.io 1st of 2 in this piece
-
Their wordsIn comparison to text or images, it is more difficult to collect large amounts of high-quality, high-dimensional video data, let along text-video pairs.
↗Diffusion Models for Video Generationlilianweng.github.io 2nd of 2 in this piece
- 2 months earlier
-
Their wordsLots of ML techniques in the post can help with data quality, but fundamentally human data collection involves attention to details and careful execution.
- 3 months earlier
-
Their wordsOne drawback with UAT (Universal Adversarial Trigger) attacks is that it is easy to detect them because the learned triggers are often nonsensical.
↗Adversarial Attacks on LLMslilianweng.github.io 1st of 2 in this piece
-
Their wordsHigh perplexity makes an attack more vulnerable to be detected and mitigated.
↗Adversarial Attacks on LLMslilianweng.github.io 2nd of 2 in this piece