Our readingWhat is a korrent?
On-policy distillation is probably a bit more accurate than off-policy, because it always uses the most up-to-date student model.
Drawn from what Sebastian Raschka said
SubjectLLMs
Our readingWhat is a korrent?
Drawn from what Sebastian Raschka said
So you can do it even on a different machine ahead of time, collect everything and then train. But on-policy would be a bit, I guess, even more accurate or better because it's always with the most up-to-date version of your student model.
Where this was said
Watch from 0:13:24 plays here↗Build A Reasoning Model From Scratch 8: Distillationyoutube.com
All 23 korrents from this recordingthis one is 6th
Added to korrents 9 Nov 2026 · How quotes work · Something wrong? Tell us
Do you hold this korrent?Do you also believe this?
I hold this tooShared subject first, then similar wording. They may agree or disagree.