Our readingWhat is a korrent?
Matching vocabularies across model families for soft distillation is super hacky, so most people just do hard distillation.
Drawn from what Sebastian Raschka said
SubjectLLMs
Our readingWhat is a korrent?
Drawn from what Sebastian Raschka said
Of course, there are hacks and workaround you can kind of synchronize these vocabularies and kind of make them match up and stuff like that. But it's it's super hacky and tricky. And so most people in practice, they just do hard distillation because it's just easier than doing this.
Where this was said
Watch from 0:08:47 plays here↗Build A Reasoning Model From Scratch 8: Distillationyoutube.com
All 23 korrents from this recordingthis one is 3rd
Added to korrents 9 Nov 2026 · How quotes work · Something wrong? Tell us
Do you hold this korrent?Do you also believe this?
I hold this tooShared subject first, then similar wording. They may agree or disagree.