korrents

Our readingWhat is a korrent?

Distillation gives a small model a much richer training signal than ordinary pre-training, which weights only the one true next token.

Drawn from what Grant Sanderson said

What this subject means

scaling laws Whether more compute keeps buying more capability, and where the curve now bends -- pre-training, post-training, test-time.

What Grant Sanderson actually said

  1. Grant Sanderson Mathematics explainer, creator of the YouTube channel 3Blue1Brown
    When you think about it, this gives you something much, much richer than ordinary pre-training, since pre-training is just giving full weight to only one true next token, the one that happens to be there.

    Watch from 27:28 plays here

    ↗But what is cross-entropy? | Compression is Intelligence Part 2youtube.com

    16 Jul 2026 · video · 33m · spoken · machine transcript

    All 8 korrents from this recordingthis one is 7th

Added to korrents 8 Nov 2026 · How quotes work · Something wrong? Tell us

Do you hold this korrent?Do you also believe this?

I hold this too

Related korrents

Shared subject first, then similar wording. They may agree or disagree.

See this claim on the map