korrents

Our readingWhat is a korrent?

For GPU performance, flops do not matter so much as the whole memory access pattern does, and torch.compile cannot find every optimization.

Drawn from what Andrej Karpathy said

What this subject means

software performance Speed as a mark of quality: bloat, boot times, and software that got slower on hardware that got faster.

What Andrej Karpathy actually said

  1. Andrej Karpathy Founding member of OpenAI and former director of AI at Tesla
    great example I think of being aware of um memory hierarchy the fact that flops don't matter uh the entire memory access pattern matters and that torch compile is amazing but there are many optimizations that are still available to us that potentially torch compile cannot find

    Watch from 2:04:42 plays here

    ↗Let's reproduce GPT-2 (124M)youtube.com

    9 Jun 2024 · video · 4h 01m · spoken · machine transcript

    All 14 korrents from this recordingthis one is 4th

Added to korrents 9 Nov 2026 · How quotes work · Something wrong? Tell us

Do you hold this korrent?Do you also believe this?

I hold this too

Related korrents

Shared subject first, then similar wording. They may agree or disagree.

See this claim on the map