korrents

On the map

Tap a claim on the ring to put it at the centre.

← Flash attention does more flops than standard attention yet runs significantly faster.

17 connected korrents · 15 moments from 12 Jun 2017 to 19 Oct 2026.

Everything filed under transformers transformers Everything filed under software performance software performance Everything filed under semiconductors semiconductors Everything filed under AI agents AI agents Everything filed under benchmarks benchmarks Everything filed under coding agents coding agents Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Flash attention does more flops than standard attention yet runs significantly faster. Flash attention does more flops thanstandard attention yet runs significantlyfaster. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: A brain keyboard may go the way of AR glasses: attention was already fully occupied, so moving the interface closer changes nothing. — tap to centre the map on it A brain keyboard may go the way ofAR glasses: attention was alreadyfully occupied, so moving theinterface closer changes nothing. Last stated 3 months ago 20 Aug 2026 MH Max Hodak — holds since 2026-08-20 — tap for who they are Same subject: For GPU performance, flops do not matter so much as the whole memory access pattern does, and torch.compile cannot find every optimization. — tap to centre the map on it For GPU performance, flops do notmatter so much as the whole memoryaccess pattern does, andtorch.compile cannot find everyoptimization. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches. — tap to centre the map on it Research on transformer-styleattention mechanisms has not yetreached diminishing returns comparedto alternative approaches. Last stated 5 years ago 3 Jun 2021 GW Gwern — holds since 2021-06-03 — tap for who they are Same subject: The scarce skill is now context switching across parallel agents, not sustained deep work. — tap to centre the map on it The scarce skill is now contextswitching across parallel agents,not sustained deep work. Last stated 9 months ago 4 Mar 2026 BC Boris Cherny — holds since 2026-03-04 — tap for who they are Same subject: Criticism that a simpler, scaled-up architecture underperforms a heavily hand-optimized baseline actually demonstrates why that architecture-scaling research matters. — tap to centre the map on it Criticism that a simpler, scaled-uparchitecture underperforms a heavilyhand-optimized baseline actuallydemonstrates why thatarchitecture-scaling researchmatters. Last stated 5 years ago 11 Jun 2021 GW Gwern — holds since 2021-06-11 — tap for who they are Same subject: CPU performance keeps improving quickly rather than stagnating. — tap to centre the map on it CPU performance keeps improvingquickly rather than stagnating. Last stated 2 months ago 19 Sept 2026 DL Daniel Lemire — holds since 2026-09-19 — tap for who they are Same subject: Inference, not training, is now the constraint, and specialized low-energy hardware will beat general-purpose GPUs and TPUs on latency. — tap to centre the map on it Inference, not training, is now theconstraint, and specializedlow-energy hardware will beatgeneral-purpose GPUs and TPUs onlatency. Last stated 4 months ago 30 Jul 2026 JD Jeff Dean — holds since 2026-07-30 — tap for who they are Same subject: Most people multitasking with coding agents would do better to slow down and give their top priority their full attention. — tap to centre the map on it Most people multitasking with codingagents would do better to slow downand give their top priority theirfull attention. Last stated a month ago 19 Oct 2026 TS Thariq Shihipar — holds since 2026-10-19 — tap for who they are Same subject: "Is the page loaded?" was never a real question, and web performance only became measurable once it was broken into the distinct moments a user actually notices. — tap to centre the map on it "Is the page loaded?" was never areal question, and web performanceonly became measurable once it wasbroken into the distinct moments auser actually notices. Last stated 3 months ago 19 Aug 2026 AO Addy Osmani — holds since 2026-08-19 — tap for who they are Same subject: "Optimize it later" would be safe advice only if every engineer already knew how to avoid the architectural mistakes that no later optimization can remove. — tap to centre the map on it "Optimize it later" would be safeadvice only if every engineeralready knew how to avoid thearchitectural mistakes that no lateroptimization can remove. Last stated 3 months ago 26 Aug 2026 CM Casey Muratori — holds since 2026-08-26 — tap for who they are Same subject: A 300-millisecond response budget is not an achievement but an admission: 300 milliseconds is an eternity in computing, and a product pitching it shows how far the bar has fallen. — tap to centre the map on it A 300-millisecond response budget isnot an achievement but an admission:300 milliseconds is an eternity incomputing, and a product pitching itshows how far the bar has fallen. Last stated 3 months ago 26 Aug 2026 CM Casey Muratori — holds since 2026-08-26 — tap for who they are Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it A single AI paper rarely changeseverything; the last one that didwas 'Attention Is All You Need' in2017. Last stated 2 years ago 6 Jan 2025 AR Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can relyentirely on self-attention for inputand output representations withoutsequence-aligned RNNs orconvolution. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context. — tap to centre the map on it As reasoning models and agentsretain more tokens for longer,KV-cache size and attention costbecome the main constraint onscaling context. Last stated 6 months ago 16 May 2026 SR Sebastian Raschka — holds since 2026-05-16 — tap for who they are Same subject: Linear probes are very strong classifiers, and attention-head probes are even stronger. — tap to centre the map on it Linear probes are very strongclassifiers, and attention-headprobes are even stronger. Last stated 11 months ago 20 Dec 2025 NN Neel Nanda — holds since 2025-12-20 — tap for who they are Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it Looped/recurrent-depth transformerarchitectures can improve modelquality at a fixed compute budgetwhen the model is large enough. Last stated 2 months ago 9 Sept 2026 SR Sebastian Raschka — holds since 2026-09-09 — tap for who they are Same subject: Self-attention can yield more interpretable models. — tap to centre the map on it Self-attention can yield moreinterpretable models. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Flash attention does more flops than standard attention yet runs significantly faster. Last stated 9 Jun 2024 · 2 years ago Holds Andrej Karpathy Read this korrent →