Tap a claim on the ring to put it at the centre.
← Flash attention does more flops than standard attention yet runs significantly faster.
17 connected korrents · 15 moments from 12 Jun 2017 to 19 Oct 2026.
Everything filed under transformers
transformers
Everything filed under software performance
software performance
Everything filed under semiconductors
semiconductors
Everything filed under AI agents
AI agents
Everything filed under benchmarks
benchmarks
Everything filed under coding agents
coding agents
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Flash attention does more flops than standard attention yet runs significantly faster.
Flash attention does more flops than standard attention yet runs significantly faster.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: A brain keyboard may go the way of AR glasses: attention was already fully occupied, so moving the interface closer changes nothing. — tap to centre the map on it
A brain keyboard may go the way of AR glasses: attention was already fully occupied, so moving the interface closer changes nothing.
Last stated 3 months ago
20 Aug 2026
MH
Max Hodak — holds since 2026-08-20 — tap for who they are
Same subject: For GPU performance, flops do not matter so much as the whole memory access pattern does, and torch.compile cannot find every optimization. — tap to centre the map on it
For GPU performance, flops do not matter so much as the whole memory access pattern does, and torch.compile cannot find every optimization.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches. — tap to centre the map on it
Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches.
Last stated 5 years ago
3 Jun 2021
GW
Gwern — holds since 2021-06-03 — tap for who they are
Same subject: The scarce skill is now context switching across parallel agents, not sustained deep work. — tap to centre the map on it
The scarce skill is now context switching across parallel agents, not sustained deep work.
Last stated 9 months ago
4 Mar 2026
BC
Boris Cherny — holds since 2026-03-04 — tap for who they are
Same subject: Criticism that a simpler, scaled-up architecture underperforms a heavily hand-optimized baseline actually demonstrates why that architecture-scaling research matters. — tap to centre the map on it
Criticism that a simpler, scaled-up architecture underperforms a heavily hand-optimized baseline actually demonstrates why that architecture-scaling research matters.
Last stated 5 years ago
11 Jun 2021
GW
Gwern — holds since 2021-06-11 — tap for who they are
Same subject: CPU performance keeps improving quickly rather than stagnating. — tap to centre the map on it
CPU performance keeps improving quickly rather than stagnating.
Last stated 2 months ago
19 Sept 2026
DL
Daniel Lemire — holds since 2026-09-19 — tap for who they are
Same subject: Inference, not training, is now the constraint, and specialized low-energy hardware will beat general-purpose GPUs and TPUs on latency. — tap to centre the map on it
Inference, not training, is now the constraint, and specialized low-energy hardware will beat general-purpose GPUs and TPUs on latency.
Last stated 4 months ago
30 Jul 2026
JD
Jeff Dean — holds since 2026-07-30 — tap for who they are
Same subject: Most people multitasking with coding agents would do better to slow down and give their top priority their full attention. — tap to centre the map on it
Most people multitasking with coding agents would do better to slow down and give their top priority their full attention.
Last stated a month ago
19 Oct 2026
TS
Thariq Shihipar — holds since 2026-10-19 — tap for who they are
Same subject: "Is the page loaded?" was never a real question, and web performance only became measurable once it was broken into the distinct moments a user actually notices. — tap to centre the map on it
"Is the page loaded?" was never a real question, and web performance only became measurable once it was broken into the distinct moments a user actually notices.
Last stated 3 months ago
19 Aug 2026
AO
Addy Osmani — holds since 2026-08-19 — tap for who they are
Same subject: "Optimize it later" would be safe advice only if every engineer already knew how to avoid the architectural mistakes that no later optimization can remove. — tap to centre the map on it
"Optimize it later" would be safe advice only if every engineer already knew how to avoid the architectural mistakes that no later optimization can remove.
Last stated 3 months ago
26 Aug 2026
CM
Casey Muratori — holds since 2026-08-26 — tap for who they are
Same subject: A 300-millisecond response budget is not an achievement but an admission: 300 milliseconds is an eternity in computing, and a product pitching it shows how far the bar has fallen. — tap to centre the map on it
A 300-millisecond response budget is not an achievement but an admission: 300 milliseconds is an eternity in computing, and a product pitching it shows how far the bar has fallen.
Last stated 3 months ago
26 Aug 2026
CM
Casey Muratori — holds since 2026-08-26 — tap for who they are
Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it
A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017.
Last stated 2 years ago
6 Jan 2025
AR
Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are
Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context. — tap to centre the map on it
As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.
Last stated 6 months ago
16 May 2026
SR
Sebastian Raschka — holds since 2026-05-16 — tap for who they are
Same subject: Linear probes are very strong classifiers, and attention-head probes are even stronger. — tap to centre the map on it
Linear probes are very strong classifiers, and attention-head probes are even stronger.
Last stated 11 months ago
20 Dec 2025
NN
Neel Nanda — holds since 2025-12-20 — tap for who they are
Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it
Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.
Last stated 2 months ago
9 Sept 2026
SR
Sebastian Raschka — holds since 2026-09-09 — tap for who they are
Same subject: Self-attention can yield more interpretable models. — tap to centre the map on it
Self-attention can yield more interpretable models.
Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
Flash attention does more flops than standard attention yet runs significantly faster.
Last stated 9 Jun 2024 · 2 years ago
Holds Andrej Karpathy
Read this korrent →
Similar wording
A brain keyboard may go the way of AR glasses: attention was already fully occupied, so moving the interface closer changes nothing.
Last stated 20 Aug 2026 · 3 months ago
Holds Max Hodak
Similar wording
For GPU performance, flops do not matter so much as the whole memory access pattern does, and torch.compile cannot find every optimization.
Last stated 9 Jun 2024 · 2 years ago
Holds Andrej Karpathy
Similar wording
Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches.
Last stated 3 Jun 2021 · 5 years ago
Holds Gwern
Similar wording
The scarce skill is now context switching across parallel agents, not sustained deep work.
Last stated 4 Mar 2026 · 9 months ago
Holds Boris Cherny
Similar wording
Criticism that a simpler, scaled-up architecture underperforms a heavily hand-optimized baseline actually demonstrates why that architecture-scaling research matters.
Last stated 11 Jun 2021 · 5 years ago
Holds Gwern
Similar wording
CPU performance keeps improving quickly rather than stagnating.
Last stated 19 Sept 2026 · 2 months ago
Holds Daniel Lemire
Similar wording
Inference, not training, is now the constraint, and specialized low-energy hardware will beat general-purpose GPUs and TPUs on latency.
Last stated 30 Jul 2026 · 4 months ago
Holds Jeff Dean
Similar wording
Most people multitasking with coding agents would do better to slow down and give their top priority their full attention.
Last stated 19 Oct 2026 · a month ago
Holds Thariq Shihipar
Same subject: software performance
"Is the page loaded?" was never a real question, and web performance only became measurable once it was broken into the distinct moments a user actually notices.
Last stated 19 Aug 2026 · 3 months ago
Holds Addy Osmani
Same subject: software performance
"Optimize it later" would be safe advice only if every engineer already knew how to avoid the architectural mistakes that no later optimization can remove.
Last stated 26 Aug 2026 · 3 months ago
Holds Casey Muratori
Same subject: software performance
A 300-millisecond response budget is not an achievement but an admission: 300 milliseconds is an eternity in computing, and a product pitching it shows how far the bar has fallen.
Last stated 26 Aug 2026 · 3 months ago
Holds Casey Muratori
Same subject: transformers
A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017.
Last stated 6 Jan 2025 · 2 years ago
Holds Aishwarya Naresh Reganti
Same subject: transformers
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Last stated 12 Jun 2017 · 9 years ago
Holds Illia Polosukhin Jakob Uszkoreit Niki Parmar Ashish Vaswani Noam Shazeer Aidan N. Gomez Llion Jones Łukasz Kaiser
Same subject: transformers
As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.
Last stated 16 May 2026 · 6 months ago
Holds Sebastian Raschka
Same subject: neural networks
Linear probes are very strong classifiers, and attention-head probes are even stronger.
Last stated 20 Dec 2025 · 11 months ago
Holds Neel Nanda
Same subject: neural networks
Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.
Last stated 9 Sept 2026 · 2 months ago
Holds Sebastian Raschka
Same subject: neural networks
Self-attention can yield more interpretable models.
Last stated 12 Jun 2017 · 9 years ago
Holds Illia Polosukhin Jakob Uszkoreit Niki Parmar Ashish Vaswani Noam Shazeer Aidan N. Gomez Llion Jones Łukasz Kaiser