Tap a claim on the ring to put it at the centre.
← Self-attention can yield more interpretable models.
17 connected korrents · 18 moments from 1 Jun 2009 to 9 Oct 2026.
Everything filed under transformers
transformers
Everything filed under computer vision
computer vision
Everything filed under pruning
pruning
Everything filed under scaling laws
scaling laws
Everything filed under LLMs
LLMs
Everything filed under AI and human skill
AI and human skill
Everything filed under robotics
robotics
Everything filed under optimism
optimism Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Self-attention can yield more interpretable models. Self-attention can yield more interpretable models. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
LJ
Llion Jones — holds since 2017-06-12 — tap for who they are
ŁK
Łukasz Kaiser — holds since 2017-06-12 — tap for who they are
Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: Context engineering has stayed relevant for a year because it is grounded in how transformer attention works, and it will matter to anyone building on AI until post-transformer models arrive. — tap to centre the map on it Context engineering has stayed relevant for a year because it is grounded in how transformer attention works, and it will matter to anyone building on AI until post-transformer models arrive. Last stated 3 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches. — tap to centre the map on it Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches. Last stated 5 years ago
3 Jun 2021
GW
Gwern — holds since 2021-06-03 — tap for who they are
Same subject: Renouncing your own cognitive ability because the models are good is premature, because companies still pay an enormous premium for it. — tap to centre the map on it Renouncing your own cognitive ability because the models are good is premature, because companies still pay an enormous premium for it. Last stated 3 months ago
31 Jul 2026
PC
Patrick Collison — holds since 2026-07-31 — tap for who they are
Same subject: Letting a robot imagine the next image helps, but it is not essential — the model was surprisingly good without it. — tap to centre the map on it Letting a robot imagine the next image helps, but it is not essential — the model was surprisingly good without it. Last stated 2 months ago
12 Aug 2026
CF
Chelsea Finn — holds since 2026-08-12 — tap for who they are
Same subject: As models get better at following instructions, techniques like finetuning and constrained sampling for structured outputs will become less necessary — tap to centre the map on it As models get better at following instructions, techniques like finetuning and constrained sampling for structured outputs will become less necessary Last stated 3 years ago
16 Jan 2024
CH
Chip Huyen — holds since 2024-01-16 — tap for who they are
Same subject: A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. — tap to centre the map on it A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: The biggest effect of agents on the market may well be that the rewards for better products increase. — tap to centre the map on it The biggest effect of agents on the market may well be that the rewards for better products increase. Last stated 2 weeks ago
9 Oct 2026
PC
Patrick Collison — holds since 2026-10-09 — tap for who they are
Same subject: Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. — tap to centre the map on it Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. Last stated 11 months ago
8 Dec 2025
CH
Casey Handmer — holds since 2023-10-11 — tap for who they are
CH
Casey Handmer — holds since 2025-12-08 — tap for who they are
Same subject: Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with. — tap to centre the map on it Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with. Last stated a month ago
18 Sept 2026
AR
Armin Ronacher — holds since 2026-09-18 — tap for who they are
Same subject: Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay. — tap to centre the map on it Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay. Last stated 3 months ago
20 Jul 2026
TW
Tommy Wood — holds since 2026-07-20 — tap for who they are
Same subject: AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function. — tap to centre the map on it AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function. Last stated 3 months ago
26 Jul 2026
JH
Jensen Huang — holds since 2026-07-26 — tap for who they are
Same subject: Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century. — tap to centre the map on it Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century. Last stated 2 months ago
4 Sept 2026
FL
Fei-Fei Li — holds since 2026-09-04 — tap for who they are
Same subject: Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia. — tap to centre the map on it Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia. Last stated 17 years ago
1 Jun 2009
FL
Fei-Fei Li — holds since 2009-06-01 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 7 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
At the centre
Self-attention can yield more interpretable models.
Holds Read this korrent →
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Holds
Context engineering has stayed relevant for a year because it is grounded in how transformer attention works, and it will matter to anyone building on AI until post-transformer models arrive.
Holds
Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches.
Holds
Renouncing your own cognitive ability because the models are good is premature, because companies still pay an enormous premium for it.
Holds
Letting a robot imagine the next image helps, but it is not essential — the model was surprisingly good without it.
Holds
As models get better at following instructions, techniques like finetuning and constrained sampling for structured outputs will become less necessary
Holds
A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models.
Holds
The biggest effect of agents on the market may well be that the rewards for better products increase.
Holds
Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow.
Holds
Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with.
Holds
Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay.
Holds
AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function.
Holds
Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century.
Holds
Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia.
Holds
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Holds
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds