Tap a claim on the ring to put it at the centre.
← Pre-trained representations reduce the need for many heavily-engineered task-specific architectures.
17 connected korrents · 16 moments from 1 Jun 2009 to 18 Sept 2026.
Everything filed under neural networks
neural networks
Everything filed under LLMs
LLMs
Everything filed under computer vision
computer vision
Everything filed under pruning
pruning
Everything filed under robotics
robotics
Everything filed under transformers
transformers Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Pre-trained representations reduce the need for many heavily-engineered task-specific architectures. Pre-trained representations reduce the need for many heavily-engineered task-specific architectures. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: Current techniques restrict pre-trained representation power because standard language models are unidirectional. — tap to centre the map on it Current techniques restrict pre-trained representation power because standard language models are unidirectional. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. — tap to centre the map on it Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. Last stated 6 years ago
28 May 2020
DA
Dario Amodei — holds since 2020-05-28 — tap for who they are
Same subject: Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them. — tap to centre the map on it Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them. Last stated 5 months ago
16 May 2026
SR
Sebastian Raschka — holds since 2026-05-16 — tap for who they are
Same subject: The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. — tap to centre the map on it The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: It is the diversity of a robot data set, not its size, that produces generalization: dropping the most diverse slice hurts far more than dropping a random fifth. — tap to centre the map on it It is the diversity of a robot data set, not its size, that produces generalization: dropping the most diverse slice hurts far more than dropping a random fifth. Last stated 2 months ago
12 Aug 2026
CF
Chelsea Finn — holds since 2026-08-12 — tap for who they are
Same subject: Computer-use agents had to wait for language models: without pre-trained representations the reward is too sparse to ever learn from. — tap to centre the map on it Computer-use agents had to wait for language models: without pre-trained representations the reward is too sparse to ever learn from. Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: As models get better at following instructions, techniques like finetuning and constrained sampling for structured outputs will become less necessary — tap to centre the map on it As models get better at following instructions, techniques like finetuning and constrained sampling for structured outputs will become less necessary Last stated 3 years ago
16 Jan 2024
CH
Chip Huyen — holds since 2024-01-16 — tap for who they are
Same subject: We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning. — tap to centre the map on it We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning. Last stated a year ago
26 Sept 2025
RS
Richard Sutton — holds since 2025-09-26 — tap for who they are
Same subject: Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. — tap to centre the map on it Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. Last stated 11 months ago
8 Dec 2025
CH
Casey Handmer — holds since 2023-10-11 — tap for who they are
CH
Casey Handmer — holds since 2025-12-08 — tap for who they are
Same subject: Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with. — tap to centre the map on it Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with. Last stated a month ago
18 Sept 2026
AR
Armin Ronacher — holds since 2026-09-18 — tap for who they are
Same subject: Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay. — tap to centre the map on it Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay. Last stated 3 months ago
20 Jul 2026
TW
Tommy Wood — holds since 2026-07-20 — tap for who they are
Same subject: Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. — tap to centre the map on it Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. Last stated 2 months ago
9 Sept 2026
SR
Sebastian Raschka — holds since 2026-09-09 — tap for who they are
Same subject: AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function. — tap to centre the map on it AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function. Last stated 3 months ago
26 Jul 2026
JH
Jensen Huang — holds since 2026-07-26 — tap for who they are
Same subject: Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century. — tap to centre the map on it Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century. Last stated 2 months ago
4 Sept 2026
FL
Fei-Fei Li — holds since 2026-09-04 — tap for who they are
Same subject: Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia. — tap to centre the map on it Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia. Last stated 17 years ago
1 Jun 2009
FL
Fei-Fei Li — holds since 2009-06-01 — tap for who they are
At the centre
Pre-trained representations reduce the need for many heavily-engineered task-specific architectures.
Holds Read this korrent →
Current techniques restrict pre-trained representation power because standard language models are unidirectional.
Holds
Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches.
Holds
Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them.
Holds
The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out.
Holds
It is the diversity of a robot data set, not its size, that produces generalization: dropping the most diverse slice hurts far more than dropping a random fifth.
Holds
Computer-use agents had to wait for language models: without pre-trained representations the reward is too sparse to ever learn from.
Holds
As models get better at following instructions, techniques like finetuning and constrained sampling for structured outputs will become less necessary
Holds
We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning.
Holds
Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow.
Holds
Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with.
Holds
Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay.
Holds
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.
Holds
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Holds
Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.
Holds
AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function.
Holds
Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century.
Holds
Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia.
Holds