Tap a claim on the ring to put it at the centre.
← If the goal is just to engineer a good system, there is no reason not to use backpropagation.
17 connected korrents · 17 moments from 22 Dec 2014 to 21 Nov 2026.
Everything filed under neural networks
neural networks
Everything filed under optimizers
optimizers
Everything filed under transformers
transformers
Everything filed under pruning
pruning
Everything filed under simplicity
simplicity
Everything filed under LLMs
LLMs
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: If the goal is just to engineer a good system, there is no reason not to use backpropagation.
If the goal is just to engineer a good system, there is no reason not to use backpropagation.
Last stated 4 years ago
27 Mar 2023
IS
Ilya Sutskever — holds since 2023-03-27 — tap for who they are
Same subject: The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. — tap to centre the map on it
The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out.
Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning. — tap to centre the map on it
We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning.
Last stated a year ago
26 Sept 2025
RS
Richard Sutton — holds since 2025-09-26 — tap for who they are
Same subject: Growing evidence suggests the brain performs a form of backpropagation-like learning. — tap to centre the map on it
Growing evidence suggests the brain performs a form of backpropagation-like learning.
Last stated 6 years ago
13 Mar 2021
GW
Gwern — holds since 2021-03-13 — tap for who they are
Same subject: Going back to your roots and using tools you know how to use to solve problems is not a bad thing, and AI tools should not be the only thing relied on to solve problems. — tap to centre the map on it
Going back to your roots and using tools you know how to use to solve problems is not a bad thing, and AI tools should not be the only thing relied on to solve problems.
Last stated 8 months ago
2 Apr 2026
MS
Marvin Schwaibold — holds since 2026-04-02 — tap for who they are
Same subject: The current AI stack is not among the top 50% most efficient approaches to learning. — tap to centre the map on it
The current AI stack is not among the top 50% most efficient approaches to learning.
Last stated yesterday
21 Nov 2026
FC
François Chollet — holds since 2026-11-21 — tap for who they are
Same subject: The bottleneck in AI research is good ideas and insights, which neural nets could help with. — tap to centre the map on it
The bottleneck in AI research is good ideas and insights, which neural nets could help with.
Last stated 4 years ago
27 Mar 2023
IS
Ilya Sutskever — holds since 2023-03-27 — tap for who they are
Same subject: AI is good at giving artifacts that are not necessarily grounded in understanding of the system, which works for prototypes and one-off experiments but leaves you lost the moment you need to debug. — tap to centre the map on it
AI is good at giving artifacts that are not necessarily grounded in understanding of the system, which works for prototypes and one-off experiments but leaves you lost the moment you need to debug.
Last stated 8 months ago
25 Mar 2026
FG
Flora Guo — holds since 2026-03-25 — tap for who they are
Same subject: Using a neural network is likely to become easier than designing a custom-tuned prediction algorithm for many tasks. — tap to centre the map on it
Using a neural network is likely to become easier than designing a custom-tuned prediction algorithm for many tasks.
Last stated a year ago
4 Jul 2025
NE
Nelson Elhage — holds since 2025-07-04 — tap for who they are
Same subject: AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings. — tap to centre the map on it
AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings.
Last stated 2 months ago
3 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-03 — tap for who they are
Same subject: An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train. — tap to centre the map on it
An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train.
Last stated 12 years ago
22 Dec 2014
JB
Jimmy Ba — holds since 2014-12-22 — tap for who they are
DK
Diederik P. Kingma — holds since 2014-12-22 — tap for who they are
Same subject: An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning. — tap to centre the map on it
An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning.
Last stated 12 years ago
22 Dec 2014
JB
Jimmy Ba — holds since 2014-12-22 — tap for who they are
DK
Diederik P. Kingma — holds since 2014-12-22 — tap for who they are
Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it
A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017.
Last stated 2 years ago
6 Jan 2025
AR
Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are
Same subject: Flash attention does more flops than standard attention yet runs significantly faster. — tap to centre the map on it
Flash attention does more flops than standard attention yet runs significantly faster.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. — tap to centre the map on it
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.
Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. — tap to centre the map on it
Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow.
Last stated 11 months ago
8 Dec 2025
CH
Casey Handmer — holds since 2023-10-11 — tap for who they are
CH
Casey Handmer — holds since 2025-12-08 — tap for who they are
Same subject: Especially in technology, the world very often belongs to the fast: those who iterate, adjust and prune away what is unnecessary. — tap to centre the map on it
Especially in technology, the world very often belongs to the fast: those who iterate, adjust and prune away what is unnecessary.
Last stated 10 months ago
18 Jan 2026
TL
Tobias Lütke — holds since 2026-01-18 — tap for who they are
Same subject: Latent experience remains even when specific knowledge is forgotten. — tap to centre the map on it
Latent experience remains even when specific knowledge is forgotten.
Last stated a month ago
9 Oct 2026
KD
Kent C. Dodds — holds since 2026-10-09 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2014 to today (stretched back to the oldest claim here) — full is today a face: someone who holds the claim — tap it for who they are
At the centre
If the goal is just to engineer a good system, there is no reason not to use backpropagation.
Last stated 27 Mar 2023 · 4 years ago
Holds Ilya Sutskever
Read this korrent →
Similar wording
The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out.
Last stated 17 Oct 2025 · a year ago
Holds Andrej Karpathy
Similar wording
We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning.
Last stated 26 Sept 2025 · a year ago
Holds Richard Sutton
Similar wording
Growing evidence suggests the brain performs a form of backpropagation-like learning.
Last stated 13 Mar 2021 · 6 years ago
Holds Gwern
Similar wording
Going back to your roots and using tools you know how to use to solve problems is not a bad thing, and AI tools should not be the only thing relied on to solve problems.
Last stated 2 Apr 2026 · 8 months ago
Holds Marvin Schwaibold
Similar wording
The current AI stack is not among the top 50% most efficient approaches to learning.
Last stated 21 Nov 2026 · yesterday
Holds François Chollet
Similar wording
The bottleneck in AI research is good ideas and insights, which neural nets could help with.
Last stated 27 Mar 2023 · 4 years ago
Holds Ilya Sutskever
Similar wording
AI is good at giving artifacts that are not necessarily grounded in understanding of the system, which works for prototypes and one-off experiments but leaves you lost the moment you need to debug.
Last stated 25 Mar 2026 · 8 months ago
Holds Flora Guo
Similar wording
Using a neural network is likely to become easier than designing a custom-tuned prediction algorithm for many tasks.
Last stated 4 Jul 2025 · a year ago
Holds Nelson Elhage
Same subject: optimizers
AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings.
Last stated 3 Oct 2026 · 2 months ago
Holds Sebastian Raschka
Same subject: optimizers
An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train.
Last stated 22 Dec 2014 · 12 years ago
Holds Jimmy Ba Diederik P. Kingma
Same subject: optimizers
An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning.
Last stated 22 Dec 2014 · 12 years ago
Holds Jimmy Ba Diederik P. Kingma
Same subject: transformers
A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017.
Last stated 6 Jan 2025 · 2 years ago
Holds Aishwarya Naresh Reganti
Same subject: transformers
Flash attention does more flops than standard attention yet runs significantly faster.
Last stated 9 Jun 2024 · 2 years ago
Holds Andrej Karpathy
Same subject: transformers
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.
Last stated 12 Jun 2017 · 9 years ago
Holds Illia Polosukhin Jakob Uszkoreit Niki Parmar Ashish Vaswani Noam Shazeer Aidan N. Gomez Llion Jones Łukasz Kaiser
Same subject: pruning
Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow.
Last stated 8 Dec 2025 · 11 months ago
Holds Casey Handmer
Same subject: pruning
Especially in technology, the world very often belongs to the fast: those who iterate, adjust and prune away what is unnecessary.
Last stated 18 Jan 2026 · 10 months ago
Holds Tobias Lütke
Same subject: pruning
Latent experience remains even when specific knowledge is forgotten.
Last stated 9 Oct 2026 · a month ago
Holds Kent C. Dodds