Tap a claim on the ring to put it at the centre.
← A winning ticket wins on its initial weights: the connections it keeps…
17 connected korrents · 15 moments from 1 Jun 2009 to 18 Sept 2026. Nearly all of them are about pruning .
Everything filed under neural networks
neural networks
Everything filed under computer vision
computer vision
Everything filed under LLMs
LLMs
Everything filed under scaling laws
scaling laws
Everything filed under energy
energy Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: A winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work. A winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work. Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. — tap to centre the map on it A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. — tap to centre the map on it Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. Last stated 11 months ago
8 Dec 2025
CH
Casey Handmer — holds since 2023-10-11 — tap for who they are
CH
Casey Handmer — holds since 2025-12-08 — tap for who they are
Same subject: Intelligence is the execution of small, simplicity-weighted programs found by search over smooth loss landscapes in overparameterized networks, approaching Bayes-optimal reasoning as scale increases. — tap to centre the map on it Intelligence is the execution of small, simplicity-weighted programs found by search over smooth loss landscapes in overparameterized networks, approaching Bayes-optimal reasoning as scale increases. Last stated 5 years ago
11 Jun 2021
GW
Gwern — holds since 2021-06-11 — tap for who they are
Same subject: Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with. — tap to centre the map on it Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with. Last stated a month ago
18 Sept 2026
AR
Armin Ronacher — holds since 2026-09-18 — tap for who they are
Same subject: Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay. — tap to centre the map on it Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay. Last stated 3 months ago
20 Jul 2026
TW
Tommy Wood — holds since 2026-07-20 — tap for who they are
Same subject: Pruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free. — tap to centre the map on it Pruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free. Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: Pruning does not merely shrink a network; it uncovers subnetworks whose original initialization was what made them trainable at all. — tap to centre the map on it Pruning does not merely shrink a network; it uncovers subnetworks whose original initialization was what made them trainable at all. Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start. — tap to centre the map on it The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start. Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context. — tap to centre the map on it As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context. Last stated 6 months ago
16 May 2026
SR
Sebastian Raschka — holds since 2026-05-16 — tap for who they are
Same subject: Context engineering has stayed relevant for a year because it is grounded in how transformer attention works, and it will matter to anyone building on AI until post-transformer models arrive. — tap to centre the map on it Context engineering has stayed relevant for a year because it is grounded in how transformer attention works, and it will matter to anyone building on AI until post-transformer models arrive. Last stated 4 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: Directly optimizing prompt embeddings (prefix-tuning) yields better performance than even a single optimized text prompt. — tap to centre the map on it Directly optimizing prompt embeddings (prefix-tuning) yields better performance than even a single optimized text prompt. Last stated 6 years ago
4 Feb 2021
GW
Gwern — holds since 2021-02-04 — tap for who they are
Same subject: AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function. — tap to centre the map on it AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function. Last stated 3 months ago
26 Jul 2026
JH
Jensen Huang — holds since 2026-07-26 — tap for who they are
Same subject: Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century. — tap to centre the map on it Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century. Last stated 2 months ago
4 Sept 2026
FL
Fei-Fei Li — holds since 2026-09-04 — tap for who they are
Same subject: Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia. — tap to centre the map on it Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia. Last stated 17 years ago
1 Jun 2009
FL
Fei-Fei Li — holds since 2009-06-01 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 8 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
At the centre
A winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work.
Holds Read this korrent →
A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it.
Holds
Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow.
Holds
Intelligence is the execution of small, simplicity-weighted programs found by search over smooth loss landscapes in overparameterized networks, approaching Bayes-optimal reasoning as scale increases.
Holds
Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with.
Holds
Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay.
Holds
Pruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free.
Holds
Pruning does not merely shrink a network; it uncovers subnetworks whose original initialization was what made them trainable at all.
Holds
The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start.
Holds
As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.
Holds
Context engineering has stayed relevant for a year because it is grounded in how transformer attention works, and it will matter to anyone building on AI until post-transformer models arrive.
Holds
Directly optimizing prompt embeddings (prefix-tuning) yields better performance than even a single optimized text prompt.
Holds
AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function.
Holds
Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century.
Holds
Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia.
Holds
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Holds
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds