korrents

On the map

Tap a claim on the ring to put it at the centre.

← If the goal is just to engineer a good system, there is no reason not to use backpropagation.

17 connected korrents · 17 moments from 22 Dec 2014 to 21 Nov 2026.

Everything filed under neural networks neural networks Everything filed under optimizers optimizers Everything filed under transformers transformers Everything filed under pruning pruning Everything filed under simplicity simplicity Everything filed under LLMs LLMs Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: If the goal is just to engineer a good system, there is no reason not to use backpropagation. If the goal is just to engineer a goodsystem, there is no reason not to usebackpropagation. Last stated 4 years ago 27 Mar 2023 IS Ilya Sutskever — holds since 2023-03-27 — tap for who they are Same subject: The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. — tap to centre the map on it The knowledge a model soaks up inpre-training is holding it back;what we actually want is theintelligence with the knowledgestripped out. Last stated a year ago 17 Oct 2025 AK Andrej Karpathy — holds since 2025-10-17 — tap for who they are Same subject: We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning. — tap to centre the map on it We have almost no automatedtechniques for making a systemtransfer what it learns, and none ofthe few we have are used in moderndeep learning. Last stated a year ago 26 Sept 2025 RS Richard Sutton — holds since 2025-09-26 — tap for who they are Same subject: Growing evidence suggests the brain performs a form of backpropagation-like learning. — tap to centre the map on it Growing evidence suggests the brainperforms a form ofbackpropagation-like learning. Last stated 6 years ago 13 Mar 2021 GW Gwern — holds since 2021-03-13 — tap for who they are Same subject: Going back to your roots and using tools you know how to use to solve problems is not a bad thing, and AI tools should not be the only thing relied on to solve problems. — tap to centre the map on it Going back to your roots and usingtools you know how to use to solveproblems is not a bad thing, and AItools should not be the only thingrelied on to solve problems. Last stated 8 months ago 2 Apr 2026 MS Marvin Schwaibold — holds since 2026-04-02 — tap for who they are Same subject: The current AI stack is not among the top 50% most efficient approaches to learning. — tap to centre the map on it The current AI stack is not amongthe top 50% most efficientapproaches to learning. Last stated yesterday 21 Nov 2026 FC François Chollet — holds since 2026-11-21 — tap for who they are Same subject: The bottleneck in AI research is good ideas and insights, which neural nets could help with. — tap to centre the map on it The bottleneck in AI research isgood ideas and insights, whichneural nets could help with. Last stated 4 years ago 27 Mar 2023 IS Ilya Sutskever — holds since 2023-03-27 — tap for who they are Same subject: AI is good at giving artifacts that are not necessarily grounded in understanding of the system, which works for prototypes and one-off experiments but leaves you lost the moment you need to debug. — tap to centre the map on it AI is good at giving artifacts thatare not necessarily grounded inunderstanding of the system, whichworks for prototypes and one-offexperiments but leaves you lost themoment you need to debug. Last stated 8 months ago 25 Mar 2026 FG Flora Guo — holds since 2026-03-25 — tap for who they are Same subject: Using a neural network is likely to become easier than designing a custom-tuned prediction algorithm for many tasks. — tap to centre the map on it Using a neural network is likely tobecome easier than designing acustom-tuned prediction algorithmfor many tasks. Last stated a year ago 4 Jul 2025 NE Nelson Elhage — holds since 2025-07-04 — tap for who they are Same subject: AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings. — tap to centre the map on it AdamW is still one of the mostwidely used optimizers, though Muonhas become popular for layers otherthan embeddings. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train. — tap to centre the map on it An optimizer that is invariant torescaling the gradients and cheap inmemory is what makes very largemodels practical to train. Last stated 12 years ago 22 Dec 2014 JB Jimmy Ba — holds since 2014-12-22 — tap for who they are DK Diederik P. Kingma — holds since 2014-12-22 — tap for who they are Same subject: An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning. — tap to centre the map on it An optimizer's hyper-parametersshould mean something a practitionercan reason about, and should rarelyneed tuning. Last stated 12 years ago 22 Dec 2014 JB Jimmy Ba — holds since 2014-12-22 — tap for who they are DK Diederik P. Kingma — holds since 2014-12-22 — tap for who they are Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it A single AI paper rarely changeseverything; the last one that didwas 'Attention Is All You Need' in2017. Last stated 2 years ago 6 Jan 2025 AR Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are Same subject: Flash attention does more flops than standard attention yet runs significantly faster. — tap to centre the map on it Flash attention does more flops thanstandard attention yet runssignificantly faster. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. — tap to centre the map on it Sequence transduction can use asimple architecture based solely onattention, without recurrence orconvolutions. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. — tap to centre the map on it Batteries will replace transmissionas the cheapest way to keep thelights on, and the grid will shrinkrather than grow. Last stated 11 months ago 8 Dec 2025 CH Casey Handmer — holds since 2023-10-11 — tap for who they are CH Casey Handmer — holds since 2025-12-08 — tap for who they are Same subject: Especially in technology, the world very often belongs to the fast: those who iterate, adjust and prune away what is unnecessary. — tap to centre the map on it Especially in technology, the worldvery often belongs to the fast:those who iterate, adjust and pruneaway what is unnecessary. Last stated 10 months ago 18 Jan 2026 TL Tobias Lütke — holds since 2026-01-18 — tap for who they are Same subject: Latent experience remains even when specific knowledge is forgotten. — tap to centre the map on it Latent experience remains even whenspecific knowledge is forgotten. Last stated a month ago 9 Oct 2026 KD Kent C. Dodds — holds since 2026-10-09 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2014 to today (stretched back to the oldest claim here) — full is todaya face: someone who holds the claim — tap it for who they are

At the centre If the goal is just to engineer a good system, there is no reason not to use backpropagation. Last stated 27 Mar 2023 · 4 years ago Holds Ilya Sutskever Read this korrent →