korrents

On the map

Tap a claim on the ring to put it at the centre.

← A layer should learn a residual with reference to its own input rather…

17 connected korrents · 16 moments from 1 Jun 2009 to 18 Sept 2026.

Everything filed under neural networks neural networks Everything filed under computer vision computer vision Everything filed under pruning pruning Everything filed under optimizers optimizers Everything filed under LLMs LLMs Everything filed under GANs GANs Everything filed under mathematics mathematics Everything filed under transformers transformers Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. A layer should learn a residual withreference to its own input rather than anunreferenced function, which is what makesgreat depth trainable. Last stated 11 years ago 10 Dec 2015 JS Jian Sun — holds since 2015-12-10 — tap for who they are KH Kaiming He — holds since 2015-12-10 — tap for who they are Same subject: Residual networks are easier to optimize than plain ones and keep gaining accuracy as depth increases. — tap to centre the map on it Residual networks are easier tooptimize than plain ones and keepgaining accuracy as depth increases. Last stated 11 years ago 10 Dec 2015 JS Jian Sun — holds since 2015-12-10 — tap for who they are KH Kaiming He — holds since 2015-12-10 — tap for who they are Same subject: An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train. — tap to centre the map on it An optimizer that is invariant torescaling the gradients and cheap inmemory is what makes very largemodels practical to train. Last stated 12 years ago 22 Dec 2014 JB Jimmy Ba — holds since 2014-12-22 — tap for who they are DK Diederik P. Kingma — holds since 2014-12-22 — tap for who they are Same subject: Current techniques restrict pre-trained representation power because standard language models are unidirectional. — tap to centre the map on it Current techniques restrictpre-trained representation powerbecause standard language models areunidirectional. Last stated 8 years ago 11 Oct 2018 KT Kristina Toutanova — holds since 2018-10-11 — tap for who they are MC Ming-Wei Chang — holds since 2018-10-11 — tap for who they are KL Kenton Lee — holds since 2018-10-11 — tap for who they are JD Jacob Devlin — holds since 2018-10-11 — tap for who they are Same subject: An adversarial GAN loss should only supplement a larger unsupervised or semi-supervised generative model rather than stand alone as a primary approach. — tap to centre the map on it An adversarial GAN loss should onlysupplement a larger unsupervised orsemi-supervised generative modelrather than stand alone as a primaryapproach. Last stated 6 years ago 4 Feb 2021 GW Gwern — holds since 2021-02-04 — tap for who they are Same subject: Supervised pretraining on ImageNet is now obsolete for transfer learning, and ImageNet itself is just a contaminated validation set. — tap to centre the map on it Supervised pretraining on ImageNetis now obsolete for transferlearning, and ImageNet itself isjust a contaminated validation set. Last stated 6 years ago 6 Apr 2021 GW Gwern — holds since 2021-04-06 — tap for who they are Same subject: The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. — tap to centre the map on it The knowledge a model soaks up inpre-training is holding it back;what we actually want is theintelligence with the knowledgestripped out. Last stated a year ago 17 Oct 2025 AK Andrej Karpathy — holds since 2025-10-17 — tap for who they are Same subject: Depth is not free: past a point, deeper neural networks are harder to train rather than simply better. — tap to centre the map on it Depth is not free: past a point,deeper neural networks are harder totrain rather than simply better. Last stated 11 years ago 10 Dec 2015 JS Jian Sun — holds since 2015-12-10 — tap for who they are KH Kaiming He — holds since 2015-12-10 — tap for who they are Same subject: Machine learning is a shallow field compared with mathematics: even its most important ideas can be explained in a couple of minutes. — tap to centre the map on it Machine learning is a shallow fieldcompared with mathematics: even itsmost important ideas can beexplained in a couple of minutes. Last stated 2 months ago 11 Aug 2026 RG Ryan Greenblatt — holds since 2026-08-11 — tap for who they are Same subject: Batteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow. — tap to centre the map on it Batteries will replace transmissionas the cheapest way to keep thelights on, and the grid will shrinkrather than grow. Last stated 11 months ago 8 Dec 2025 CH Casey Handmer — holds since 2023-10-11 — tap for who they are CH Casey Handmer — holds since 2025-12-08 — tap for who they are Same subject: Most agent harnesses need some pruning during context compaction anyway for cost reasons, which Jev might help with. — tap to centre the map on it Most agent harnesses need somepruning during context compactionanyway for cost reasons, which Jevmight help with. Last stated a month ago 18 Sept 2026 AR Armin Ronacher — holds since 2026-09-18 — tap for who they are Same subject: Neuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay. — tap to centre the map on it Neuroplasticity is as much theremoval of connections as the growthof new ones, so pruning is a form oflearning rather than a form ofdecay. Last stated 3 months ago 20 Jul 2026 TW Tommy Wood — holds since 2026-07-20 — tap for who they are Same subject: Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. — tap to centre the map on it Sequence transduction can use asimple architecture based solely onattention, without recurrence orconvolutions. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can relyentirely on self-attention for inputand output representations withoutsequence-aligned RNNs orconvolution. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it Looped/recurrent-depth transformerarchitectures can improve modelquality at a fixed compute budgetwhen the model is large enough. Last stated 2 months ago 9 Sept 2026 SR Sebastian Raschka — holds since 2026-09-09 — tap for who they are Same subject: AlexNet's real breakthrough was not computer vision but the discovery of a universal function approximator that can learn any function. — tap to centre the map on it AlexNet's real breakthrough was notcomputer vision but the discovery ofa universal function approximatorthat can learn any function. Last stated 3 months ago 26 Jul 2026 JH Jensen Huang — holds since 2026-07-26 — tap for who they are Same subject: Atlas is the first model to unify pixel generation and pixel reconstruction, two tracks computer vision has kept apart for over half a century. — tap to centre the map on it Atlas is the first model to unifypixel generation and pixelreconstruction, two tracks computervision has kept apart for over halfa century. Last stated 2 months ago 4 Sept 2026 FL Fei-Fei Li — holds since 2026-09-04 — tap for who they are Same subject: Growth of internet image data can enable more sophisticated models for indexing and interacting with multimedia. — tap to centre the map on it Growth of internet image data canenable more sophisticated models forindexing and interacting withmultimedia. Last stated 17 years ago 1 Jun 2009 FL Fei-Fei Li — holds since 2009-06-01 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2009 to today (stretched back to the oldest claim here) — full is todaya face: someone who holds the claim — tap it for who they are

At the centre A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. Last stated 10 Dec 2015 · 11 years ago Holds Jian SunKaiming He Read this korrent →