korrents

On the map

Tap a claim on the ring to put it at the centre.

← Shuffling the training data between epochs is a good idea because it is better for the optimization.

17 connected korrents · 16 moments from 22 Dec 2014 to 21 Nov 2026.

Everything filed under optimizers optimizers Everything filed under neural networks neural networks Everything filed under LLMs LLMs Everything filed under semiconductors semiconductors Everything filed under reinforcement learning reinforcement learning Everything filed under robotics robotics Everything filed under pruning pruning Everything filed under transformers transformers Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Shuffling the training data between epochs is a good idea because it is better for the optimization. Shuffling the training data between epochsis a good idea because it is better forthe optimization. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: When pre-training data is limited, it is better to train a smaller model for multiple epochs than to train a larger model on unique data once. — tap to centre the map on it When pre-training data is limited,it is better to train a smallermodel for multiple epochs than totrain a larger model on unique dataonce. Last stated 3 years ago 1 Dec 2023 SR Sebastian Ruder — holds since 2023-12-01 — tap for who they are Same subject: In AI, how good your training data is matters more than how long you train on it. — tap to centre the map on it In AI, how good your training datais matters more than how long youtrain on it. Last stated 2 months ago 1 Oct 2026 ZM Zvi Mowshowitz — holds since 2026-10-01 — tap for who they are PL Pieter Levels — holds since 2024-08-20 — tap for who they are Same subject: It is never a bad idea to check your training data, because data is often the most precious thing and errors in it give poor results. — tap to centre the map on it It is never a bad idea to check yourtraining data, because data is oftenthe most precious thing and errorsin it give poor results. Last stated a month ago 17 Oct 2026 SR Sebastian Raschka — holds since 2026-10-17 — tap for who they are Same subject: If a model answers every rollout correctly all the time, training might as well stop, because there is no learning signal left. — tap to centre the map on it If a model answers every rolloutcorrectly all the time, trainingmight as well stop, because there isno learning signal left. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: Progress in data, GPUs and neural nets is intertwined, so it is no coincidence they matured together. — tap to centre the map on it Progress in data, GPUs and neuralnets is intertwined, so it is nocoincidence they matured together. Last stated 4 years ago 27 Mar 2023 IS Ilya Sutskever — holds since 2023-03-27 — tap for who they are Same subject: A robot going down a wrong path is not producing training data; the efficient move is for a human to intervene and show the recovery. — tap to centre the map on it A robot going down a wrong path isnot producing training data; theefficient move is for a human tointervene and show the recovery. Last stated 3 months ago 12 Aug 2026 CF Chelsea Finn — holds since 2026-08-12 — tap for who they are Same subject: Data is becoming more valuable not only for training models but as a complement to cheaper intelligence. — tap to centre the map on it Data is becoming more valuable notonly for training models but as acomplement to cheaper intelligence. Last stated a month ago 9 Oct 2026 PC Patrick Collison — holds since 2026-10-09 — tap for who they are Same subject: The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start. — tap to centre the map on it The saving pruning offers is onlyever at inference, because thesparse architectures it produces arehard to train from the start. Last stated 9 years ago 9 Mar 2018 MC Michael Carbin — holds since 2018-03-09 — tap for who they are JF Jonathan Frankle — holds since 2018-03-09 — tap for who they are Same subject: AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings. — tap to centre the map on it AdamW is still one of the mostwidely used optimizers, though Muonhas become popular for layers otherthan embeddings. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train. — tap to centre the map on it An optimizer that is invariant torescaling the gradients and cheap inmemory is what makes very largemodels practical to train. Last stated 12 years ago 22 Dec 2014 JB Jimmy Ba — holds since 2014-12-22 — tap for who they are DK Diederik P. Kingma — holds since 2014-12-22 — tap for who they are Same subject: An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning. — tap to centre the map on it An optimizer's hyper-parametersshould mean something a practitionercan reason about, and should rarelyneed tuning. Last stated 12 years ago 22 Dec 2014 JB Jimmy Ba — holds since 2014-12-22 — tap for who they are DK Diederik P. Kingma — holds since 2014-12-22 — tap for who they are Same subject: The current AI stack is not among the top 50% most efficient approaches to learning. — tap to centre the map on it The current AI stack is not amongthe top 50% most efficientapproaches to learning. Last stated yesterday 21 Nov 2026 FC François Chollet — holds since 2026-11-21 — tap for who they are Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it A single AI paper rarely changeseverything; the last one that didwas 'Attention Is All You Need' in2017. Last stated 2 years ago 6 Jan 2025 AR Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can relyentirely on self-attention for inputand output representations withoutsequence-aligned RNNs orconvolution. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: If the goal is just to engineer a good system, there is no reason not to use backpropagation. — tap to centre the map on it If the goal is just to engineer agood system, there is no reason notto use backpropagation. Last stated 4 years ago 27 Mar 2023 IS Ilya Sutskever — holds since 2023-03-27 — tap for who they are Same subject: A curve, which is what a deep learning model is, is a very bad substrate for discrete computation. — tap to centre the map on it A curve, which is what a deeplearning model is, is a very badsubstrate for discrete computation. Last stated 2 years ago 6 Nov 2024 FC François Chollet — holds since 2024-11-06 — tap for who they are Same subject: A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. — tap to centre the map on it A layer should learn a residual withreference to its own input ratherthan an unreferenced function, whichis what makes great depth trainable. Last stated 11 years ago 10 Dec 2015 JS Jian Sun — holds since 2015-12-10 — tap for who they are KH Kaiming He — holds since 2015-12-10 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2014 to today (stretched back to the oldest claim here) — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Shuffling the training data between epochs is a good idea because it is better for the optimization. Last stated 17 Oct 2026 · a month ago Holds Sebastian Raschka Read this korrent →