Tap a claim on the ring to put it at the centre.
← Shuffling the training data between epochs is a good idea because it is better for the optimization.
17 connected korrents · 16 moments from 22 Dec 2014 to 21 Nov 2026.
Everything filed under optimizers
optimizers
Everything filed under neural networks
neural networks
Everything filed under LLMs
LLMs
Everything filed under semiconductors
semiconductors
Everything filed under reinforcement learning
reinforcement learning
Everything filed under robotics
robotics
Everything filed under pruning
pruning
Everything filed under transformers
transformers
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Shuffling the training data between epochs is a good idea because it is better for the optimization.
Shuffling the training data between epochs is a good idea because it is better for the optimization.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: When pre-training data is limited, it is better to train a smaller model for multiple epochs than to train a larger model on unique data once. — tap to centre the map on it
When pre-training data is limited, it is better to train a smaller model for multiple epochs than to train a larger model on unique data once.
Last stated 3 years ago
1 Dec 2023
SR
Sebastian Ruder — holds since 2023-12-01 — tap for who they are
Same subject: In AI, how good your training data is matters more than how long you train on it. — tap to centre the map on it
In AI, how good your training data is matters more than how long you train on it.
Last stated 2 months ago
1 Oct 2026
ZM
Zvi Mowshowitz — holds since 2026-10-01 — tap for who they are
PL
Pieter Levels — holds since 2024-08-20 — tap for who they are
Same subject: It is never a bad idea to check your training data, because data is often the most precious thing and errors in it give poor results. — tap to centre the map on it
It is never a bad idea to check your training data, because data is often the most precious thing and errors in it give poor results.
Last stated a month ago
17 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-17 — tap for who they are
Same subject: If a model answers every rollout correctly all the time, training might as well stop, because there is no learning signal left. — tap to centre the map on it
If a model answers every rollout correctly all the time, training might as well stop, because there is no learning signal left.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: Progress in data, GPUs and neural nets is intertwined, so it is no coincidence they matured together. — tap to centre the map on it
Progress in data, GPUs and neural nets is intertwined, so it is no coincidence they matured together.
Last stated 4 years ago
27 Mar 2023
IS
Ilya Sutskever — holds since 2023-03-27 — tap for who they are
Same subject: A robot going down a wrong path is not producing training data; the efficient move is for a human to intervene and show the recovery. — tap to centre the map on it
A robot going down a wrong path is not producing training data; the efficient move is for a human to intervene and show the recovery.
Last stated 3 months ago
12 Aug 2026
CF
Chelsea Finn — holds since 2026-08-12 — tap for who they are
Same subject: Data is becoming more valuable not only for training models but as a complement to cheaper intelligence. — tap to centre the map on it
Data is becoming more valuable not only for training models but as a complement to cheaper intelligence.
Last stated a month ago
9 Oct 2026
PC
Patrick Collison — holds since 2026-10-09 — tap for who they are
Same subject: The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start. — tap to centre the map on it
The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start.
Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings. — tap to centre the map on it
AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings.
Last stated 2 months ago
3 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-03 — tap for who they are
Same subject: An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train. — tap to centre the map on it
An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train.
Last stated 12 years ago
22 Dec 2014
JB
Jimmy Ba — holds since 2014-12-22 — tap for who they are
DK
Diederik P. Kingma — holds since 2014-12-22 — tap for who they are
Same subject: An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning. — tap to centre the map on it
An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning.
Last stated 12 years ago
22 Dec 2014
JB
Jimmy Ba — holds since 2014-12-22 — tap for who they are
DK
Diederik P. Kingma — holds since 2014-12-22 — tap for who they are
Same subject: The current AI stack is not among the top 50% most efficient approaches to learning. — tap to centre the map on it
The current AI stack is not among the top 50% most efficient approaches to learning.
Last stated yesterday
21 Nov 2026
FC
François Chollet — holds since 2026-11-21 — tap for who they are
Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it
A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017.
Last stated 2 years ago
6 Jan 2025
AR
Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are
Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: If the goal is just to engineer a good system, there is no reason not to use backpropagation. — tap to centre the map on it
If the goal is just to engineer a good system, there is no reason not to use backpropagation.
Last stated 4 years ago
27 Mar 2023
IS
Ilya Sutskever — holds since 2023-03-27 — tap for who they are
Same subject: A curve, which is what a deep learning model is, is a very bad substrate for discrete computation. — tap to centre the map on it
A curve, which is what a deep learning model is, is a very bad substrate for discrete computation.
Last stated 2 years ago
6 Nov 2024
FC
François Chollet — holds since 2024-11-06 — tap for who they are
Same subject: A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. — tap to centre the map on it
A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable.
Last stated 11 years ago
10 Dec 2015
JS
Jian Sun — holds since 2015-12-10 — tap for who they are
KH
Kaiming He — holds since 2015-12-10 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2014 to today (stretched back to the oldest claim here) — full is today a face: someone who holds the claim — tap it for who they are
At the centre
Shuffling the training data between epochs is a good idea because it is better for the optimization.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Read this korrent →
Similar wording
When pre-training data is limited, it is better to train a smaller model for multiple epochs than to train a larger model on unique data once.
Last stated 1 Dec 2023 · 3 years ago
Holds Sebastian Ruder
Similar wording
In AI, how good your training data is matters more than how long you train on it.
Last stated 1 Oct 2026 · 2 months ago
Holds Zvi Mowshowitz Pieter Levels
Similar wording
It is never a bad idea to check your training data, because data is often the most precious thing and errors in it give poor results.
Last stated 17 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
If a model answers every rollout correctly all the time, training might as well stop, because there is no learning signal left.
Last stated 10 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
Progress in data, GPUs and neural nets is intertwined, so it is no coincidence they matured together.
Last stated 27 Mar 2023 · 4 years ago
Holds Ilya Sutskever
Similar wording
A robot going down a wrong path is not producing training data; the efficient move is for a human to intervene and show the recovery.
Last stated 12 Aug 2026 · 3 months ago
Holds Chelsea Finn
Similar wording
Data is becoming more valuable not only for training models but as a complement to cheaper intelligence.
Last stated 9 Oct 2026 · a month ago
Holds Patrick Collison
Similar wording
The saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start.
Last stated 9 Mar 2018 · 9 years ago
Holds Michael Carbin Jonathan Frankle
Same subject: optimizers
AdamW is still one of the most widely used optimizers, though Muon has become popular for layers other than embeddings.
Last stated 3 Oct 2026 · 2 months ago
Holds Sebastian Raschka
Same subject: optimizers
An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train.
Last stated 22 Dec 2014 · 12 years ago
Holds Jimmy Ba Diederik P. Kingma
Same subject: optimizers
An optimizer's hyper-parameters should mean something a practitioner can reason about, and should rarely need tuning.
Last stated 22 Dec 2014 · 12 years ago
Holds Jimmy Ba Diederik P. Kingma
Same subject: transformers
The current AI stack is not among the top 50% most efficient approaches to learning.
Last stated 21 Nov 2026 · yesterday
Holds François Chollet
Same subject: transformers
A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017.
Last stated 6 Jan 2025 · 2 years ago
Holds Aishwarya Naresh Reganti
Same subject: transformers
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Last stated 12 Jun 2017 · 9 years ago
Holds Illia Polosukhin Jakob Uszkoreit Niki Parmar Ashish Vaswani Noam Shazeer Aidan N. Gomez Llion Jones Łukasz Kaiser
Same subject: neural networks
If the goal is just to engineer a good system, there is no reason not to use backpropagation.
Last stated 27 Mar 2023 · 4 years ago
Holds Ilya Sutskever
Same subject: neural networks
A curve, which is what a deep learning model is, is a very bad substrate for discrete computation.
Last stated 6 Nov 2024 · 2 years ago
Holds François Chollet
Same subject: neural networks
A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable.
Last stated 10 Dec 2015 · 11 years ago
Holds Jian Sun Kaiming He