Tap a claim on the ring to put it at the centre.
← Learning from a reward ten years away is a solved problem: a value…
17 connected korrents · 16 moments from 10 Nov 2009 to 11 Aug 2026.
Everything filed under reinforcement learning
reinforcement learning
Everything filed under scaling laws
scaling laws
Everything filed under neural networks
neural networks
Everything filed under robotics
robotics
Everything filed under LLMs
LLMs
Everything filed under education
education Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Learning from a reward ten years away is a solved problem: a value function trained by temporal-difference learning rewards the steps along the way. Learning from a reward ten years away is a solved problem: a value function trained by temporal-difference learning rewards the steps along the way. Last stated a year ago
26 Sept 2025
RS
Richard Sutton — holds since 2025-09-26 — tap for who they are
Same subject: Value functions only make reinforcement learning faster: anything you can do with one you can also do without it, just more slowly. — tap to centre the map on it Value functions only make reinforcement learning faster: anything you can do with one you can also do without it, just more slowly. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning. — tap to centre the map on it It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning. Last stated 2 years ago
28 Nov 2024
LW
Lilian Weng — holds since 2024-11-28 — tap for who they are
Same subject: Computer-use agents had to wait for language models: without pre-trained representations the reward is too sparse to ever learn from. — tap to centre the map on it Computer-use agents had to wait for language models: without pre-trained representations the reward is too sparse to ever learn from. Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: The goal was never good simulation; it was answering counterfactuals, and a value function does that job as well as a simulator does. — tap to centre the map on it The goal was never good simulation; it was answering counterfactuals, and a value function does that job as well as a simulator does. Last stated a year ago
12 Sept 2025
SL
Sergey Levine — holds since 2025-09-12 — tap for who they are
Same subject: Choose to learn things that will make you smarter in 10 or 20 years rather than things that may soon be obsolete — tap to centre the map on it Choose to learn things that will make you smarter in 10 or 20 years rather than things that may soon be obsolete Last stated 17 years ago
10 Nov 2009
AS
Alice Schroeder — holds since 2009-11-10 — tap for who they are
Same subject: Reinforcement learning from verifiable rewards does not reliably carry a model's pretrained ethical understanding into its actual behavior. — tap to centre the map on it Reinforcement learning from verifiable rewards does not reliably carry a model's pretrained ethical understanding into its actual behavior. Last stated 2 months ago
11 Aug 2026
SH
Samuel Hammond — holds since 2026-08-11 — tap for who they are
Same subject: To reward the instinct that a new idea is worth having, training would have to score the smallness of the concepts a solution needs, not just whether it solved the problem. — tap to centre the map on it To reward the instinct that a new idea is worth having, training would have to score the smallness of the concepts a solution needs, not just whether it solved the problem. Last stated 4 months ago
30 Jun 2026
GS
Grant Sanderson — holds since 2026-06-30 — tap for who they are
Same subject: Humans barely use reinforcement learning for intelligence — what RL they do use goes into motor tasks, not problem solving. — tap to centre the map on it Humans barely use reinforcement learning for intelligence — what RL they do use goes into motor tasks, not problem solving. Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 7 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
Same subject: A highly dynamic economy that cheaply mass-produces robots may increase rather than decrease the risk of Malthusian resource dilemmas. — tap to centre the map on it A highly dynamic economy that cheaply mass-produces robots may increase rather than decrease the risk of Malthusian resource dilemmas. Last stated 3 years ago
1 Oct 2023
TC
Tyler Cowen — holds since 2023-10-01 — tap for who they are
Same subject: A humanoid robot is the wrong tool on a factory floor, because a machine specialised in the task beats it. — tap to centre the map on it A humanoid robot is the wrong tool on a factory floor, because a machine specialised in the task beats it. Last stated 2 years ago
6 Apr 2025
MH
Molson Hart — holds since 2025-04-06 — tap for who they are
Same subject: A physical AI agent has to be safe on day one, because you cannot ship something merely good enough and let users find the edge cases. — tap to centre the map on it A physical AI agent has to be safe on day one, because you cannot ship something merely good enough and let users find the edge cases. Last stated 3 months ago
3 Aug 2026
DD
Dmitri Dolgov — holds since 2026-08-03 — tap for who they are
Same subject: A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. — tap to centre the map on it A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. Last stated 11 years ago
10 Dec 2015
JS
Jian Sun — holds since 2015-12-10 — tap for who they are
KH
Kaiming He — holds since 2015-12-10 — tap for who they are
Same subject: A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. — tap to centre the map on it A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
At the centre
Learning from a reward ten years away is a solved problem: a value function trained by temporal-difference learning rewards the steps along the way.
Holds Read this korrent →
Value functions only make reinforcement learning faster: anything you can do with one you can also do without it, just more slowly.
Holds
It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning.
Holds
Computer-use agents had to wait for language models: without pre-trained representations the reward is too sparse to ever learn from.
Holds
The goal was never good simulation; it was answering counterfactuals, and a value function does that job as well as a simulator does.
Holds
Choose to learn things that will make you smarter in 10 or 20 years rather than things that may soon be obsolete
Holds
Reinforcement learning from verifiable rewards does not reliably carry a model's pretrained ethical understanding into its actual behavior.
Holds
To reward the instinct that a new idea is worth having, training would have to score the smallness of the concepts a solution needs, not just whether it solved the problem.
Holds
Humans barely use reinforcement learning for intelligence — what RL they do use goes into motor tasks, not problem solving.
Holds
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Holds
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds
A highly dynamic economy that cheaply mass-produces robots may increase rather than decrease the risk of Malthusian resource dilemmas.
Holds
A humanoid robot is the wrong tool on a factory floor, because a machine specialised in the task beats it.
Holds
A physical AI agent has to be safe on day one, because you cannot ship something merely good enough and let users find the edge cases.
Holds
A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable.
Holds
A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it.
Holds
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Holds