Tap a claim on the ring to put it at the centre.
← Reinforcement learning only works once a model already knows…
17 connected korrents · 16 moments from 21 May 2024 to 15 Sept 2026. Nearly all of them are about reinforcement learning .
Everything filed under robotics
robotics
Everything filed under scaling laws
scaling laws
Everything filed under AI agents
AI agents
Everything filed under AI alignment
AI alignment
Everything filed under self-driving cars
self-driving cars
Everything filed under neural networks
neural networks Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Reinforcement learning only works once a model already knows something, which is why robots must be pre-trained by imitation first. Reinforcement learning only works once a model already knows something, which is why robots must be pre-trained by imitation first. Last stated a year ago
12 Sept 2025
SL
Sergey Levine — holds since 2025-09-12 — tap for who they are
Same subject: One pre-trained robot model now matches or beats the specialists that were fine-tuned with reinforcement learning for the very tasks they were built for. — tap to centre the map on it One pre-trained robot model now matches or beats the specialists that were fine-tuned with reinforcement learning for the very tasks they were built for. Last stated 2 months ago
12 Aug 2026
CF
Chelsea Finn — holds since 2026-08-12 — tap for who they are
Same subject: Reinforcement learning cannot be scaled for robots the way it was for language models, because every attempt spends real robot hours instead of data-centre compute. — tap to centre the map on it Reinforcement learning cannot be scaled for robots the way it was for language models, because every attempt spends real robot hours instead of data-centre compute. Last stated 2 months ago
12 Aug 2026
CF
Chelsea Finn — holds since 2026-08-12 — tap for who they are
Same subject: A verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well. — tap to centre the map on it A verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well. Last stated 11 months ago
17 Nov 2025
AK
Andrej Karpathy — holds since 2025-11-17 — tap for who they are
Same subject: Capability does not generalise for free: a model that will move mountains on an agentic task still tells the same bad joke it told five years ago. — tap to centre the map on it Capability does not generalise for free: a model that will move mountains on an agentic task still tells the same bad joke it told five years ago. Last stated 7 months ago
20 Mar 2026
AK
Andrej Karpathy — holds since 2026-03-20 — tap for who they are
Same subject: Current models are unsafe because they are not smart enough, not because they are too smart. — tap to centre the map on it Current models are unsafe because they are not smart enough, not because they are too smart. Last stated a month ago
13 Sept 2026
FC
François Chollet — holds since 2026-09-13 — tap for who they are
Same subject: Humans barely use reinforcement learning for intelligence — what RL they do use goes into motor tasks, not problem solving. — tap to centre the map on it Humans barely use reinforcement learning for intelligence — what RL they do use goes into motor tasks, not problem solving. Last stated a year ago
17 Oct 2025
AK
Andrej Karpathy — holds since 2025-10-17 — tap for who they are
Same subject: Humans keep their place in AI as judges rather than authors, because telling which of two answers is better is far easier than writing a good one. — tap to centre the map on it Humans keep their place in AI as judges rather than authors, because telling which of two answers is better is far easier than writing a good one. Last stated 2 years ago
3 Feb 2025
NL
Nathan Lambert — holds since 2025-02-03 — tap for who they are
Same subject: It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning. — tap to centre the map on it It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning. Last stated 2 years ago
28 Nov 2024
LW
Lilian Weng — holds since 2024-11-28 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 7 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
Same subject: A five-year-old's vision is already good enough to drive a car, on a tiny and strikingly undiverse amount of data. — tap to centre the map on it A five-year-old's vision is already good enough to drive a car, on a tiny and strikingly undiverse amount of data. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A good robotaxi demonstration in one city does not mean the company is ready to scale the service. — tap to centre the map on it A good robotaxi demonstration in one city does not mean the company is ready to scale the service. Last stated a year ago
8 Jul 2025
TL
Timothy B. Lee — holds since 2025-07-08 — tap for who they are
Same subject: A huge fleet collecting driving data is not a silver bullet for self-driving, because the data arrives unlabelled. — tap to centre the map on it A huge fleet collecting driving data is not a silver bullet for self-driving, because the data arrives unlabelled. Last stated 2 years ago
21 May 2024
TL
Timothy B. Lee — holds since 2024-05-21 — tap for who they are
Same subject: A collection of AI agents does not automatically form a functioning organization any more than a group of smart people does. — tap to centre the map on it A collection of AI agents does not automatically form a functioning organization any more than a group of smart people does. Last stated 5 months ago
21 May 2026
RK
Rohit Krishnan — holds since 2026-05-21 — tap for who they are
Same subject: A company cannot be led by machines, because nobody would have recourse against them. — tap to centre the map on it A company cannot be led by machines, because nobody would have recourse against them. Last stated a month ago
15 Sept 2026
TL
Tobias Lütke — holds since 2026-09-15 — tap for who they are
Same subject: A computer-using agent needs a machine of its own, or you will spend the day fighting it for the mouse cursor. — tap to centre the map on it A computer-using agent needs a machine of its own, or you will spend the day fighting it for the mouse cursor. Last stated 2 months ago
10 Aug 2026
PS
Peter Steinberger — holds since 2026-08-10 — tap for who they are
At the centre
Reinforcement learning only works once a model already knows something, which is why robots must be pre-trained by imitation first.
Holds Read this korrent →
One pre-trained robot model now matches or beats the specialists that were fine-tuned with reinforcement learning for the very tasks they were built for.
Holds
Reinforcement learning cannot be scaled for robots the way it was for language models, because every attempt spends real robot hours instead of data-centre compute.
Holds
A verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well.
Holds
Capability does not generalise for free: a model that will move mountains on an agentic task still tells the same bad joke it told five years ago.
Holds
Current models are unsafe because they are not smart enough, not because they are too smart.
Holds
Humans barely use reinforcement learning for intelligence — what RL they do use goes into motor tasks, not problem solving.
Holds
Humans keep their place in AI as judges rather than authors, because telling which of two answers is better is far easier than writing a good one.
Holds
It is fundamentally difficult to design a reward function that accurately captures the intended goal in reinforcement learning.
Holds
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Holds
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds
A five-year-old's vision is already good enough to drive a car, on a tiny and strikingly undiverse amount of data.
Holds
A good robotaxi demonstration in one city does not mean the company is ready to scale the service.
Holds
A huge fleet collecting driving data is not a silver bullet for self-driving, because the data arrives unlabelled.
Holds
A collection of AI agents does not automatically form a functioning organization any more than a group of smart people does.
Holds
A company cannot be led by machines, because nobody would have recourse against them.
Holds
A computer-using agent needs a machine of its own, or you will spend the day fighting it for the mouse cursor.
Holds