Tap a claim on the ring to put it at the centre.
← Running out of data will probably not block scaling, because data can be found and generated in many ways.
17 connected korrents · 17 moments from 10 Dec 2015 to 10 Oct 2026.
Everything filed under scaling laws
scaling laws
Everything filed under LLMs
LLMs
Everything filed under reinforcement learning
reinforcement learning
Everything filed under neural networks
neural networks
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Running out of data will probably not block scaling, because data can be found and generated in many ways.
Running out of data will probably not block scaling, because data can be found and generated in many ways.
Last stated 3 years ago
8 Aug 2023
DA
Dario Amodei — holds since 2023-08-08 — tap for who they are
Same subject: Scaling laws are still working, but the models people actually use run a few months behind the maximum capability Google can deliver, because the biggest model is too slow and expensive to serve. — tap to centre the map on it
Scaling laws are still working, but the models people actually use run a few months behind the maximum capability Google can deliver, because the biggest model is too slow and expensive to serve.
Last stated a year ago
5 Jun 2025
SP
Sundar Pichai — holds since 2025-06-05 — tap for who they are
Same subject: AI will not run out of data, because there is already enough real-world data to build the simulators that generate more of it. — tap to centre the map on it
AI will not run out of data, because there is already enough real-world data to build the simulators that generate more of it.
Last stated a year ago
23 Jul 2025
DH
Demis Hassabis — holds since 2025-07-23 — tap for who they are
Same subject: Progress is not driven mainly by material resources, so running out of them is not what would stop it. — tap to centre the map on it
Progress is not driven mainly by material resources, so running out of them is not what would stop it.
Last stated 6 years ago
12 Sept 2020
JC
Jason Crawford — holds since 2020-09-12 — tap for who they are
Same subject: Scaling will probably continue, and there is something to it that has not yet been explained on a theoretical basis. — tap to centre the map on it
Scaling will probably continue, and there is something to it that has not yet been explained on a theoretical basis.
Last stated 2 years ago
11 Nov 2024
DA
Dario Amodei — holds since 2024-11-11 — tap for who they are
Same subject: Training data is still plentiful for now, but it will eventually run out. — tap to centre the map on it
Training data is still plentiful for now, but it will eventually run out.
Last stated 4 years ago
27 Mar 2023
IS
Ilya Sutskever — holds since 2023-03-27 — tap for who they are
Same subject: With continued scaling, models will keep improving across the board, with no area where they are not starting to make progress. — tap to centre the map on it
With continued scaling, models will keep improving across the board, with no area where they are not starting to make progress.
Last stated 3 years ago
8 Aug 2023
DA
Dario Amodei — holds since 2023-08-08 — tap for who they are
Same subject: Training is no longer limited by data but by compute, because most of the data models learn from is now synthetic. — tap to centre the map on it
Training is no longer limited by data but by compute, because most of the data models learn from is now synthetic.
Last stated 8 months ago
23 Mar 2026
JH
Jensen Huang — holds since 2026-03-23 — tap for who they are
Same subject: The scaling law for world models was a conviction held in advance, not a discovery; the devils were always in the architecture choices and the data mixture. — tap to centre the map on it
The scaling law for world models was a conviction held in advance, not a discovery; the devils were always in the architecture choices and the data mixture.
Last stated 3 months ago
4 Sept 2026
FL
Fei-Fei Li — holds since 2026-09-04 — tap for who they are
Same subject: "Return on tokens" has the same measurement problems as hours and capacity: it still hinges on a theory of value. — tap to centre the map on it
"Return on tokens" has the same measurement problems as hours and capacity: it still hinges on a theory of value.
Last stated 3 months ago
17 Aug 2026
JC
John Cutler — holds since 2026-08-17 — tap for who they are
Same subject: A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI. — tap to centre the map on it
A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI.
Last stated 11 months ago
1 Jan 2026
JL
Jason Lemkin — holds since 2026-01-01 — tap for who they are
Same subject: A base model is essentially a very expensive autocomplete that continues text according to the statistics of its training documents. — tap to centre the map on it
A base model is essentially a very expensive autocomplete that continues text according to the statistics of its training documents.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: A feedback loop that reinforces a behavior pulls it toward whatever improves the loop's own score. — tap to centre the map on it
A feedback loop that reinforces a behavior pulls it toward whatever improves the loop's own score.
Last stated 3 months ago
26 Aug 2026
HK
Henrik Karlsson — holds since 2026-08-26 — tap for who they are
Same subject: A format reward should only be added when the answer is already correct. — tap to centre the map on it
A format reward should only be added when the answer is already correct.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards. — tap to centre the map on it
A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards.
Last stated 2 months ago
3 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-03 — tap for who they are
Same subject: A curve, which is what a deep learning model is, is a very bad substrate for discrete computation. — tap to centre the map on it
A curve, which is what a deep learning model is, is a very bad substrate for discrete computation.
Last stated 2 years ago
6 Nov 2024
FC
François Chollet — holds since 2024-11-06 — tap for who they are
Same subject: A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. — tap to centre the map on it
A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models.
Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. — tap to centre the map on it
A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable.
Last stated 11 years ago
10 Dec 2015
JS
Jian Sun — holds since 2015-12-10 — tap for who they are
KH
Kaiming He — holds since 2015-12-10 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
Running out of data will probably not block scaling, because data can be found and generated in many ways.
Last stated 8 Aug 2023 · 3 years ago
Holds Dario Amodei
Read this korrent →
Similar wording
Scaling laws are still working, but the models people actually use run a few months behind the maximum capability Google can deliver, because the biggest model is too slow and expensive to serve.
Last stated 5 Jun 2025 · a year ago
Holds Sundar Pichai
Similar wording
AI will not run out of data, because there is already enough real-world data to build the simulators that generate more of it.
Last stated 23 Jul 2025 · a year ago
Holds Demis Hassabis
Similar wording
Progress is not driven mainly by material resources, so running out of them is not what would stop it.
Last stated 12 Sept 2020 · 6 years ago
Holds Jason Crawford
Similar wording
Scaling will probably continue, and there is something to it that has not yet been explained on a theoretical basis.
Last stated 11 Nov 2024 · 2 years ago
Holds Dario Amodei
Similar wording
Training data is still plentiful for now, but it will eventually run out.
Last stated 27 Mar 2023 · 4 years ago
Holds Ilya Sutskever
Similar wording
With continued scaling, models will keep improving across the board, with no area where they are not starting to make progress.
Last stated 8 Aug 2023 · 3 years ago
Holds Dario Amodei
Similar wording
Training is no longer limited by data but by compute, because most of the data models learn from is now synthetic.
Last stated 23 Mar 2026 · 8 months ago
Holds Jensen Huang
Similar wording
The scaling law for world models was a conviction held in advance, not a discovery; the devils were always in the architecture choices and the data mixture.
Last stated 4 Sept 2026 · 3 months ago
Holds Fei-Fei Li
Same subject: LLMs
"Return on tokens" has the same measurement problems as hours and capacity: it still hinges on a theory of value.
Last stated 17 Aug 2026 · 3 months ago
Holds John Cutler
Same subject: LLMs
A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI.
Last stated 1 Jan 2026 · 11 months ago
Holds Jason Lemkin
Same subject: LLMs
A base model is essentially a very expensive autocomplete that continues text according to the statistics of its training documents.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Same subject: reinforcement learning
A feedback loop that reinforces a behavior pulls it toward whatever improves the loop's own score.
Last stated 26 Aug 2026 · 3 months ago
Holds Henrik Karlsson
Same subject: reinforcement learning
A format reward should only be added when the answer is already correct.
Last stated 10 Oct 2026 · a month ago
Holds Sebastian Raschka
Same subject: reinforcement learning
A model should not be called a reasoning model unless it has been trained with reinforcement learning with verifiable rewards.
Last stated 3 Oct 2026 · 2 months ago
Holds Sebastian Raschka
Same subject: neural networks
A curve, which is what a deep learning model is, is a very bad substrate for discrete computation.
Last stated 6 Nov 2024 · 2 years ago
Holds François Chollet
Same subject: neural networks
A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models.
Last stated 11 Oct 2018 · 8 years ago
Holds Kristina Toutanova Ming-Wei Chang Kenton Lee Jacob Devlin
Same subject: neural networks
A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable.
Last stated 10 Dec 2015 · 11 years ago
Holds Jian Sun Kaiming He