Tap a claim on the ring to put it at the centre.
← GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive.
17 connected korrents · 18 moments from 12 Sept 2022 to 2 Nov 2026.
Everything filed under LLMs
LLMs
Everything filed under Anthropic
Anthropic
Everything filed under scaling laws
scaling laws
Everything filed under open source
open source
Everything filed under neural networks
neural networks
Everything filed under AI alignment
AI alignment
Everything filed under AI and human skill
AI and human skill
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive.
GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: Training in fp16 required gradient scaling, which is annoying and an extra source of state and complexity. — tap to centre the map on it
Training in fp16 required gradient scaling, which is annoying and an extra source of state and complexity.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it. — tap to centre the map on it
Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it.
Last stated 6 months ago
19 May 2026
TG
Tommy Geoco — holds since 2026-05-19 — tap for who they are
Same subject: There will probably be a few steps between GPT-4 and AI that could pose a takeover risk, in a ramp-up that is terrifyingly fast but fairly continuous. — tap to centre the map on it
There will probably be a few steps between GPT-4 and AI that could pose a takeover risk, in a ramp-up that is terrifyingly fast but fairly continuous.
Last stated 4 years ago
12 May 2023
AC
Ajeya Cotra — holds since 2023-05-12 — tap for who they are
Same subject: AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard. — tap to centre the map on it
AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.
Last stated 2 years ago
2 Feb 2025
AC
Ajeya Cotra — holds since 2023-08-29 — tap for who they are
SA
Scott Alexander — holds since 2022-09-12 — tap for who they are
MB
Miles Brundage — holds since 2025-02-02 — tap for who they are
Same subject: Math, code and multilingual data in GPT-2's training set may have taken capacity away from English performance. — tap to centre the map on it
Math, code and multilingual data in GPT-2's training set may have taken capacity away from English performance.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: The gap between GPT-5 and GPT-4 will be as large as the gap between GPT-4 and GPT-3, and the job is to live a few years ahead and remember that today's tools will look terrible from there. — tap to centre the map on it
The gap between GPT-5 and GPT-4 will be as large as the gap between GPT-4 and GPT-3, and the job is to live a few years ahead and remember that today's tools will look terrible from there.
Last stated 2 months ago
1 Oct 2026
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
ZM
Zvi Mowshowitz — holds since 2026-10-01 — tap for who they are
Same subject: AI will not make you world-class at a skill, but it takes a near-beginner to roughly the sixtieth or seventieth percentile of it today. — tap to centre the map on it
AI will not make you world-class at a skill, but it takes a near-beginner to roughly the sixtieth or seventieth percentile of it today.
Last stated a year ago
21 Sept 2025
JZ
Julie Zhuo — holds since 2025-09-21 — tap for who they are
Same subject: OpenAI will likely achieve a sense of fun in its models comparable to Claude Opus 5.5 by the end of the year. — tap to centre the map on it
OpenAI will likely achieve a sense of fun in its models comparable to Claude Opus 5.5 by the end of the year.
Last stated 4 weeks ago
28 Oct 2026
SW
Simon Willison — holds since 2026-10-28 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 12 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A CPU and memory crunch is coming. — tap to centre the map on it
A CPU and memory crunch is coming.
Last stated 4 weeks ago
26 Oct 2026
DM
Dillon Mulroy — holds since 2026-10-26 — tap for who they are
Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it
A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it
A codec does not really exist until everyone can decode it.
Last stated 6 months ago
31 May 2026
JK
Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are
Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it
A company that will not accept a project’s terms is free to go and use a more permissive project instead.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
Same subject: "I don't know" becomes a kind of superpower when you can work with Claude Code to figure things out. — tap to centre the map on it
"I don't know" becomes a kind of superpower when you can work with Claude Code to figure things out.
Last stated 8 months ago
19 Mar 2026
KP
Kris Puckett — holds since 2026-03-19 — tap for who they are
Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 months ago
3 Sept 2026
DR
Dax Raad — holds since 2026-09-03 — tap for who they are
Same subject: A large share of Anthropic's revenue being consumption-based rather than subscriptions raises the question of how resilient that revenue is. — tap to centre the map on it
A large share of Anthropic's revenue being consumption-based rather than subscriptions raises the question of how resilient that revenue is.
Last stated 3 weeks ago
2 Nov 2026
AA
Azeem Azhar — holds since 2026-11-02 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive.
Last stated 9 Jun 2024 · 2 years ago
Holds Andrej Karpathy
Read this korrent →
Similar wording
Training in fp16 required gradient scaling, which is annoying and an extra source of state and complexity.
Last stated 9 Jun 2024 · 2 years ago
Holds Andrej Karpathy
Similar wording
Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it.
Last stated 19 May 2026 · 6 months ago
Holds Tommy Geoco
Similar wording
There will probably be a few steps between GPT-4 and AI that could pose a takeover risk, in a ramp-up that is terrifyingly fast but fairly continuous.
Last stated 12 May 2023 · 4 years ago
Holds Ajeya Cotra
Similar wording
AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.
Last stated 2 Feb 2025 · 2 years ago
Holds Ajeya Cotra Scott Alexander Miles Brundage
Similar wording
Math, code and multilingual data in GPT-2's training set may have taken capacity away from English performance.
Last stated 9 Jun 2024 · 2 years ago
Holds Andrej Karpathy
Similar wording
The gap between GPT-5 and GPT-4 will be as large as the gap between GPT-4 and GPT-3, and the job is to live a few years ahead and remember that today's tools will look terrible from there.
Last stated 1 Oct 2026 · 2 months ago
Holds Sam Altman Zvi Mowshowitz
Similar wording
AI will not make you world-class at a skill, but it takes a near-beginner to roughly the sixtieth or seventieth percentile of it today.
Last stated 21 Sept 2025 · a year ago
Holds Julie Zhuo
Similar wording
OpenAI will likely achieve a sense of fun in its models comparable to Claude Opus 5.5 by the end of the year.
Last stated 28 Oct 2026 · 4 weeks ago
Holds Simon Willison
Same subject: scaling laws
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 25 Nov 2025 · 12 months ago
Holds Ilya Sutskever
Same subject: scaling laws
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Same subject: scaling laws
A CPU and memory crunch is coming.
Last stated 26 Oct 2026 · 4 weeks ago
Holds Dillon Mulroy
Same subject: open source
A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither.
Last stated 26 Sept 2024 · 2 years ago
Holds Matt Mullenweg
Same subject: open source
A codec does not really exist until everyone can decode it.
Last stated 31 May 2026 · 6 months ago
Holds Jean-Baptiste Kempf
Same subject: open source
A company that will not accept a project’s terms is free to go and use a more permissive project instead.
Last stated 26 Sept 2024 · 2 years ago
Holds Matt Mullenweg
Same subject: Anthropic
"I don't know" becomes a kind of superpower when you can work with Claude Code to figure things out.
Last stated 19 Mar 2026 · 8 months ago
Holds Kris Puckett
Same subject: Anthropic
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 Sept 2026 · 3 months ago
Holds Dax Raad
Same subject: Anthropic
A large share of Anthropic's revenue being consumption-based rather than subscriptions raises the question of how resilient that revenue is.
Last stated 2 Nov 2026 · 3 weeks ago
Holds Azeem Azhar