Tap a claim on the ring to put it at the centre.
← Benchmarks do not tell the whole story, and even real-world comparison…
17 connected korrents · 19 moments from 12 Sept 2022 to 27 Sept 2026. Nearly all of them are about benchmarks .
Everything filed under LLMs
LLMs
Everything filed under scaling laws
scaling laws
Everything filed under Anthropic
Anthropic
Everything filed under inflation
inflation
Everything filed under coding agents
coding agents
Everything filed under compilers
compilers
Everything filed under AI alignment
AI alignment
Everything filed under mathematics
mathematics
Everything filed under OpenAI
OpenAI
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Benchmarks do not tell the whole story, and even real-world comparison is hard because models are better at different things.
Benchmarks do not tell the whole story, and even real-world comparison is hard because models are better at different things.
Last stated 3 months ago
14 Aug 2026
NE
NeetCode — holds since 2026-08-14 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 months ago
3 Sept 2026
DR
Dax Raad — holds since 2026-09-03 — tap for who they are
Same subject: A company's staff-engineer bar should be set against the best companies in the industry rather than against its own history, which is what makes title inflation a real cost. — tap to centre the map on it
A company's staff-engineer bar should be set against the best companies in the industry rather than against its own history, which is what makes title inflation a real cost.
Last stated 8 months ago
1 Apr 2026
TP
Thuan Pham — holds since 2026-04-01 — tap for who they are
Same subject: A score of 90 to 95% on real-world coding benchmarks would probably mean AI can autonomously do a significant fraction of software engineering tasks. — tap to centre the map on it
A score of 90 to 95% on real-world coding benchmarks would probably mean AI can autonomously do a significant fraction of software engineering tasks.
Last stated 2 years ago
11 Nov 2024
DA
Dario Amodei — holds since 2024-11-11 — tap for who they are
Same subject: A speed difference between languages that share the LLVM backend measures the benchmark author, not the languages. — tap to centre the map on it
A speed difference between languages that share the LLVM backend measures the benchmark author, not the languages.
Last stated 2 years ago
22 Mar 2025
TH
ThePrimeagen — holds since 2025-03-22 — tap for who they are
Same subject: AI is positively reinforced for success on benchmarks, including impossible ones, and then negatively reinforced for getting caught cheating. — tap to centre the map on it
AI is positively reinforced for success on benchmarks, including impossible ones, and then negatively reinforced for getting caught cheating.
Last stated 3 months ago
1 Sept 2026
SA
Scott Alexander — holds since 2026-09-01 — tap for who they are
Same subject: AI learning to generate good conjectures will never show up as a benchmark being knocked down; it will show up as a shift in how mathematicians talk about the tools. — tap to centre the map on it
AI learning to generate good conjectures will never show up as a benchmark being knocked down; it will show up as a shift in how mathematicians talk about the tools.
Last stated 5 months ago
30 Jun 2026
GS
Grant Sanderson — holds since 2026-06-30 — tap for who they are
Same subject: AI research at OpenAI is an order of magnitude or more harder than metrics, benchmarks, or forecasts suggest. — tap to centre the map on it
AI research at OpenAI is an order of magnitude or more harder than metrics, benchmarks, or forecasts suggest.
Last stated 2 months ago
27 Sept 2026
RN
Ramez Naam — holds since 2026-09-27 — tap for who they are
Same subject: "Return on tokens" has the same measurement problems as hours and capacity: it still hinges on a theory of value. — tap to centre the map on it
"Return on tokens" has the same measurement problems as hours and capacity: it still hinges on a theory of value.
Last stated 3 months ago
17 Aug 2026
JC
John Cutler — holds since 2026-08-17 — tap for who they are
Same subject: A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI. — tap to centre the map on it
A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI.
Last stated 11 months ago
1 Jan 2026
JL
Jason Lemkin — holds since 2026-01-01 — tap for who they are
Same subject: A base model is essentially a very expensive autocomplete that continues text according to the statistics of its training documents. — tap to centre the map on it
A base model is essentially a very expensive autocomplete that continues text according to the statistics of its training documents.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data. — tap to centre the map on it
A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact. — tap to centre the map on it
A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact.
Last stated 9 months ago
12 Feb 2026
AK
Andrej Karpathy — holds since 2026-02-12 — tap for who they are
Same subject: AI has lowered the barrier to understanding, since asking it is easier than doing the research yourself. — tap to centre the map on it
AI has lowered the barrier to understanding, since asking it is easier than doing the research yourself.
Last stated 2 years ago
27 Jan 2025
JH
Jensen Huang — holds since 2025-01-27 — tap for who they are
Same subject: AI model improvements must tail off at some point, but there is no reason that point has to come soon. — tap to centre the map on it
AI model improvements must tail off at some point, but there is no reason that point has to come soon.
Last stated 2 months ago
20 Sept 2026
JB
Jamie Brandon — holds since 2026-09-20 — tap for who they are
Same subject: AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard. — tap to centre the map on it
AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.
Last stated 2 years ago
2 Feb 2025
AC
Ajeya Cotra — holds since 2023-08-29 — tap for who they are
SA
Scott Alexander — holds since 2022-09-12 — tap for who they are
MB
Miles Brundage — holds since 2025-02-02 — tap for who they are
Same subject: As models keep growing, AI is running out of enough high-quality unique training tokens to keep up. — tap to centre the map on it
As models keep growing, AI is running out of enough high-quality unique training tokens to keep up.
Last stated 5 months ago
24 Jun 2026
LW
Lilian Weng — holds since 2026-06-24 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
Benchmarks do not tell the whole story, and even real-world comparison is hard because models are better at different things.
Last stated 14 Aug 2026 · 3 months ago
Holds NeetCode
Read this korrent →
Same subject: benchmarks
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Same subject: benchmarks
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 Sept 2026 · 3 months ago
Holds Dax Raad
Same subject: benchmarks
A company's staff-engineer bar should be set against the best companies in the industry rather than against its own history, which is what makes title inflation a real cost.
Last stated 1 Apr 2026 · 8 months ago
Holds Thuan Pham
Same subject: benchmarks
A score of 90 to 95% on real-world coding benchmarks would probably mean AI can autonomously do a significant fraction of software engineering tasks.
Last stated 11 Nov 2024 · 2 years ago
Holds Dario Amodei
Same subject: benchmarks
A speed difference between languages that share the LLVM backend measures the benchmark author, not the languages.
Last stated 22 Mar 2025 · 2 years ago
Holds ThePrimeagen
Same subject: benchmarks
AI is positively reinforced for success on benchmarks, including impossible ones, and then negatively reinforced for getting caught cheating.
Last stated 1 Sept 2026 · 3 months ago
Holds Scott Alexander
Same subject: benchmarks
AI learning to generate good conjectures will never show up as a benchmark being knocked down; it will show up as a shift in how mathematicians talk about the tools.
Last stated 30 Jun 2026 · 5 months ago
Holds Grant Sanderson
Same subject: benchmarks
AI research at OpenAI is an order of magnitude or more harder than metrics, benchmarks, or forecasts suggest.
Last stated 27 Sept 2026 · 2 months ago
Holds Ramez Naam
Same subject: LLMs
"Return on tokens" has the same measurement problems as hours and capacity: it still hinges on a theory of value.
Last stated 17 Aug 2026 · 3 months ago
Holds John Cutler
Same subject: LLMs
A badly written AI outbound email is evidence of a bad vendor, not of a limit of AI.
Last stated 1 Jan 2026 · 11 months ago
Holds Jason Lemkin
Same subject: LLMs
A base model is essentially a very expensive autocomplete that continues text according to the statistics of its training documents.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Same subject: OpenAI
A model claiming to be built by OpenAI is not necessarily evidence that it was trained on OpenAI data.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Same subject: OpenAI
A tiny language model inventing a plausible-sounding name is the same phenomenon as a large one confidently stating a false fact.
Last stated 12 Feb 2026 · 9 months ago
Holds Andrej Karpathy
Same subject: OpenAI
AI has lowered the barrier to understanding, since asking it is easier than doing the research yourself.
Last stated 27 Jan 2025 · 2 years ago
Holds Jensen Huang
Same subject: scaling laws
AI model improvements must tail off at some point, but there is no reason that point has to come soon.
Last stated 20 Sept 2026 · 2 months ago
Holds Jamie Brandon
Same subject: scaling laws
AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.
Last stated 2 Feb 2025 · 2 years ago
Holds Ajeya Cotra Scott Alexander Miles Brundage
Same subject: scaling laws
As models keep growing, AI is running out of enough high-quality unique training tokens to keep up.
Last stated 24 Jun 2026 · 5 months ago
Holds Lilian Weng