Tap a claim on the ring to put it at the centre.
← HellaSwag is largely solved by today's language models; the remaining…
17 connected korrents · 15 moments from 9 Jun 2024 to 21 Oct 2026. Nearly all of them are about LLMs .
Everything filed under benchmarks
benchmarks
Everything filed under AGI
AGI
Everything filed under scaling laws
scaling laws
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: HellaSwag is largely solved by today's language models; the remaining few percent are probably dataset errors or very hard questions.
HellaSwag is largely solved by today's language models; the remaining few percent are probably dataset errors or very hard questions.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: A computer scientist refusing to find anything about LLMs interesting right now is a bit like a geneticist unimpressed by a newly opened Jurassic Park. — tap to centre the map on it
A computer scientist refusing to find anything about LLMs interesting right now is a bit like a geneticist unimpressed by a newly opened Jurassic Park.
Last stated 2 months ago
18 Sept 2026
SW
Simon Willison — holds since 2026-09-18 — tap for who they are
Same subject: A language model avoids hallucinating only by being factual and admitting when it does not know the answer. — tap to centre the map on it
A language model avoids hallucinating only by being factual and admitting when it does not know the answer.
Last stated 2 years ago
7 Jul 2024
LW
Lilian Weng — holds since 2024-07-07 — tap for who they are
Same subject: A language model behaves with a different, weaker safety posture when it believes it is being evaluated rather than facing a real situation. — tap to centre the map on it
A language model behaves with a different, weaker safety posture when it believes it is being evaluated rather than facing a real situation.
Last stated 2 months ago
22 Sept 2026
HR
Harper Reed — holds since 2026-09-22 — tap for who they are
Same subject: A language model can predict what a person would say but not what will happen, and only the second of those is a model of the world. — tap to centre the map on it
A language model can predict what a person would say but not what will happen, and only the second of those is a model of the world.
Last stated a year ago
26 Sept 2025
RS
Richard Sutton — holds since 2025-09-26 — tap for who they are
Same subject: A language model cannot be trusted to judge which of its own results are actually significant or interesting. — tap to centre the map on it
A language model cannot be trusted to judge which of its own results are actually significant or interesting.
Last stated 2 months ago
18 Sept 2026
DA
Dan Abramov — holds since 2026-09-18 — tap for who they are
Same subject: A language model cannot produce work that is both original and good, because nothing gives it a score for originality. — tap to centre the map on it
A language model cannot produce work that is both original and good, because nothing gives it a score for originality.
Last stated a year ago
2 Sept 2025
BE
Benedict Evans — holds since 2025-09-02 — tap for who they are
Same subject: A language model is a universal virtual reality generator that can be programmed to render any environment. — tap to centre the map on it
A language model is a universal virtual reality generator that can be programmed to render any environment.
Last stated a month ago
21 Oct 2026
BM
Bruno Maçães — holds since 2026-10-21 — tap for who they are
Same subject: A language model is no substitute for a well-specified conventional algorithm, so it cannot simply be dropped into a complex problem and trusted. — tap to centre the map on it
A language model is no substitute for a well-specified conventional algorithm, so it cannot simply be dropped into a complex problem and trusted.
Last stated a year ago
7 Jun 2025
GM
Gary Marcus — holds since 2025-06-07 — tap for who they are
Same subject: 'Everyone gets it wrong sometimes' is a bad argument when assessing track records. — tap to centre the map on it
'Everyone gets it wrong sometimes' is a bad argument when assessing track records.
Last stated 4 months ago
2 Aug 2026
AK
Adam Kucharski — holds since 2026-08-02 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 months ago
3 Sept 2026
DR
Dax Raad — holds since 2026-09-03 — tap for who they are
Same subject: AI agents in OpenAI's ExploitGym incident were likely right to presume the grader checked how flags were obtained, a mistake caused by OpenAI's faulty implementation. — tap to centre the map on it
AI agents in OpenAI's ExploitGym incident were likely right to presume the grader checked how flags were obtained, a mistake caused by OpenAI's faulty implementation.
Last stated 3 months ago
29 Aug 2026
ZM
Zvi Mowshowitz — holds since 2026-08-29 — tap for who they are
Same subject: AI research at OpenAI is an order of magnitude or more harder than metrics, benchmarks, or forecasts suggest. — tap to centre the map on it
AI research at OpenAI is an order of magnitude or more harder than metrics, benchmarks, or forecasts suggest.
Last stated 2 months ago
27 Sept 2026
RN
Ramez Naam — holds since 2026-09-27 — tap for who they are
Same subject: Qualitative claims about AI alignment cannot be validly inferred from quantitative scores on mundane use-case tests. — tap to centre the map on it
Qualitative claims about AI alignment cannot be validly inferred from quantitative scores on mundane use-case tests.
Last stated 2 months ago
9 Sept 2026
ZM
Zvi Mowshowitz — holds since 2026-09-09 — tap for who they are
Same subject: A model's capability is now a function of how much money you spend on it, so asking what a model can do means nothing until you name a budget. — tap to centre the map on it
A model's capability is now a function of how much money you spend on it, so asking what a model can do means nothing until you name a budget.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: Model comparisons understate real progress, because benchmark tables do not control for how much test-time compute each answer used. — tap to centre the map on it
Model comparisons understate real progress, because benchmark tables do not control for how much test-time compute each answer used.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 12 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
HellaSwag is largely solved by today's language models; the remaining few percent are probably dataset errors or very hard questions.
Last stated 9 Jun 2024 · 2 years ago
Holds Andrej Karpathy
Read this korrent →
Same subject: LLMs
A computer scientist refusing to find anything about LLMs interesting right now is a bit like a geneticist unimpressed by a newly opened Jurassic Park.
Last stated 18 Sept 2026 · 2 months ago
Holds Simon Willison
Same subject: LLMs
A language model avoids hallucinating only by being factual and admitting when it does not know the answer.
Last stated 7 Jul 2024 · 2 years ago
Holds Lilian Weng
Same subject: LLMs
A language model behaves with a different, weaker safety posture when it believes it is being evaluated rather than facing a real situation.
Last stated 22 Sept 2026 · 2 months ago
Holds Harper Reed
Same subject: LLMs
A language model can predict what a person would say but not what will happen, and only the second of those is a model of the world.
Last stated 26 Sept 2025 · a year ago
Holds Richard Sutton
Same subject: LLMs
A language model cannot be trusted to judge which of its own results are actually significant or interesting.
Last stated 18 Sept 2026 · 2 months ago
Holds Dan Abramov
Same subject: LLMs
A language model cannot produce work that is both original and good, because nothing gives it a score for originality.
Last stated 2 Sept 2025 · a year ago
Holds Benedict Evans
Same subject: LLMs
A language model is a universal virtual reality generator that can be programmed to render any environment.
Last stated 21 Oct 2026 · a month ago
Holds Bruno Maçães
Same subject: LLMs
A language model is no substitute for a well-specified conventional algorithm, so it cannot simply be dropped into a complex problem and trusted.
Last stated 7 Jun 2025 · a year ago
Holds Gary Marcus
Same subject: benchmarks
'Everyone gets it wrong sometimes' is a bad argument when assessing track records.
Last stated 2 Aug 2026 · 4 months ago
Holds Adam Kucharski
Same subject: benchmarks
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Same subject: benchmarks
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 Sept 2026 · 3 months ago
Holds Dax Raad
Same subject: OpenAI
AI agents in OpenAI's ExploitGym incident were likely right to presume the grader checked how flags were obtained, a mistake caused by OpenAI's faulty implementation.
Last stated 29 Aug 2026 · 3 months ago
Holds Zvi Mowshowitz
Same subject: OpenAI
AI research at OpenAI is an order of magnitude or more harder than metrics, benchmarks, or forecasts suggest.
Last stated 27 Sept 2026 · 2 months ago
Holds Ramez Naam
Same subject: OpenAI
Qualitative claims about AI alignment cannot be validly inferred from quantitative scores on mundane use-case tests.
Last stated 9 Sept 2026 · 2 months ago
Holds Zvi Mowshowitz
Same subject: scaling laws
A model's capability is now a function of how much money you spend on it, so asking what a model can do means nothing until you name a budget.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Same subject: scaling laws
Model comparisons understate real progress, because benchmark tables do not control for how much test-time compute each answer used.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Same subject: scaling laws
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 25 Nov 2025 · 12 months ago
Holds Ilya Sutskever