Tap a claim on the ring to put it at the centre.
← Most enterprises’ AI evals have terrible data quality.
17 connected korrents · 18 moments from 5 Nov 2019 to 31 Oct 2026.
Everything filed under benchmarks
benchmarks
Everything filed under LLMs
LLMs
Everything filed under ARC-AGI
ARC-AGI
Everything filed under software quality
software quality
Everything filed under AI alignment
AI alignment
Everything filed under startups
startups
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Most enterprises’ AI evals have terrible data quality.
Most enterprises’ AI evals have terrible data quality.
Last stated a month ago
19 Oct 2026
BF
Brendan Foody — holds since 2026-10-19 — tap for who they are
Same subject: Inconsistent output from an AI product is not necessarily bad; it is how AI behaves. — tap to centre the map on it
Inconsistent output from an AI product is not necessarily bad; it is how AI behaves.
Last stated 3 weeks ago
31 Oct 2026
MN
Marily Nika — holds since 2026-10-31 — tap for who they are
Same subject: Evals matter for AI products because bad launches hurt your brand and your users. — tap to centre the map on it
Evals matter for AI products because bad launches hurt your brand and your users.
Last stated 2 years ago
18 Mar 2025
JM
John Maeda — holds since 2025-03-18 — tap for who they are
Same subject: AI cannot get a product to the quality bar on its own, because it still works in the realm of averages. — tap to centre the map on it
AI cannot get a product to the quality bar on its own, because it still works in the realm of averages.
Last stated a year ago
16 Oct 2025
DF
Dylan Field — holds since 2025-10-16 — tap for who they are
Same subject: AI's business results are lagging because organisations have a low tolerance for non-determinism, not because the models are not good enough. — tap to centre the map on it
AI's business results are lagging because organisations have a low tolerance for non-determinism, not because the models are not good enough.
Last stated 8 months ago
11 Mar 2026
SY
Steve Yegge — holds since 2026-03-11 — tap for who they are
Same subject: Current AI models are worse colleagues than humans, because they routinely imply they did a task they did not actually do. — tap to centre the map on it
Current AI models are worse colleagues than humans, because they routinely imply they did a task they did not actually do.
Last stated 3 months ago
11 Aug 2026
RG
Ryan Greenblatt — holds since 2026-08-11 — tap for who they are
Same subject: Current AI coding models make poor decisions despite strong mechanical execution — tap to centre the map on it
Current AI coding models make poor decisions despite strong mechanical execution
Last stated 5 months ago
1 Jul 2026
JB
Jamie Brandon — holds since 2026-07-01 — tap for who they are
Same subject: Swearing at or being rude to an AI assistant makes it perform worse than treating it respectfully. — tap to centre the map on it
Swearing at or being rude to an AI assistant makes it perform worse than treating it respectfully.
Last stated 10 months ago
7 Jan 2026
SK
Steve Klabnik — holds since 2026-01-07 — tap for who they are
Same subject: Every AI startup presents its product as groundbreaking and perfect, when in truth it is bad about half the time. — tap to centre the map on it
Every AI startup presents its product as groundbreaking and perfect, when in truth it is bad about half the time.
Last stated 2 years ago
20 Aug 2024
PL
Pieter Levels — holds since 2024-08-20 — tap for who they are
Same subject: 'Everyone gets it wrong sometimes' is a bad argument when assessing track records. — tap to centre the map on it
'Everyone gets it wrong sometimes' is a bad argument when assessing track records.
Last stated 4 months ago
2 Aug 2026
AK
Adam Kucharski — holds since 2026-08-02 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 months ago
3 Sept 2026
DR
Dax Raad — holds since 2026-09-03 — tap for who they are
Same subject: A unit of measure needs defining, perhaps via a chain of increasingly hard problems, each pair of which a single model can solve. — tap to centre the map on it
A unit of measure needs defining, perhaps via a chain of increasingly hard problems, each pair of which a single model can solve.
Last stated 2 months ago
15 Sept 2026
PG
Paul Graham — holds since 2026-09-15 — tap for who they are
Same subject: One-shot or zero-shot settings are the fairest comparisons to human performance and important targets for future work. — tap to centre the map on it
One-shot or zero-shot settings are the fairest comparisons to human performance and important targets for future work.
Last stated 6 years ago
28 May 2020
DA
Dario Amodei — holds since 2020-05-28 — tap for who they are
Same subject: Performance gains from scaling on exam-style benchmarks are orthogonal to intelligence, because those benchmarks can be hacked by memorizing in advance. — tap to centre the map on it
Performance gains from scaling on exam-style benchmarks are orthogonal to intelligence, because those benchmarks can be hacked by memorizing in advance.
Last stated 2 years ago
6 Nov 2024
FC
François Chollet — holds since 2024-11-06 — tap for who they are
Same subject: The Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans. — tap to centre the map on it
The Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans.
Last stated 7 years ago
5 Nov 2019
FC
François Chollet — holds since 2019-11-05 — tap for who they are
Same subject: Combining an avocado and a chair into one image is evidence a model conceptually understands both, not that it memorised pictures of them. — tap to centre the map on it
Combining an avocado and a chair into one image is evidence a model conceptually understands both, not that it memorised pictures of them.
Last stated 3 months ago
12 Aug 2026
CF
Chelsea Finn — holds since 2026-08-12 — tap for who they are
Same subject: Passing ARC-AGI does not amount to achieving AGI: o3 still fails on some very easy tasks, indicating fundamental differences from human intelligence. — tap to centre the map on it
Passing ARC-AGI does not amount to achieving AGI: o3 still fails on some very easy tasks, indicating fundamental differences from human intelligence.
Last stated 2 years ago
20 Dec 2024
FC
François Chollet — holds since 2024-12-20 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
Most enterprises’ AI evals have terrible data quality.
Last stated 19 Oct 2026 · a month ago
Holds Brendan Foody
Read this korrent →
Similar wording
Inconsistent output from an AI product is not necessarily bad; it is how AI behaves.
Last stated 31 Oct 2026 · 3 weeks ago
Holds Marily Nika
Similar wording
Evals matter for AI products because bad launches hurt your brand and your users.
Last stated 18 Mar 2025 · 2 years ago
Holds John Maeda
Similar wording
AI cannot get a product to the quality bar on its own, because it still works in the realm of averages.
Last stated 16 Oct 2025 · a year ago
Holds Dylan Field
Similar wording
AI's business results are lagging because organisations have a low tolerance for non-determinism, not because the models are not good enough.
Last stated 11 Mar 2026 · 8 months ago
Holds Steve Yegge
Similar wording
Current AI models are worse colleagues than humans, because they routinely imply they did a task they did not actually do.
Last stated 11 Aug 2026 · 3 months ago
Holds Ryan Greenblatt
Similar wording
Current AI coding models make poor decisions despite strong mechanical execution
Last stated 1 Jul 2026 · 5 months ago
Holds Jamie Brandon
Similar wording
Swearing at or being rude to an AI assistant makes it perform worse than treating it respectfully.
Last stated 7 Jan 2026 · 10 months ago
Holds Steve Klabnik
Similar wording
Every AI startup presents its product as groundbreaking and perfect, when in truth it is bad about half the time.
Last stated 20 Aug 2024 · 2 years ago
Holds Pieter Levels
Same subject: benchmarks
'Everyone gets it wrong sometimes' is a bad argument when assessing track records.
Last stated 2 Aug 2026 · 4 months ago
Holds Adam Kucharski
Same subject: benchmarks
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 26 Jun 2026 · 5 months ago
Holds Noam Brown
Same subject: benchmarks
A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year.
Last stated 3 Sept 2026 · 3 months ago
Holds Dax Raad
Same subject: measuring intelligence
A unit of measure needs defining, perhaps via a chain of increasingly hard problems, each pair of which a single model can solve.
Last stated 15 Sept 2026 · 2 months ago
Holds Paul Graham
Same subject: measuring intelligence
One-shot or zero-shot settings are the fairest comparisons to human performance and important targets for future work.
Last stated 28 May 2020 · 6 years ago
Holds Dario Amodei
Same subject: measuring intelligence
Performance gains from scaling on exam-style benchmarks are orthogonal to intelligence, because those benchmarks can be hacked by memorizing in advance.
Last stated 6 Nov 2024 · 2 years ago
Holds François Chollet
Same subject: ARC-AGI
The Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans.
Last stated 5 Nov 2019 · 7 years ago
Holds François Chollet
Same subject: ARC-AGI
Combining an avocado and a chair into one image is evidence a model conceptually understands both, not that it memorised pictures of them.
Last stated 12 Aug 2026 · 3 months ago
Holds Chelsea Finn
Same subject: ARC-AGI
Passing ARC-AGI does not amount to achieving AGI: o3 still fails on some very easy tasks, indicating fundamental differences from human intelligence.
Last stated 20 Dec 2024 · 2 years ago
Holds François Chollet