korrents

On the map

Tap a claim on the ring to put it at the centre.

← The most important thing about evals is that they are actionable.

12 connected korrents · 13 moments from 5 Nov 2019 to 22 Oct 2026.

Everything filed under benchmarks benchmarks Everything filed under ARC-AGI ARC-AGI Everything filed under product management product management Everything filed under design design Everything filed under software quality software quality Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: The most important thing about evals is that they are actionable. The most important thing about evals isthat they are actionable. Last stated a month ago 22 Oct 2026 AR Aishwarya Naresh Reganti — holds since 2026-10-22 — tap for who they are Same subject: For a product person working with AI research, learning to write good evals is the most important thing. — tap to centre the map on it For a product person working with AIresearch, learning to write goodevals is the most important thing. Last stated 2 months ago 10 Sept 2026 TS Tara Seshan — holds since 2026-09-10 — tap for who they are Same subject: It is really important for designers to embrace evals, because that is how the system is created and it is the definition of good. — tap to centre the map on it It is really important for designersto embrace evals, because that ishow the system is created and it isthe definition of good. Last stated 5 months ago 24 Jun 2026 LC Loredana Crisan — holds since 2026-06-24 — tap for who they are Same subject: Evals matter for AI products because bad launches hurt your brand and your users. — tap to centre the map on it Evals matter for AI products becausebad launches hurt your brand andyour users. Last stated 2 years ago 18 Mar 2025 JM John Maeda — holds since 2025-03-18 — tap for who they are Same subject: 'Everyone gets it wrong sometimes' is a bad argument when assessing track records. — tap to centre the map on it 'Everyone gets it wrong sometimes'is a bad argument when assessingtrack records. Last stated 4 months ago 2 Aug 2026 AK Adam Kucharski — holds since 2026-08-02 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it A benchmark that ranks Claude Codelast while it stays first in use ismeasuring the wrong thing, and hasbeen for a year. Last stated 3 months ago 3 Sept 2026 DR Dax Raad — holds since 2026-09-03 — tap for who they are Same subject: A unit of measure needs defining, perhaps via a chain of increasingly hard problems, each pair of which a single model can solve. — tap to centre the map on it A unit of measure needs defining,perhaps via a chain of increasinglyhard problems, each pair of which asingle model can solve. Last stated 2 months ago 15 Sept 2026 PG Paul Graham — holds since 2026-09-15 — tap for who they are Same subject: One-shot or zero-shot settings are the fairest comparisons to human performance and important targets for future work. — tap to centre the map on it One-shot or zero-shot settings arethe fairest comparisons to humanperformance and important targetsfor future work. Last stated 6 years ago 28 May 2020 DA Dario Amodei — holds since 2020-05-28 — tap for who they are Same subject: Performance gains from scaling on exam-style benchmarks are orthogonal to intelligence, because those benchmarks can be hacked by memorizing in advance. — tap to centre the map on it Performance gains from scaling onexam-style benchmarks are orthogonalto intelligence, because thosebenchmarks can be hacked bymemorizing in advance. Last stated 2 years ago 6 Nov 2024 FC François Chollet — holds since 2024-11-06 — tap for who they are Same subject: The Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans. — tap to centre the map on it The Abstraction and Reasoning Corpuscan measure human-like general fluidintelligence and enable faircomparisons between AI systems andhumans. Last stated 7 years ago 5 Nov 2019 FC François Chollet — holds since 2019-11-05 — tap for who they are Same subject: Combining an avocado and a chair into one image is evidence a model conceptually understands both, not that it memorised pictures of them. — tap to centre the map on it Combining an avocado and a chairinto one image is evidence a modelconceptually understands both, notthat it memorised pictures of them. Last stated 3 months ago 12 Aug 2026 CF Chelsea Finn — holds since 2026-08-12 — tap for who they are Same subject: Passing ARC-AGI does not amount to achieving AGI: o3 still fails on some very easy tasks, indicating fundamental differences from human intelligence. — tap to centre the map on it Passing ARC-AGI does not amount toachieving AGI: o3 still fails onsome very easy tasks, indicatingfundamental differences from humanintelligence. Last stated 2 years ago 20 Dec 2024 FC François Chollet — holds since 2024-12-20 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre The most important thing about evals is that they are actionable. Last stated 22 Oct 2026 · a month ago Holds Aishwarya Naresh Reganti Read this korrent →