korrents

On the map

Tap a claim on the ring to put it at the centre.

← AI model evaluations should move to agentic evals.

17 connected korrents · 18 moments from 15 Jan 2025 to 3 Nov 2026.

Everything filed under AI agents AI agents Everything filed under LLMs LLMs Everything filed under benchmarks benchmarks Everything filed under AI and jobs AI and jobs Everything filed under coding agents coding agents Everything filed under code generation code generation Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: AI model evaluations should move to agentic evals. AI model evaluations should move toagentic evals. Last stated 2 years ago 15 Jan 2025 TW Thomas Wolf — holds since 2025-01-15 — tap for who they are Same subject: AI is likely to become a more relevant issue before 2028. — tap to centre the map on it AI is likely to become a morerelevant issue before 2028. Last stated 2 months ago 6 Oct 2026 NS Nate Silver — holds since 2026-10-06 — tap for who they are Same subject: There will be no sudden fast takeoff in AI, even though model progress is happening faster than ever. — tap to centre the map on it There will be no sudden fast takeoffin AI, even though model progress ishappening faster than ever. Last stated 3 months ago 6 Sept 2026 AA Anish Acharya — holds since 2026-09-06 — tap for who they are Same subject: The next shift in AI will probably be agents that are more proactive toward you than you are toward them. — tap to centre the map on it The next shift in AI will probablybe agents that are more proactivetoward you than you are toward them. Last stated 2 months ago 8 Sept 2026 RU Roman Ugarte — holds since 2026-09-08 — tap for who they are Same subject: AI evals are the most important skill to learn for AI roles in 2026. — tap to centre the map on it AI evals are the most importantskill to learn for AI roles in 2026. Last stated 2 months ago 21 Sept 2026 AR Aishwarya Naresh Reganti — holds since 2026-09-21 — tap for who they are Same subject: The step change in AI coding in 2025 came from the harnesses and the tooling around the models, not from the models themselves. — tap to centre the map on it The step change in AI coding in 2025came from the harnesses and thetooling around the models, not fromthe models themselves. Last stated 3 months ago 12 Aug 2026 CM Charity Majors — holds since 2026-08-12 — tap for who they are Same subject: The current capabilities of existing AI models are barely being used and are often not even well understood. — tap to centre the map on it The current capabilities of existingAI models are barely being used andare often not even well understood. Last stated 2 months ago 18 Sept 2026 EM Ethan Mollick — holds since 2026-09-18 — tap for who they are Same subject: Future AI, say in fifteen years, will not be built on the LLM stack; it will have to move to symbolic learning. — tap to centre the map on it Future AI, say in fifteen years,will not be built on the LLM stack;it will have to move to symboliclearning. Last stated 4 months ago 7 Aug 2026 FC François Chollet — holds since 2026-08-07 — tap for who they are Same subject: AI-generated code became good enough to be practical by the end of 2025 — tap to centre the map on it AI-generated code became good enoughto be practical by the end of 2025 Last stated 2 months ago 4 Oct 2026 CW Chris Wellons — holds since 2026-10-04 — tap for who they are Same subject: AI agents in OpenAI's ExploitGym incident were likely right to presume the grader checked how flags were obtained, a mistake caused by OpenAI's faulty implementation. — tap to centre the map on it AI agents in OpenAI's ExploitGymincident were likely right topresume the grader checked how flagswere obtained, a mistake caused byOpenAI's faulty implementation. Last stated 3 months ago 29 Aug 2026 ZM Zvi Mowshowitz — holds since 2026-08-29 — tap for who they are Same subject: A codebase is the most important context an AI agent has, and letting many different models add to it without consistent patterns leads to trouble. — tap to centre the map on it A codebase is the most importantcontext an AI agent has, and lettingmany different models add to itwithout consistent patterns leads totrouble. Last stated 3 weeks ago 3 Nov 2026 DB Derek Briggs — holds since 2026-11-03 — tap for who they are Same subject: A collection of AI agents does not automatically form a functioning organization any more than a group of smart people does. — tap to centre the map on it A collection of AI agents does notautomatically form a functioningorganization any more than a groupof smart people does. Last stated 6 months ago 21 May 2026 RK Rohit Krishnan — holds since 2026-05-21 — tap for who they are Same subject: 'Everyone gets it wrong sometimes' is a bad argument when assessing track records. — tap to centre the map on it 'Everyone gets it wrong sometimes'is a bad argument when assessingtrack records. Last stated 4 months ago 2 Aug 2026 AK Adam Kucharski — holds since 2026-08-02 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it A benchmark that ranks Claude Codelast while it stays first in use ismeasuring the wrong thing, and hasbeen for a year. Last stated 3 months ago 3 Sept 2026 DR Dax Raad — holds since 2026-09-03 — tap for who they are Same subject: A real friend will eventually refuse to help, and later turn on you, if you are sufficiently evil or selfish. — tap to centre the map on it A real friend will eventually refuseto help, and later turn on you, ifyou are sufficiently evil orselfish. Last stated 2 months ago 20 Sept 2026 ZM Zvi Mowshowitz — holds since 2026-09-20 — tap for who they are Same subject: Agents become far more powerful only with a trust model that makes autonomous action safe. — tap to centre the map on it Agents become far more powerful onlywith a trust model that makesautonomous action safe. Last stated a month ago 18 Oct 2026 AK Alex Komoroske — holds since 2026-10-18 — tap for who they are Same subject: Agents should reach integrations only through locked packages of deterministic code that can do only what you want them to do. — tap to centre the map on it Agents should reach integrationsonly through locked packages ofdeterministic code that can do onlywhat you want them to do. Last stated 2 months ago 15 Sept 2026 KD Kent C. Dodds — holds since 2026-09-15 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre AI model evaluations should move to agentic evals. Last stated 15 Jan 2025 · 2 years ago Holds Thomas Wolf Read this korrent →