korrents

On the map

Tap a claim on the ring to put it at the centre.

← Evals are just testing: a way to test LLM outputs the way unit tests test software.

17 connected korrents · 15 moments from 18 Mar 2024 to 1 Nov 2026. Nearly all of them are about software quality.

Everything filed under LLMs LLMs Everything filed under coding agents coding agents Everything filed under OpenAI OpenAI Everything filed under scaling laws scaling laws Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Evals are just testing: a way to test LLM outputs the way unit tests test software. Evals are just testing: a way to test LLMoutputs the way unit tests test software. Last stated 2 years ago 18 Mar 2025 JM John Maeda — holds since 2025-03-18 — tap for who they are Same subject: Code quality still matters even as LLMs make writing code much faster — tap to centre the map on it Code quality still matters even asLLMs make writing code much faster Last stated 2 years ago 13 May 2025 DH David Heinemeier Hansson — holds since 2025-05-13 — tap for who they are Same subject: Code quality still matters even as LLMs make writing code much faster. — tap to centre the map on it Code quality still matters even asLLMs make writing code much faster. Last stated 4 months ago 10 Jul 2026 FS Fabien Sanglard — holds since 2026-07-10 — tap for who they are Same subject: 100% test coverage is the only correct test coverage. — tap to centre the map on it 100% test coverage is the onlycorrect test coverage. Last stated 2 years ago 24 Feb 2025 HS Hynek Schlawack — holds since 2025-02-24 — tap for who they are Same subject: A language-agnostic conformance suite is the most powerful thing you can hand a coding agent, because the whole instruction becomes: write code until these tests pass. — tap to centre the map on it A language-agnostic conformancesuite is the most powerful thing youcan hand a coding agent, because thewhole instruction becomes: writecode until these tests pass. Last stated 8 months ago 19 Mar 2026 SW Simon Willison — holds since 2026-03-19 — tap for who they are Same subject: A passing test suite is not evidence that the software works, so an agent must also be made to start the thing and exercise it the way a person would. — tap to centre the map on it A passing test suite is not evidencethat the software works, so an agentmust also be made to start the thingand exercise it the way a personwould. Last stated 8 months ago 19 Mar 2026 SW Simon Willison — holds since 2026-03-19 — tap for who they are Same subject: A test suite is not strictly necessary to rebuild something. — tap to centre the map on it A test suite is not strictlynecessary to rebuild something. Last stated 3 weeks ago 1 Nov 2026 AR Armin Ronacher — holds since 2026-11-01 — tap for who they are Same subject: Adding a config option so nobody breaks compounds into permutations no test suite can cover, because it is infinitely harder to evolve software that has users. — tap to centre the map on it Adding a config option so nobodybreaks compounds into permutationsno test suite can cover, because itis infinitely harder to evolvesoftware that has users. Last stated 3 months ago 10 Aug 2026 PS Peter Steinberger — holds since 2026-08-10 — tap for who they are Same subject: Agentic code review raises the floor but cannot be trusted, because the model reading the code is the same model that wrote it, and it will tell you the code is great. — tap to centre the map on it Agentic code review raises the floorbut cannot be trusted, because themodel reading the code is the samemodel that wrote it, and it willtell you the code is great. Last stated 4 months ago 15 Jul 2026 DH Dex Horthy — holds since 2026-07-15 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as anonprofit and bolt a for-profit armon later, whatever OpenAI's ownhistory suggests. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phasefor the internet but a momentaryindustry, and an AI people pay foris better because they know theanswers are not influenced byadvertisers. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because itwas one word: naming a researchdirection is what tells a wholefield what to do next. Last stated 12 months ago 25 Nov 2025 IS Ilya Sutskever — holds since 2025-11-25 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A CPU and memory crunch is coming. — tap to centre the map on it A CPU and memory crunch is coming. Last stated 4 weeks ago 26 Oct 2026 DM Dillon Mulroy — holds since 2026-10-26 — tap for who they are Same subject: "I don't know" becomes a kind of superpower when you can work with Claude Code to figure things out. — tap to centre the map on it "I don't know" becomes a kind ofsuperpower when you can work withClaude Code to figure things out. Last stated 8 months ago 19 Mar 2026 KP Kris Puckett — holds since 2026-03-19 — tap for who they are Same subject: A code-review agent works best with a clean context, sharing nothing with the coding agent beforehand. — tap to centre the map on it A code-review agent works best witha clean context, sharing nothingwith the coding agent beforehand. Last stated 7 months ago 22 Apr 2026 WY Walden Yan — holds since 2026-04-22 — tap for who they are Same subject: A coding agent does not learn from its mistakes the way a person does -- it repeats the same error indefinitely unless a human notices and writes it down. — tap to centre the map on it A coding agent does not learn fromits mistakes the way a person does-- it repeats the same errorindefinitely unless a human noticesand writes it down. Last stated 8 months ago 25 Mar 2026 MZ Mario Zechner — holds since 2026-03-25 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Evals are just testing: a way to test LLM outputs the way unit tests test software. Last stated 18 Mar 2025 · 2 years ago Holds John Maeda Read this korrent →