korrents

On the map

Tap a claim on the ring to put it at the centre.

← Judging one model worse than another from a one-shot, probabilistic…

17 connected korrents · 15 moments from 18 Mar 2024 to 31 Oct 2026.

Everything filed under LLMs LLMs Everything filed under AI writing AI writing Everything filed under open source open source Everything filed under OpenAI OpenAI Everything filed under AI alignment AI alignment Everything filed under Anthropic Anthropic Everything filed under benchmarks benchmarks Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it. Judging one model worse than another froma one-shot, probabilistic output on agiven day is not a good way to evaluateit. Last stated 6 months ago 19 May 2026 TG Tommy Geoco — holds since 2026-05-19 — tap for who they are Same subject: No single interpretability technique is reliable enough alone to justify a verdict like a model being unsafe; decorrelated techniques are needed. — tap to centre the map on it No single interpretability techniqueis reliable enough alone to justifya verdict like a model being unsafe;decorrelated techniques are needed. Last stated 11 months ago 20 Dec 2025 NN Neel Nanda — holds since 2025-12-20 — tap for who they are Same subject: Trusting model-written code you have not read became defensible only with Opus 4.5, and only for classes of problem you have already watched a model handle. — tap to centre the map on it Trusting model-written code you havenot read became defensible only withOpus 4.5, and only for classes ofproblem you have already watched amodel handle. Last stated 8 months ago 19 Mar 2026 SW Simon Willison — holds since 2026-03-19 — tap for who they are Same subject: Opus 4.5 does not seem likely to cause substantial damage, mainly because it probably cannot reliably hide things from its chain of thought. — tap to centre the map on it Opus 4.5 does not seem likely tocause substantial damage, mainlybecause it probably cannot reliablyhide things from its chain ofthought. Last stated 11 months ago 27 Dec 2025 NN Neel Nanda — holds since 2025-12-27 — tap for who they are Same subject: Inconsistent output from an AI product is not necessarily bad; it is how AI behaves. — tap to centre the map on it Inconsistent output from an AIproduct is not necessarily bad; itis how AI behaves. Last stated 3 weeks ago 31 Oct 2026 MN Marily Nika — holds since 2026-10-31 — tap for who they are Same subject: Running a model five times and keeping the best answer buys a higher benchmark score without buying a better model. — tap to centre the map on it Running a model five times andkeeping the best answer buys ahigher benchmark score withoutbuying a better model. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: Benchmarks do not tell the whole story, and even real-world comparison is hard because models are better at different things. — tap to centre the map on it Benchmarks do not tell the wholestory, and even real-worldcomparison is hard because modelsare better at different things. Last stated 3 months ago 14 Aug 2026 NE NeetCode — holds since 2026-08-14 — tap for who they are Same subject: Language models perform better on benchmark problems released before their training data cutoff, indicating data contamination inflates scores. — tap to centre the map on it Language models perform better onbenchmark problems released beforetheir training data cutoff,indicating data contaminationinflates scores. Last stated 3 years ago 13 May 2024 SR Sebastian Ruder — holds since 2024-05-13 — tap for who they are Same subject: Model comparisons understate real progress, because benchmark tables do not control for how much test-time compute each answer used. — tap to centre the map on it Model comparisons understate realprogress, because benchmark tablesdo not control for how muchtest-time compute each answer used. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as anonprofit and bolt a for-profit armon later, whatever OpenAI's ownhistory suggests. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phasefor the internet but a momentaryindustry, and an AI people pay foris better because they know theanswers are not influenced byadvertisers. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it A business making hundreds ofmillions from an open source projectowes it money or contribution, andcan be refused its trademarks if itgives neither. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it A codec does not really exist untileveryone can decode it. Last stated 6 months ago 31 May 2026 JK Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it A company that will not accept aproject’s terms is free to go anduse a more permissive projectinstead. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A CEO who sends out an AI-written strategy memo is modelling that it is fine to outsource thinking and strategy. — tap to centre the map on it A CEO who sends out an AI-writtenstrategy memo is modelling that itis fine to outsource thinking andstrategy. Last stated 2 months ago 27 Sept 2026 MG Molly Graham — holds since 2026-09-27 — tap for who they are Same subject: A deep research report should be treated as a first draft and a list of papers to check, never as definitely true. — tap to centre the map on it A deep research report should betreated as a first draft and a listof papers to check, never asdefinitely true. Last stated 2 years ago 27 Feb 2025 AK Andrej Karpathy — holds since 2025-02-27 — tap for who they are Same subject: A human-authorship certification should not allow partial AI use. — tap to centre the map on it A human-authorship certificationshould not allow partial AI use. Last stated 6 months ago 1 Jun 2026 NH Nick Harkaway — holds since 2026-06-01 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it. Last stated 19 May 2026 · 6 months ago Holds Tommy Geoco Read this korrent →