Language models perform better on benchmark problems released before their training data cutoff, indicating data contamination inflates scores.

Added to korrents 20 Sept 2026 · How quotes work · Something wrong? Tell us

Shared subject first, then similar wording. They may agree or disagree.

• A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. NBNoam Brown26 Jun 2026benchmarksscaling laws
• Model comparisons understate real progress, because benchmark tables do not control for how much test-time compute each answer used. NBNoam Brown26 Jun 2026benchmarksscaling laws
• Current techniques restrict pre-trained representation power because standard language models are unidirectional. JDJacob Devlin11 Oct 2018LLMsscaling laws
• Large language models could still plateau, and that possibility should be held open even though no evidence of it has appeared. DHDavid Heinemeier Hansson26 Aug 2026LLMsscaling laws
• Reasoning models are an upgrade from conventional LLMs achieved by adding post-training via reinforcement learning and optionally inference scaling. SRSebastian Raschka24 Oct 2026LLMsscaling laws

See this claim on the map