korrents

On the map

Tap a claim on the ring to put it at the centre.

← GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive.

17 connected korrents · 18 moments from 12 Sept 2022 to 2 Nov 2026.

Everything filed under LLMs LLMs Everything filed under Anthropic Anthropic Everything filed under scaling laws scaling laws Everything filed under open source open source Everything filed under neural networks neural networks Everything filed under AI alignment AI alignment Everything filed under AI and human skill AI and human skill Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive. GPT-3's learning-rate warm-up is very mildand could probably be made significantlymore aggressive. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: Training in fp16 required gradient scaling, which is annoying and an extra source of state and complexity. — tap to centre the map on it Training in fp16 required gradientscaling, which is annoying and anextra source of state andcomplexity. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it. — tap to centre the map on it Judging one model worse than anotherfrom a one-shot, probabilisticoutput on a given day is not a goodway to evaluate it. Last stated 6 months ago 19 May 2026 TG Tommy Geoco — holds since 2026-05-19 — tap for who they are Same subject: There will probably be a few steps between GPT-4 and AI that could pose a takeover risk, in a ramp-up that is terrifyingly fast but fairly continuous. — tap to centre the map on it There will probably be a few stepsbetween GPT-4 and AI that could posea takeover risk, in a ramp-up thatis terrifyingly fast but fairlycontinuous. Last stated 4 years ago 12 May 2023 AC Ajeya Cotra — holds since 2023-05-12 — tap for who they are Same subject: AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard. — tap to centre the map on it AI progress is faster than peopleexpect, and ordinary scaling can beenough to solve problems that lookedvery hard. Last stated 2 years ago 2 Feb 2025 AC Ajeya Cotra — holds since 2023-08-29 — tap for who they are SA Scott Alexander — holds since 2022-09-12 — tap for who they are MB Miles Brundage — holds since 2025-02-02 — tap for who they are Same subject: Math, code and multilingual data in GPT-2's training set may have taken capacity away from English performance. — tap to centre the map on it Math, code and multilingual data inGPT-2's training set may have takencapacity away from Englishperformance. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: The gap between GPT-5 and GPT-4 will be as large as the gap between GPT-4 and GPT-3, and the job is to live a few years ahead and remember that today's tools will look terrible from there. — tap to centre the map on it The gap between GPT-5 and GPT-4 willbe as large as the gap between GPT-4and GPT-3, and the job is to live afew years ahead and remember thattoday's tools will look terriblefrom there. Last stated 2 months ago 1 Oct 2026 SA Sam Altman — holds since 2024-03-18 — tap for who they are ZM Zvi Mowshowitz — holds since 2026-10-01 — tap for who they are Same subject: AI will not make you world-class at a skill, but it takes a near-beginner to roughly the sixtieth or seventieth percentile of it today. — tap to centre the map on it AI will not make you world-class ata skill, but it takes anear-beginner to roughly thesixtieth or seventieth percentile ofit today. Last stated a year ago 21 Sept 2025 JZ Julie Zhuo — holds since 2025-09-21 — tap for who they are Same subject: OpenAI will likely achieve a sense of fun in its models comparable to Claude Opus 5.5 by the end of the year. — tap to centre the map on it OpenAI will likely achieve a senseof fun in its models comparable toClaude Opus 5.5 by the end of theyear. Last stated 4 weeks ago 28 Oct 2026 SW Simon Willison — holds since 2026-10-28 — tap for who they are Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because itwas one word: naming a researchdirection is what tells a wholefield what to do next. Last stated 12 months ago 25 Nov 2025 IS Ilya Sutskever — holds since 2025-11-25 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A CPU and memory crunch is coming. — tap to centre the map on it A CPU and memory crunch is coming. Last stated 4 weeks ago 26 Oct 2026 DM Dillon Mulroy — holds since 2026-10-26 — tap for who they are Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it A business making hundreds ofmillions from an open source projectowes it money or contribution, andcan be refused its trademarks if itgives neither. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it A codec does not really exist untileveryone can decode it. Last stated 6 months ago 31 May 2026 JK Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it A company that will not accept aproject’s terms is free to go anduse a more permissive projectinstead. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: "I don't know" becomes a kind of superpower when you can work with Claude Code to figure things out. — tap to centre the map on it "I don't know" becomes a kind ofsuperpower when you can work withClaude Code to figure things out. Last stated 8 months ago 19 Mar 2026 KP Kris Puckett — holds since 2026-03-19 — tap for who they are Same subject: A benchmark that ranks Claude Code last while it stays first in use is measuring the wrong thing, and has been for a year. — tap to centre the map on it A benchmark that ranks Claude Codelast while it stays first in use ismeasuring the wrong thing, and hasbeen for a year. Last stated 3 months ago 3 Sept 2026 DR Dax Raad — holds since 2026-09-03 — tap for who they are Same subject: A large share of Anthropic's revenue being consumption-based rather than subscriptions raises the question of how resilient that revenue is. — tap to centre the map on it A large share of Anthropic's revenuebeing consumption-based rather thansubscriptions raises the question ofhow resilient that revenue is. Last stated 3 weeks ago 2 Nov 2026 AA Azeem Azhar — holds since 2026-11-02 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive. Last stated 9 Jun 2024 · 2 years ago Holds Andrej Karpathy Read this korrent →