korrents

On the map

Tap a claim on the ring to put it at the centre.

← When a KL loss term is not needed, it is just overhead, since it…

17 connected korrents · 14 moments from 8 Aug 2023 to 5 Nov 2026.

Everything filed under software performance software performance Everything filed under LLMs LLMs Everything filed under AI alignment AI alignment Everything filed under reinforcement learning reinforcement learning Everything filed under developer tools developer tools Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: When a KL loss term is not needed, it is just overhead, since it requires keeping an extra reference model in memory. When a KL loss term is not needed, it isjust overhead, since it requires keepingan extra reference model in memory. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: A KL loss term is very important in domains with a lot of reward hacking but usually not necessary for math models. — tap to centre the map on it A KL loss term is very important indomains with a lot of reward hackingbut usually not necessary for mathmodels. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: The KL term in GRPO is not essential; whether to use it is a hyperparameter to decide case by case, and on math models may do better without it. — tap to centre the map on it The KL term in GRPO is notessential; whether to use it is ahyperparameter to decide case bycase, and on math models may dobetter without it. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. — tap to centre the map on it At the level of loss functions,language-model pre-training isremarkably simple, though running itat scale is a complete engineeringfeat. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often. — tap to centre the map on it Language models should be thought ofas somewhat out of date, becausepre-training is too expensive torepeat often. Last stated 2 years ago 27 Feb 2025 AK Andrej Karpathy — holds since 2025-02-27 — tap for who they are Same subject: The argument that RAG and long context make fine-tuning unnecessary holds when information is missing, but not when the model struggles with the task itself. — tap to centre the map on it The argument that RAG and longcontext make fine-tuning unnecessaryholds when information is missing,but not when the model struggleswith the task itself. Last stated 2 weeks ago 5 Nov 2026 AR Aishwarya Naresh Reganti — holds since 2026-11-05 — tap for who they are Same subject: The knowledge a model soaks up in pre-training is holding it back; what we actually want is the intelligence with the knowledge stripped out. — tap to centre the map on it The knowledge a model soaks up inpre-training is holding it back;what we actually want is theintelligence with the knowledgestripped out. Last stated a year ago 17 Oct 2025 AK Andrej Karpathy — holds since 2025-10-17 — tap for who they are Same subject: Given good data, room to operate and sound numerical conditioning, models will learn on their own. — tap to centre the map on it Given good data, room to operate andsound numerical conditioning, modelswill learn on their own. Last stated 3 years ago 8 Aug 2023 DA Dario Amodei — holds since 2023-08-08 — tap for who they are Same subject: In reinforcement learning for LLMs, the loss is not as informative as in classic deep learning or pre-training. — tap to centre the map on it In reinforcement learning for LLMs,the loss is not as informative as inclassic deep learning orpre-training. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: "Is the page loaded?" was never a real question, and web performance only became measurable once it was broken into the distinct moments a user actually notices. — tap to centre the map on it "Is the page loaded?" was never areal question, and web performanceonly became measurable once it wasbroken into the distinct moments auser actually notices. Last stated 3 months ago 19 Aug 2026 AO Addy Osmani — holds since 2026-08-19 — tap for who they are Same subject: "Optimize it later" would be safe advice only if every engineer already knew how to avoid the architectural mistakes that no later optimization can remove. — tap to centre the map on it "Optimize it later" would be safeadvice only if every engineeralready knew how to avoid thearchitectural mistakes that no lateroptimization can remove. Last stated 3 months ago 26 Aug 2026 CM Casey Muratori — holds since 2026-08-26 — tap for who they are Same subject: A 300-millisecond response budget is not an achievement but an admission: 300 milliseconds is an eternity in computing, and a product pitching it shows how far the bar has fallen. — tap to centre the map on it A 300-millisecond response budget isnot an achievement but an admission:300 milliseconds is an eternity incomputing, and a product pitching itshows how far the bar has fallen. Last stated 3 months ago 26 Aug 2026 CM Casey Muratori — holds since 2026-08-26 — tap for who they are Same subject: A profiler measures time already spent in an operation, not how much time would be saved by optimizing it. — tap to centre the map on it A profiler measures time alreadyspent in an operation, not how muchtime would be saved by optimizingit. Last stated 3 years ago 18 Dec 2023 NE Nelson Elhage — holds since 2023-12-18 — tap for who they are Same subject: A tool's speed comes primarily from its design decisions rather than the programming language it is written in. — tap to centre the map on it A tool's speed comes primarily fromits design decisions rather than theprogramming language it is writtenin. Last stated 11 months ago 29 Dec 2025 AP Aaron Patterson — holds since 2025-12-29 — tap for who they are Same subject: For GPU performance, flops do not matter so much as the whole memory access pattern does, and torch.compile cannot find every optimization. — tap to centre the map on it For GPU performance, flops do notmatter so much as the whole memoryaccess pattern does, andtorch.compile cannot find everyoptimization. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: Compiler mechanisms that force a required optimization to occur, rather than relying on incidental compiler behavior, are a more robust way to write performance-sensitive code. — tap to centre the map on it Compiler mechanisms that force arequired optimization to occur,rather than relying on incidentalcompiler behavior, are a more robustway to write performance-sensitivecode. Last stated 2 years ago 9 Mar 2025 NE Nelson Elhage — holds since 2025-03-09 — tap for who they are Same subject: What clean-code polymorphism really costs you is not the virtual call but the optimizations the compiler can no longer perform once it cannot see what runs. — tap to centre the map on it What clean-code polymorphism reallycosts you is not the virtual callbut the optimizations the compilercan no longer perform once it cannotsee what runs. Last stated 3 months ago 26 Aug 2026 CM Casey Muratori — holds since 2026-08-26 — tap for who they are Same subject: A compiler that also serves an IDE has to be lazy and deferred throughout — a different way of building compilers from the one the textbooks teach. — tap to centre the map on it A compiler that also serves an IDEhas to be lazy and deferredthroughout — a different way ofbuilding compilers from the one thetextbooks teach. Last stated 6 months ago 13 May 2026 AH Anders Hejlsberg — holds since 2026-05-13 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre When a KL loss term is not needed, it is just overhead, since it requires keeping an extra reference model in memory. Last stated 10 Oct 2026 · a month ago Holds Sebastian Raschka Read this korrent →