korrents

On the map

Tap a claim on the ring to put it at the centre.

← For a Qwen-sized vocabulary, a moderate entropy of around one to three…

17 connected korrents · 14 moments from 18 Mar 2024 to 26 Oct 2026.

Everything filed under LLMs LLMs Everything filed under open source open source Everything filed under OpenAI OpenAI Everything filed under scaling laws scaling laws Everything filed under AI alignment AI alignment Everything filed under design design Everything filed under mathematics mathematics Everything filed under neural networks neural networks Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: For a Qwen-sized vocabulary, a moderate entropy of around one to three is probably good, while zero or the maximum of about 11 would be bad. For a Qwen-sized vocabulary, a moderateentropy of around one to three is probablygood, while zero or the maximum of about11 would be bad. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: It would have been better if information content had been written as the log base one-half of a probability. — tap to centre the map on it It would have been better ifinformation content had been writtenas the log base one-half of aprobability. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: Zero token entropy during RL training is undesirable because the model is too confident and never explores other options. — tap to centre the map on it Zero token entropy during RLtraining is undesirable because themodel is too confident and neverexplores other options. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: Quoting a number for P(doom) is a ridiculous notion, because it implies a precision that nobody actually has. — tap to centre the map on it Quoting a number for P(doom) is aridiculous notion, because itimplies a precision that nobodyactually has. Last stated a year ago 23 Jul 2025 DH Demis Hassabis — holds since 2025-07-23 — tap for who they are Same subject: The common explanation of cross-entropy loss as comparison with a one-hot distribution is lame because it does not explain why cross-entropy is used. — tap to centre the map on it The common explanation ofcross-entropy loss as comparisonwith a one-hot distribution is lamebecause it does not explain whycross-entropy is used. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each. — tap to centre the map on it Using token-level or sequence-levellog probabilities is almost ahyperparameter choice, with goodreasons for each. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: The probability of some AI disaster is over 90%. — tap to centre the map on it The probability of some AI disasteris over 90%. Last stated 3 months ago 23 Aug 2026 TA Timothy Garton Ash — holds since 2026-08-23 — tap for who they are Same subject: Being liberal in what a parser accepts is not always good, but for cookie attributes like secure=1 it might be. — tap to centre the map on it Being liberal in what a parseraccepts is not always good, but forcookie attributes like secure=1 itmight be. Last stated a month ago 14 Oct 2026 DS Daniel Stenberg — holds since 2026-10-14 — tap for who they are Same subject: GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive. — tap to centre the map on it GPT-3's learning-rate warm-up isvery mild and could probably be madesignificantly more aggressive. Last stated 2 years ago 9 Jun 2024 AK Andrej Karpathy — holds since 2024-06-09 — tap for who they are Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found withGPT-5.6 Sol will cause havoc thisweek. Last stated a month ago 21 Oct 2026 PS Peter Steinberger — holds since 2026-10-21 — tap for who they are Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as anonprofit and bolt a for-profit armon later, whatever OpenAI's ownhistory suggests. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phasefor the internet but a momentaryindustry, and an AI people pay foris better because they know theanswers are not influenced byadvertisers. Last stated 3 years ago 18 Mar 2024 SA Sam Altman — holds since 2024-03-18 — tap for who they are Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because itwas one word: naming a researchdirection is what tells a wholefield what to do next. Last stated 12 months ago 25 Nov 2025 IS Ilya Sutskever — holds since 2025-11-25 — tap for who they are Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should bereported under a stated budget, oras a curve against test-time compute— never as a single number. Last stated 5 months ago 26 Jun 2026 NB Noam Brown — holds since 2026-06-26 — tap for who they are Same subject: A CPU and memory crunch is coming. — tap to centre the map on it A CPU and memory crunch is coming. Last stated 4 weeks ago 26 Oct 2026 DM Dillon Mulroy — holds since 2026-10-26 — tap for who they are Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it A business making hundreds ofmillions from an open source projectowes it money or contribution, andcan be refused its trademarks if itgives neither. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it A codec does not really exist untileveryone can decode it. Last stated 6 months ago 31 May 2026 JK Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it A company that will not accept aproject’s terms is free to go anduse a more permissive projectinstead. Last stated 2 years ago 26 Sept 2024 MM Matt Mullenweg — holds since 2024-09-26 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre For a Qwen-sized vocabulary, a moderate entropy of around one to three is probably good, while zero or the maximum of about 11 would be bad. Last stated 10 Oct 2026 · a month ago Holds Sebastian Raschka Read this korrent →