Tap a claim on the ring to put it at the centre.
← For a Qwen-sized vocabulary, a moderate entropy of around one to three…
17 connected korrents · 14 moments from 18 Mar 2024 to 26 Oct 2026.
Everything filed under LLMs
LLMs
Everything filed under open source
open source
Everything filed under OpenAI
OpenAI
Everything filed under scaling laws
scaling laws
Everything filed under AI alignment
AI alignment
Everything filed under design
design
Everything filed under mathematics
mathematics
Everything filed under neural networks
neural networks
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: For a Qwen-sized vocabulary, a moderate entropy of around one to three is probably good, while zero or the maximum of about 11 would be bad.
For a Qwen-sized vocabulary, a moderate entropy of around one to three is probably good, while zero or the maximum of about 11 would be bad.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: It would have been better if information content had been written as the log base one-half of a probability. — tap to centre the map on it
It would have been better if information content had been written as the log base one-half of a probability.
Last stated 4 months ago
16 Jul 2026
GS
Grant Sanderson — holds since 2026-07-16 — tap for who they are
Same subject: Zero token entropy during RL training is undesirable because the model is too confident and never explores other options. — tap to centre the map on it
Zero token entropy during RL training is undesirable because the model is too confident and never explores other options.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: Quoting a number for P(doom) is a ridiculous notion, because it implies a precision that nobody actually has. — tap to centre the map on it
Quoting a number for P(doom) is a ridiculous notion, because it implies a precision that nobody actually has.
Last stated a year ago
23 Jul 2025
DH
Demis Hassabis — holds since 2025-07-23 — tap for who they are
Same subject: The common explanation of cross-entropy loss as comparison with a one-hot distribution is lame because it does not explain why cross-entropy is used. — tap to centre the map on it
The common explanation of cross-entropy loss as comparison with a one-hot distribution is lame because it does not explain why cross-entropy is used.
Last stated 4 months ago
16 Jul 2026
GS
Grant Sanderson — holds since 2026-07-16 — tap for who they are
Same subject: Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each. — tap to centre the map on it
Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each.
Last stated 2 months ago
3 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-03 — tap for who they are
Same subject: The probability of some AI disaster is over 90%. — tap to centre the map on it
The probability of some AI disaster is over 90%.
Last stated 3 months ago
23 Aug 2026
TA
Timothy Garton Ash — holds since 2026-08-23 — tap for who they are
Same subject: Being liberal in what a parser accepts is not always good, but for cookie attributes like secure=1 it might be. — tap to centre the map on it
Being liberal in what a parser accepts is not always good, but for cookie attributes like secure=1 it might be.
Last stated a month ago
14 Oct 2026
DS
Daniel Stenberg — holds since 2026-10-14 — tap for who they are
Same subject: GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive. — tap to centre the map on it
GPT-3's learning-rate warm-up is very mild and could probably be made significantly more aggressive.
Last stated 2 years ago
9 Jun 2024
AK
Andrej Karpathy — holds since 2024-06-09 — tap for who they are
Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it
A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week.
Last stated a month ago
21 Oct 2026
PS
Peter Steinberger — holds since 2026-10-21 — tap for who they are
Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it
A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests.
Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it
Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers.
Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Last stated 12 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Last stated 5 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A CPU and memory crunch is coming. — tap to centre the map on it
A CPU and memory crunch is coming.
Last stated 4 weeks ago
26 Oct 2026
DM
Dillon Mulroy — holds since 2026-10-26 — tap for who they are
Same subject: A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither. — tap to centre the map on it
A business making hundreds of millions from an open source project owes it money or contribution, and can be refused its trademarks if it gives neither.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
Same subject: A codec does not really exist until everyone can decode it. — tap to centre the map on it
A codec does not really exist until everyone can decode it.
Last stated 6 months ago
31 May 2026
JK
Jean-Baptiste Kempf — holds since 2026-05-31 — tap for who they are
Same subject: A company that will not accept a project’s terms is free to go and use a more permissive project instead. — tap to centre the map on it
A company that will not accept a project’s terms is free to go and use a more permissive project instead.
Last stated 2 years ago
26 Sept 2024
MM
Matt Mullenweg — holds since 2024-09-26 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are