Tap a claim on the ring to put it at the centre.
← Using token-level or sequence-level log probabilities is almost a…
17 connected korrents · 15 moments from 1 Mar 1999 to 19 Oct 2026. Nearly all of them are about LLMs .
Everything filed under design
design
Everything filed under simplicity
simplicity
Everything filed under mathematics
mathematics
Everything filed under reinforcement learning
reinforcement learning
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each.
Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each.
Last stated 2 months ago
3 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-03 — tap for who they are
Same subject: At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. — tap to centre the map on it
At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat.
Last stated 4 months ago
16 Jul 2026
GS
Grant Sanderson — holds since 2026-07-16 — tap for who they are
Same subject: Humans are not well placed to write the token sequences a model should reason with; the model needs to discover what works for it. — tap to centre the map on it
Humans are not well placed to write the token sequences a model should reason with; the model needs to discover what works for it.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: Many people would like to get rid of tokens and use character- or byte-level models, but no one yet knows how to handle the very long sequences that would create. — tap to centre the map on it
Many people would like to get rid of tokens and use character- or byte-level models, but no one yet knows how to handle the very long sequences that would create.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: It would have been better if information content had been written as the log base one-half of a probability. — tap to centre the map on it
It would have been better if information content had been written as the log base one-half of a probability.
Last stated 4 months ago
16 Jul 2026
GS
Grant Sanderson — holds since 2026-07-16 — tap for who they are
Same subject: The more uncertain what follows a token, the more useful it is for a model to represent all relevant context on it. — tap to centre the map on it
The more uncertain what follows a token, the more useful it is for a model to represent all relevant context on it.
Last stated 11 months ago
20 Dec 2025
NN
Neel Nanda — holds since 2025-12-20 — tap for who they are
Same subject: Zero token entropy during RL training is undesirable because the model is too confident and never explores other options. — tap to centre the map on it
Zero token entropy during RL training is undesirable because the model is too confident and never explores other options.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: Tokens are not the true unit of AI inference, because better models yield more problem-solving per token. — tap to centre the map on it
Tokens are not the true unit of AI inference, because better models yield more problem-solving per token.
Last stated 2 months ago
15 Sept 2026
PG
Paul Graham — holds since 2026-09-15 — tap for who they are
Same subject: Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it. — tap to centre the map on it
Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it.
Last stated 6 months ago
19 May 2026
TG
Tommy Geoco — holds since 2026-05-19 — tap for who they are
Same subject: A $1,299, 100-gram pair of slip-on glasses seems far more enticing than a $3,699, 700-gram headset strapped to the face. — tap to centre the map on it
A $1,299, 100-gram pair of slip-on glasses seems far more enticing than a $3,699, 700-gram headset strapped to the face.
Last stated 2 months ago
24 Sept 2026
MS
M.G. Siegler — holds since 2026-09-24 — tap for who they are
Same subject: A baby-step version of eval-driven design is to sketch out 10 to 20 hero flows and run the feature or interaction through them to see where it falls apart. — tap to centre the map on it
A baby-step version of eval-driven design is to sketch out 10 to 20 hero flows and run the feature or interaction through them to see where it falls apart.
Last stated 2 months ago
9 Sept 2026
EB
Ed Bayes — holds since 2026-09-09 — tap for who they are
Same subject: A black background with green text is easy on the eyes for writing. — tap to centre the map on it
A black background with green text is easy on the eyes for writing.
Last stated 4 years ago
26 Aug 2022
CP
Christopher Paolini — holds since 2022-08-26 — tap for who they are
Same subject: A browser works better as a single window than as multiple separate windows. — tap to centre the map on it
A browser works better as a single window than as multiple separate windows.
Last stated 2 months ago
15 Sept 2026
CC
Chris Coyier — holds since 2026-09-15 — tap for who they are
Same subject: A design has to be kept simple for reliability and intellectual control, but nobody yet knows how to reach simplicity systematically. — tap to centre the map on it
A design has to be kept simple for reliability and intellectual control, but nobody yet knows how to reach simplicity systematically.
Last stated 28 years ago
1 Mar 1999
ED
Edsger W. Dijkstra — holds since 1999-03-01 — tap for who they are
Same subject: A designer's edge now is pulling things back, making them simpler and more understandable, because everyone can kind of execute right now. — tap to centre the map on it
A designer's edge now is pulling things back, making them simpler and more understandable, because everyone can kind of execute right now.
Last stated 4 months ago
29 Jul 2026
CD
Charlie Deets — holds since 2026-07-29 — tap for who they are
Same subject: A product's craft probably comes down to whether it works without pain and how it makes people feel. — tap to centre the map on it
A product's craft probably comes down to whether it works without pain and how it makes people feel.
Last stated 2 months ago
10 Sept 2026
RS
Robby Stein — holds since 2026-09-10 — tap for who they are
Same subject: A shadow in an interface implies a light source, so an elevated element needs a highlight where the light falls, not just a drop shadow. — tap to centre the map on it
A shadow in an interface implies a light source, so an elevated element needs a highlight where the light falls, not just a drop shadow.
Last stated 2 years ago
21 Feb 2025
TC
Thiago Costa — holds since 2025-02-21 — tap for who they are
Same subject: A subwoofer for music should reinforce the bass already in the recording, without adding bloat or artificial bass. — tap to centre the map on it
A subwoofer for music should reinforce the bass already in the recording, without adding bloat or artificial bass.
Last stated a month ago
19 Oct 2026
SH
Steve Huff — holds since 2026-10-19 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 1999 to today (stretched back to the oldest claim here) — full is today a face: someone who holds the claim — tap it for who they are
At the centre
Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each.
Last stated 3 Oct 2026 · 2 months ago
Holds Sebastian Raschka
Read this korrent →
Similar wording
At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat.
Last stated 16 Jul 2026 · 4 months ago
Holds Grant Sanderson
Similar wording
Humans are not well placed to write the token sequences a model should reason with; the model needs to discover what works for it.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Similar wording
Many people would like to get rid of tokens and use character- or byte-level models, but no one yet knows how to handle the very long sequences that would create.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Similar wording
It would have been better if information content had been written as the log base one-half of a probability.
Last stated 16 Jul 2026 · 4 months ago
Holds Grant Sanderson
Similar wording
The more uncertain what follows a token, the more useful it is for a model to represent all relevant context on it.
Last stated 20 Dec 2025 · 11 months ago
Holds Neel Nanda
Similar wording
Zero token entropy during RL training is undesirable because the model is too confident and never explores other options.
Last stated 10 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
Tokens are not the true unit of AI inference, because better models yield more problem-solving per token.
Last stated 15 Sept 2026 · 2 months ago
Holds Paul Graham
Similar wording
Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it.
Last stated 19 May 2026 · 6 months ago
Holds Tommy Geoco
Same subject: design
A $1,299, 100-gram pair of slip-on glasses seems far more enticing than a $3,699, 700-gram headset strapped to the face.
Last stated 24 Sept 2026 · 2 months ago
Holds M.G. Siegler
Same subject: design
A baby-step version of eval-driven design is to sketch out 10 to 20 hero flows and run the feature or interaction through them to see where it falls apart.
Last stated 9 Sept 2026 · 2 months ago
Holds Ed Bayes
Same subject: design
A black background with green text is easy on the eyes for writing.
Last stated 26 Aug 2022 · 4 years ago
Holds Christopher Paolini
Same subject: simplicity
A browser works better as a single window than as multiple separate windows.
Last stated 15 Sept 2026 · 2 months ago
Holds Chris Coyier
Same subject: simplicity
A design has to be kept simple for reliability and intellectual control, but nobody yet knows how to reach simplicity systematically.
Last stated 1 Mar 1999 · 28 years ago
Holds Edsger W. Dijkstra
Same subject: simplicity
A designer's edge now is pulling things back, making them simpler and more understandable, because everyone can kind of execute right now.
Last stated 29 Jul 2026 · 4 months ago
Holds Charlie Deets
Same subject: taste
A product's craft probably comes down to whether it works without pain and how it makes people feel.
Last stated 10 Sept 2026 · 2 months ago
Holds Robby Stein
Same subject: taste
A shadow in an interface implies a light source, so an elevated element needs a highlight where the light falls, not just a drop shadow.
Last stated 21 Feb 2025 · 2 years ago
Holds Thiago Costa
Same subject: taste
A subwoofer for music should reinforce the bass already in the recording, without adding bloat or artificial bass.
Last stated 19 Oct 2026 · a month ago
Holds Steve Huff