korrents

On the map

Tap a claim on the ring to put it at the centre.

← Using token-level or sequence-level log probabilities is almost a…

17 connected korrents · 15 moments from 1 Mar 1999 to 19 Oct 2026. Nearly all of them are about LLMs.

Everything filed under design design Everything filed under simplicity simplicity Everything filed under mathematics mathematics Everything filed under reinforcement learning reinforcement learning Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each. Using token-level or sequence-level logprobabilities is almost a hyperparameterchoice, with good reasons for each. Last stated 2 months ago 3 Oct 2026 SR Sebastian Raschka — holds since 2026-10-03 — tap for who they are Same subject: At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. — tap to centre the map on it At the level of loss functions,language-model pre-training isremarkably simple, though running itat scale is a complete engineeringfeat. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: Humans are not well placed to write the token sequences a model should reason with; the model needs to discover what works for it. — tap to centre the map on it Humans are not well placed to writethe token sequences a model shouldreason with; the model needs todiscover what works for it. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: Many people would like to get rid of tokens and use character- or byte-level models, but no one yet knows how to handle the very long sequences that would create. — tap to centre the map on it Many people would like to get rid oftokens and use character- orbyte-level models, but no one yetknows how to handle the very longsequences that would create. Last stated 2 years ago 5 Feb 2025 AK Andrej Karpathy — holds since 2025-02-05 — tap for who they are Same subject: It would have been better if information content had been written as the log base one-half of a probability. — tap to centre the map on it It would have been better ifinformation content had been writtenas the log base one-half of aprobability. Last stated 4 months ago 16 Jul 2026 GS Grant Sanderson — holds since 2026-07-16 — tap for who they are Same subject: The more uncertain what follows a token, the more useful it is for a model to represent all relevant context on it. — tap to centre the map on it The more uncertain what follows atoken, the more useful it is for amodel to represent all relevantcontext on it. Last stated 11 months ago 20 Dec 2025 NN Neel Nanda — holds since 2025-12-20 — tap for who they are Same subject: Zero token entropy during RL training is undesirable because the model is too confident and never explores other options. — tap to centre the map on it Zero token entropy during RLtraining is undesirable because themodel is too confident and neverexplores other options. Last stated a month ago 10 Oct 2026 SR Sebastian Raschka — holds since 2026-10-10 — tap for who they are Same subject: Tokens are not the true unit of AI inference, because better models yield more problem-solving per token. — tap to centre the map on it Tokens are not the true unit of AIinference, because better modelsyield more problem-solving pertoken. Last stated 2 months ago 15 Sept 2026 PG Paul Graham — holds since 2026-09-15 — tap for who they are Same subject: Judging one model worse than another from a one-shot, probabilistic output on a given day is not a good way to evaluate it. — tap to centre the map on it Judging one model worse than anotherfrom a one-shot, probabilisticoutput on a given day is not a goodway to evaluate it. Last stated 6 months ago 19 May 2026 TG Tommy Geoco — holds since 2026-05-19 — tap for who they are Same subject: A $1,299, 100-gram pair of slip-on glasses seems far more enticing than a $3,699, 700-gram headset strapped to the face. — tap to centre the map on it A $1,299, 100-gram pair of slip-onglasses seems far more enticing thana $3,699, 700-gram headset strappedto the face. Last stated 2 months ago 24 Sept 2026 MS M.G. Siegler — holds since 2026-09-24 — tap for who they are Same subject: A baby-step version of eval-driven design is to sketch out 10 to 20 hero flows and run the feature or interaction through them to see where it falls apart. — tap to centre the map on it A baby-step version of eval-drivendesign is to sketch out 10 to 20hero flows and run the feature orinteraction through them to seewhere it falls apart. Last stated 2 months ago 9 Sept 2026 EB Ed Bayes — holds since 2026-09-09 — tap for who they are Same subject: A black background with green text is easy on the eyes for writing. — tap to centre the map on it A black background with green textis easy on the eyes for writing. Last stated 4 years ago 26 Aug 2022 CP Christopher Paolini — holds since 2022-08-26 — tap for who they are Same subject: A browser works better as a single window than as multiple separate windows. — tap to centre the map on it A browser works better as a singlewindow than as multiple separatewindows. Last stated 2 months ago 15 Sept 2026 CC Chris Coyier — holds since 2026-09-15 — tap for who they are Same subject: A design has to be kept simple for reliability and intellectual control, but nobody yet knows how to reach simplicity systematically. — tap to centre the map on it A design has to be kept simple forreliability and intellectualcontrol, but nobody yet knows how toreach simplicity systematically. Last stated 28 years ago 1 Mar 1999 ED Edsger W. Dijkstra — holds since 1999-03-01 — tap for who they are Same subject: A designer's edge now is pulling things back, making them simpler and more understandable, because everyone can kind of execute right now. — tap to centre the map on it A designer's edge now is pullingthings back, making them simpler andmore understandable, becauseeveryone can kind of execute rightnow. Last stated 4 months ago 29 Jul 2026 CD Charlie Deets — holds since 2026-07-29 — tap for who they are Same subject: A product's craft probably comes down to whether it works without pain and how it makes people feel. — tap to centre the map on it A product's craft probably comesdown to whether it works withoutpain and how it makes people feel. Last stated 2 months ago 10 Sept 2026 RS Robby Stein — holds since 2026-09-10 — tap for who they are Same subject: A shadow in an interface implies a light source, so an elevated element needs a highlight where the light falls, not just a drop shadow. — tap to centre the map on it A shadow in an interface implies alight source, so an elevated elementneeds a highlight where the lightfalls, not just a drop shadow. Last stated 2 years ago 21 Feb 2025 TC Thiago Costa — holds since 2025-02-21 — tap for who they are Same subject: A subwoofer for music should reinforce the bass already in the recording, without adding bloat or artificial bass. — tap to centre the map on it A subwoofer for music shouldreinforce the bass already in therecording, without adding bloat orartificial bass. Last stated a month ago 19 Oct 2026 SH Steve Huff — holds since 2026-10-19 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 1999 to today (stretched back to the oldest claim here) — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Using token-level or sequence-level log probabilities is almost a hyperparameter choice, with good reasons for each. Last stated 3 Oct 2026 · 2 months ago Holds Sebastian Raschka Read this korrent →