Tap a claim on the ring to put it at the centre.
← A KL loss term is very important in domains with a lot of reward…
17 connected korrents · 16 moments from 26 Jan 2025 to 26 Oct 2026.
Everything filed under AI alignment
AI alignment
Everything filed under LLMs
LLMs
Everything filed under software performance
software performance
Everything filed under reinforcement learning
reinforcement learning
Everything filed under mathematics
mathematics
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: A KL loss term is very important in domains with a lot of reward hacking but usually not necessary for math models.
A KL loss term is very important in domains with a lot of reward hacking but usually not necessary for math models.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: When a KL loss term is not needed, it is just overhead, since it requires keeping an extra reference model in memory. — tap to centre the map on it
When a KL loss term is not needed, it is just overhead, since it requires keeping an extra reference model in memory.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: The KL term in GRPO is not essential; whether to use it is a hyperparameter to decide case by case, and on math models may do better without it. — tap to centre the map on it
The KL term in GRPO is not essential; whether to use it is a hyperparameter to decide case by case, and on math models may do better without it.
Last stated 2 months ago
3 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-03 — tap for who they are
Same subject: AI Math Olympiad results may partly reflect teaching to the test. — tap to centre the map on it
AI Math Olympiad results may partly reflect teaching to the test.
Last stated a year ago
13 Aug 2025
NS
Nate Silver — holds since 2025-08-13 — tap for who they are
Same subject: In reinforcement learning for LLMs, the loss is not as informative as in classic deep learning or pre-training. — tap to centre the map on it
In reinforcement learning for LLMs, the loss is not as informative as in classic deep learning or pre-training.
Last stated a month ago
10 Oct 2026
SR
Sebastian Raschka — holds since 2026-10-10 — tap for who they are
Same subject: Machine learning is a shallow field compared with mathematics: even its most important ideas can be explained in a couple of minutes. — tap to centre the map on it
Machine learning is a shallow field compared with mathematics: even its most important ideas can be explained in a couple of minutes.
Last stated 3 months ago
11 Aug 2026
RG
Ryan Greenblatt — holds since 2026-08-11 — tap for who they are
Same subject: At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat. — tap to centre the map on it
At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat.
Last stated 4 months ago
16 Jul 2026
GS
Grant Sanderson — holds since 2026-07-16 — tap for who they are
Same subject: An answer computed by a Python program deserves more trust than a language model's mental arithmetic. — tap to centre the map on it
An answer computed by a Python program deserves more trust than a language model's mental arithmetic.
Last stated 2 years ago
5 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-05 — tap for who they are
Same subject: Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often. — tap to centre the map on it
Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often.
Last stated 2 years ago
27 Feb 2025
AK
Andrej Karpathy — holds since 2025-02-27 — tap for who they are
Same subject: "Hire smart people and get out of their way" is bad management advice; leaders should stay closely involved, building trust and working through hard problems with their people to create alignment. — tap to centre the map on it
"Hire smart people and get out of their way" is bad management advice; leaders should stay closely involved, building trust and working through hard problems with their people to create alignment.
Last stated a month ago
9 Oct 2026
CR
Carl Rivera — holds since 2026-10-09 — tap for who they are
Same subject: A 5% level of AI risk is crazy and unacceptable; we would not accept a 5% chance of massive fatalities from paint companies. — tap to centre the map on it
A 5% level of AI risk is crazy and unacceptable; we would not accept a 5% chance of massive fatalities from paint companies.
Last stated a year ago
8 Jul 2025
RG
Ryan Greenblatt — holds since 2025-07-08 — tap for who they are
Same subject: A big misreading of Chinese labs is missing that they have a risk posture and do not want to cause harm. — tap to centre the map on it
A big misreading of Chinese labs is missing that they have a risk posture and do not want to cause harm.
Last stated 4 weeks ago
26 Oct 2026
NL
Nathan Lambert — holds since 2026-10-26 — tap for who they are
Same subject: A human being is not an AGI: we lack a huge amount of knowledge and rely on continual learning instead, so continual learning is what superintelligence should mean. — tap to centre the map on it
A human being is not an AGI: we lack a huge amount of knowledge and rely on continual learning instead, so continual learning is what superintelligence should mean.
Last stated 12 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A proposal to pace frontier AI development is unrealistic and mainly serves political control of AI. — tap to centre the map on it
A proposal to pace frontier AI development is unrealistic and mainly serves political control of AI.
Last stated 2 months ago
18 Sept 2026
BT
Ben Thompson — holds since 2026-09-18 — tap for who they are
Same subject: Abstract talk of AGI and superintelligence is too abstract; AI has to actually work. — tap to centre the map on it
Abstract talk of AGI and superintelligence is too abstract; AI has to actually work.
Last stated 2 years ago
26 Jan 2025
MA
Marc Andreessen — holds since 2025-01-26 — tap for who they are
Same subject: A real friend will eventually refuse to help, and later turn on you, if you are sufficiently evil or selfish. — tap to centre the map on it
A real friend will eventually refuse to help, and later turn on you, if you are sufficiently evil or selfish.
Last stated 2 months ago
20 Sept 2026
ZM
Zvi Mowshowitz — holds since 2026-09-20 — tap for who they are
Same subject: Agents become far more powerful only with a trust model that makes autonomous action safe. — tap to centre the map on it
Agents become far more powerful only with a trust model that makes autonomous action safe.
Last stated a month ago
18 Oct 2026
AK
Alex Komoroske — holds since 2026-10-18 — tap for who they are
Same subject: Agents should reach integrations only through locked packages of deterministic code that can do only what you want them to do. — tap to centre the map on it
Agents should reach integrations only through locked packages of deterministic code that can do only what you want them to do.
Last stated 2 months ago
15 Sept 2026
KD
Kent C. Dodds — holds since 2026-09-15 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are
At the centre
A KL loss term is very important in domains with a lot of reward hacking but usually not necessary for math models.
Last stated 10 Oct 2026 · a month ago
Holds Sebastian Raschka
Read this korrent →
Similar wording
When a KL loss term is not needed, it is just overhead, since it requires keeping an extra reference model in memory.
Last stated 10 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
The KL term in GRPO is not essential; whether to use it is a hyperparameter to decide case by case, and on math models may do better without it.
Last stated 3 Oct 2026 · 2 months ago
Holds Sebastian Raschka
Similar wording
AI Math Olympiad results may partly reflect teaching to the test.
Last stated 13 Aug 2025 · a year ago
Holds Nate Silver
Similar wording
In reinforcement learning for LLMs, the loss is not as informative as in classic deep learning or pre-training.
Last stated 10 Oct 2026 · a month ago
Holds Sebastian Raschka
Similar wording
Machine learning is a shallow field compared with mathematics: even its most important ideas can be explained in a couple of minutes.
Last stated 11 Aug 2026 · 3 months ago
Holds Ryan Greenblatt
Similar wording
At the level of loss functions, language-model pre-training is remarkably simple, though running it at scale is a complete engineering feat.
Last stated 16 Jul 2026 · 4 months ago
Holds Grant Sanderson
Similar wording
An answer computed by a Python program deserves more trust than a language model's mental arithmetic.
Last stated 5 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Similar wording
Language models should be thought of as somewhat out of date, because pre-training is too expensive to repeat often.
Last stated 27 Feb 2025 · 2 years ago
Holds Andrej Karpathy
Same subject: AI alignment
"Hire smart people and get out of their way" is bad management advice; leaders should stay closely involved, building trust and working through hard problems with their people to create alignment.
Last stated 9 Oct 2026 · a month ago
Holds Carl Rivera
Same subject: AI alignment
A 5% level of AI risk is crazy and unacceptable; we would not accept a 5% chance of massive fatalities from paint companies.
Last stated 8 Jul 2025 · a year ago
Holds Ryan Greenblatt
Same subject: AI alignment
A big misreading of Chinese labs is missing that they have a risk posture and do not want to cause harm.
Last stated 26 Oct 2026 · 4 weeks ago
Holds Nathan Lambert
Same subject: AGI
A human being is not an AGI: we lack a huge amount of knowledge and rely on continual learning instead, so continual learning is what superintelligence should mean.
Last stated 25 Nov 2025 · 12 months ago
Holds Ilya Sutskever
Same subject: AGI
A proposal to pace frontier AI development is unrealistic and mainly serves political control of AI.
Last stated 18 Sept 2026 · 2 months ago
Holds Ben Thompson
Same subject: AGI
Abstract talk of AGI and superintelligence is too abstract; AI has to actually work.
Last stated 26 Jan 2025 · 2 years ago
Holds Marc Andreessen
Same subject: AI agents
A real friend will eventually refuse to help, and later turn on you, if you are sufficiently evil or selfish.
Last stated 20 Sept 2026 · 2 months ago
Holds Zvi Mowshowitz
Same subject: AI agents
Agents become far more powerful only with a trust model that makes autonomous action safe.
Last stated 18 Oct 2026 · a month ago
Holds Alex Komoroske
Same subject: AI agents
Agents should reach integrations only through locked packages of deterministic code that can do only what you want them to do.
Last stated 15 Sept 2026 · 2 months ago
Holds Kent C. Dodds