Tap a claim on the ring to put it at the centre.
← Unidirectional restrictions are sub-optimal for sentence-level tasks…
17 connected korrents · 11 moments on record from 11 Oct 2018 to 3 Aug 2026.
Everything filed under LLMs
LLMs
Everything filed under scaling laws
scaling laws
Everything filed under measuring intelligence
measuring intelligence
Everything filed under reinforcement learning
reinforcement learning
Everything filed under AI alignment
AI alignment
Everything filed under AI agents
AI agents Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context. Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: Current techniques restrict pre-trained representation power because standard language models are unidirectional. — tap to centre the map on it Current techniques restrict pre-trained representation power because standard language models are unidirectional. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: The next-token language modeling objective is misaligned with following user instructions helpfully and safely. — tap to centre the map on it The next-token language modeling objective is misaligned with following user instructions helpfully and safely. Last stated 5 years ago
4 Mar 2022
JL
Jan Leike — holds since 2022-03-04 — tap for who they are
Same subject: Reasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own. — tap to centre the map on it Reasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own. Last stated 2 years ago
3 Feb 2025
NL
Nathan Lambert — holds since 2025-02-03 — tap for who they are
Same subject: A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. — tap to centre the map on it A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: The scarce skill is now context switching across parallel agents, not sustained deep work. — tap to centre the map on it The scarce skill is now context switching across parallel agents, not sustained deep work. Last stated 6 months ago
4 Mar 2026
BC
Boris Cherny — holds since 2026-03-04 — tap for who they are
Same subject: Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. — tap to centre the map on it Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. Last stated 6 years ago
28 May 2020
DA
Dario Amodei — holds since 2020-05-28 — tap for who they are
Same subject: A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. — tap to centre the map on it A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to. Last stated 2 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore. — tap to centre the map on it Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore. Last stated 2 months ago
15 Jul 2026
DH
Dex Horthy — holds since 2026-07-15 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 2 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 6 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
Same subject: A model's capability is now a function of how much money you spend on it, so asking what a model can do means nothing until you name a budget. — tap to centre the map on it A model's capability is now a function of how much money you spend on it, so asking what a model can do means nothing until you name a budget. Last stated 2 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: If intelligence is the process of acquiring skills, no single task demonstrates intelligence unless it is a meta-task of skill-acquisition across many tasks. — tap to centre the map on it If intelligence is the process of acquiring skills, no single task demonstrates intelligence unless it is a meta-task of skill-acquisition across many tasks. Last stated 7 years ago
5 Nov 2019
FC
François Chollet — holds since 2019-11-05 — tap for who they are
Same subject: Solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience. — tap to centre the map on it Solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience. Last stated 7 years ago
5 Nov 2019
FC
François Chollet — holds since 2019-11-05 — tap for who they are
Same subject: The Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans. — tap to centre the map on it The Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans. Last stated 7 years ago
5 Nov 2019
FC
François Chollet — holds since 2019-11-05 — tap for who they are
Same subject: A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel. — tap to centre the map on it A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel. Last stated a month ago
3 Aug 2026
DD
Dmitri Dolgov — holds since 2026-08-03 — tap for who they are
Same subject: A verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well. — tap to centre the map on it A verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well. Last stated 10 months ago
17 Nov 2025
AK
Andrej Karpathy — holds since 2025-11-17 — tap for who they are
Same subject: Building a realistic simulator is exactly as hard as building the agent, because the simulator is itself a large AI model. — tap to centre the map on it Building a realistic simulator is exactly as hard as building the agent, because the simulator is itself a large AI model. Last stated a month ago
3 Aug 2026
DD
Dmitri Dolgov — holds since 2026-08-03 — tap for who they are
At the centre
Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context.
Holds KT Kristina ToutanovaMC Ming-Wei ChangKL Kenton LeeJD Jacob DevlinRead this korrent →
Current techniques restrict pre-trained representation power because standard language models are unidirectional.
Holds KT Kristina ToutanovaMC Ming-Wei ChangKL Kenton LeeJD Jacob Devlin
The next-token language modeling objective is misaligned with following user instructions helpfully and safely.
Holds JL Jan Leike
Reasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own.
Holds NL Nathan Lambert
A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models.
Holds KT Kristina ToutanovaMC Ming-Wei ChangKL Kenton LeeJD Jacob Devlin
The scarce skill is now context switching across parallel agents, not sustained deep work.
Holds
Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches.
Holds DA Dario Amodei
A bigger context window does not give you a smarter model; the intelligence of the model is what decides how much of that window it can actually attend to.
Holds DH Dex Horthy
Context has two budgets, not one: the information budget everyone thinks about, and an instruction budget, where conflicting instructions cost the model real computation to ignore.
Holds DH Dex Horthy
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds NB Noam Brown
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds DP Dylan Patel
A model's capability is now a function of how much money you spend on it, so asking what a model can do means nothing until you name a budget.
Holds NB Noam Brown
If intelligence is the process of acquiring skills, no single task demonstrates intelligence unless it is a meta-task of skill-acquisition across many tasks.
Holds
Solely measuring skill at any given task falls short of measuring intelligence, because skill is heavily modulated by prior knowledge and experience.
Holds
The Abstraction and Reasoning Corpus can measure human-like general fluid intelligence and enable fair comparisons between AI systems and humans.
Holds
A physical AI company needs three AIs, not one — the agent, the simulator and the critic — turning deployment into a flywheel.
Holds DD Dmitri Dolgov
A verifiable task can be optimised by reinforcement learning until a neural network performs it extremely well.
Holds
Building a realistic simulator is exactly as hard as building the agent, because the simulator is itself a large AI model.
Holds DD Dmitri Dolgov