Tap a claim on the ring to put it at the centre.
← Unidirectional restrictions are sub-optimal for sentence-level tasks…
17 connected korrents · 13 moments from 12 Jun 2017 to 21 Oct 2026.
Everything filed under LLMs
LLMs
Everything filed under transformers
transformers
Everything filed under scaling laws
scaling laws
Everything filed under OpenAI
OpenAI
Everything filed under AI agents
AI agents Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context. Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: Current techniques restrict pre-trained representation power because standard language models are unidirectional. — tap to centre the map on it Current techniques restrict pre-trained representation power because standard language models are unidirectional. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: The next-token language modeling objective is misaligned with following user instructions helpfully and safely. — tap to centre the map on it The next-token language modeling objective is misaligned with following user instructions helpfully and safely. Last stated 5 years ago
4 Mar 2022
JL
Jan Leike — holds since 2022-03-04 — tap for who they are
Same subject: A large context window does not replace fine-tuning, because fine-tuning serves the separate purpose of instilling preferences and demonstrating a task. — tap to centre the map on it A large context window does not replace fine-tuning, because fine-tuning serves the separate purpose of instilling preferences and demonstrating a task. Last stated 3 years ago
22 Mar 2024
EG
Elad Gil — holds since 2024-03-22 — tap for who they are
Same subject: Reasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own. — tap to centre the map on it Reasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own. Last stated 2 years ago
3 Feb 2025
NL
Nathan Lambert — holds since 2025-02-03 — tap for who they are
Same subject: A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. — tap to centre the map on it A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models. Last stated 8 years ago
11 Oct 2018
KT
Kristina Toutanova — holds since 2018-10-11 — tap for who they are
MC
Ming-Wei Chang — holds since 2018-10-11 — tap for who they are
KL
Kenton Lee — holds since 2018-10-11 — tap for who they are
JD
Jacob Devlin — holds since 2018-10-11 — tap for who they are
Same subject: Full fine-tuning of large language models has become too costly, making parameter-efficient fine-tuning methods necessary. — tap to centre the map on it Full fine-tuning of large language models has become too costly, making parameter-efficient fine-tuning methods necessary. Last stated 3 years ago
5 Dec 2023
SR
Sebastian Ruder — holds since 2023-12-05 — tap for who they are
Same subject: The scarce skill is now context switching across parallel agents, not sustained deep work. — tap to centre the map on it The scarce skill is now context switching across parallel agents, not sustained deep work. Last stated 8 months ago
4 Mar 2026
BC
Boris Cherny — holds since 2026-03-04 — tap for who they are
Same subject: Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. — tap to centre the map on it Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. Last stated 6 years ago
28 May 2020
DA
Dario Amodei — holds since 2020-05-28 — tap for who they are
Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: Self-attention can yield more interpretable models. — tap to centre the map on it Self-attention can yield more interpretable models. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. — tap to centre the map on it Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. Last stated 4 days ago
21 Oct 2026
PS
Peter Steinberger — holds since 2026-10-21 — tap for who they are
Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. — tap to centre the map on it Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers. Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
Same subject: A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. — tap to centre the map on it A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from. Last stated 7 months ago
13 Mar 2026
DP
Dylan Patel — holds since 2026-03-13 — tap for who they are
At the centre
Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context.
Holds Read this korrent →
Current techniques restrict pre-trained representation power because standard language models are unidirectional.
Holds
The next-token language modeling objective is misaligned with following user instructions helpfully and safely.
Holds
A large context window does not replace fine-tuning, because fine-tuning serves the separate purpose of instilling preferences and demonstrating a task.
Holds
Reasoning will generalize the way instruction tuning did: add enough verifiable domains and, at some point nobody can yet locate, the rest start working on their own.
Holds
A deep bidirectional model is strictly more powerful than a left-to-right model or a shallow concatenation of unidirectional models.
Holds
Full fine-tuning of large language models has become too costly, making parameter-efficient fine-tuning methods necessary.
Holds
The scarce skill is now context switching across parallel agents, not sustained deep work.
Holds
Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches.
Holds
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Holds
Self-attention can yield more interpretable models.
Holds
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.
Holds
A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week.
Holds
A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests.
Holds
Advertising was a necessary phase for the internet but a momentary industry, and an AI people pay for is better because they know the answers are not influenced by advertisers.
Holds
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Holds
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds
A lab should spend most of its compute on research rather than on building the next model, because research is where the tenfold yearly efficiency gains come from.
Holds