Tap a claim on the ring to put it at the centre.
← Looped/recurrent-depth transformer architectures can improve model…
17 connected korrents · 18 moments from 10 Dec 2015 to 2 Nov 2026.
Everything filed under LLMs
LLMs
Everything filed under neural networks
neural networks
Everything filed under scaling laws
scaling laws
Everything filed under OpenAI
OpenAI
Everything filed under transformers
transformers
Everything filed under product management
product management
Everything filed under recursive self-improvement
recursive self-improvement Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Read this korrent: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. Last stated 2 months ago
9 Sept 2026
SR
Sebastian Raschka — holds since 2026-09-09 — tap for who they are
Same subject: Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. — tap to centre the map on it Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches. Last stated 6 years ago
28 May 2020
DA
Dario Amodei — holds since 2020-05-28 — tap for who they are
Same subject: Residual networks are easier to optimize than plain ones and keep gaining accuracy as depth increases. — tap to centre the map on it Residual networks are easier to optimize than plain ones and keep gaining accuracy as depth increases. Last stated 11 years ago
10 Dec 2015
JS
Jian Sun — holds since 2015-12-10 — tap for who they are
KH
Kaiming He — holds since 2015-12-10 — tap for who they are
Same subject: Test-time scaling seems to have gained a third axis: latent space reasoning iterations in looped transformers. — tap to centre the map on it Test-time scaling seems to have gained a third axis: latent space reasoning iterations in looped transformers. Last stated 2 months ago
5 Sept 2026
FC
François Chollet — holds since 2026-09-05 — tap for who they are
Same subject: Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. — tap to centre the map on it Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them. — tap to centre the map on it Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them. Last stated 6 months ago
16 May 2026
SR
Sebastian Raschka — holds since 2026-05-16 — tap for who they are
Same subject: Ranking a product's problems, fixing them and looping again is like running your own training process towards product-market fit. — tap to centre the map on it Ranking a product's problems, fixing them and looping again is like running your own training process towards product-market fit. Last stated 2 months ago
10 Sept 2026
RS
Robby Stein — holds since 2026-09-10 — tap for who they are
Same subject: Recursive self-improvement has not yet provided a bailout for large AI labs. — tap to centre the map on it Recursive self-improvement has not yet provided a bailout for large AI labs. Last stated 3 days ago
2 Nov 2026
TT
Tomasz Tunguz — holds since 2026-11-02 — tap for who they are
Same subject: Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches. — tap to centre the map on it Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches. Last stated 5 years ago
3 Jun 2021
GW
Gwern — holds since 2021-06-03 — tap for who they are
Same subject: A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. — tap to centre the map on it A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable. Last stated 11 years ago
10 Dec 2015
JS
Jian Sun — holds since 2015-12-10 — tap for who they are
KH
Kaiming He — holds since 2015-12-10 — tap for who they are
Same subject: A project-level agent memory file works like a neural net, steering the agent much as a fine-tuned LoRA steers a model. — tap to centre the map on it A project-level agent memory file works like a neural net, steering the agent much as a fine-tuned LoRA steers a model. Last stated 2 months ago
23 Aug 2026
KC
Kun Chen — holds since 2026-08-23 — tap for who they are
Same subject: A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. — tap to centre the map on it A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas. — tap to centre the map on it Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas. Last stated 2 years ago
20 Dec 2024
FC
François Chollet — holds since 2024-12-20 — tap for who they are
GM
Gary Marcus — holds since 2024-10-11 — tap for who they are
Same subject: A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. — tap to centre the map on it A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week. Last stated 2 weeks ago
21 Oct 2026
PS
Peter Steinberger — holds since 2026-10-21 — tap for who they are
Same subject: A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. — tap to centre the map on it A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests. Last stated 3 years ago
18 Mar 2024
SA
Sam Altman — holds since 2024-03-18 — tap for who they are
Same subject: Most computer scientists did not anticipate deep learning's rapid success scaling to its current level. — tap to centre the map on it Most computer scientists did not anticipate deep learning's rapid success scaling to its current level. Last stated 7 months ago
15 Apr 2026
DM
Dylan Matthews — holds since 2026-04-15 — tap for who they are
Same subject: "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. — tap to centre the map on it "Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next. Last stated 11 months ago
25 Nov 2025
IS
Ilya Sutskever — holds since 2025-11-25 — tap for who they are
Same subject: A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. — tap to centre the map on it A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number. Last stated 4 months ago
26 Jun 2026
NB
Noam Brown — holds since 2026-06-26 — tap for who they are
At the centre
Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.
Holds Read this korrent →
Scaling up language models greatly improves task-agnostic few-shot performance, sometimes matching prior fine-tuning approaches.
Holds
Residual networks are easier to optimize than plain ones and keep gaining accuracy as depth increases.
Holds
Test-time scaling seems to have gained a third axis: latent space reasoning iterations in looped transformers.
Holds
Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.
Holds
Larger language models already have enough capacity that extra per-layer embedding tricks add little benefit to them.
Holds
Ranking a product's problems, fixing them and looping again is like running your own training process towards product-market fit.
Holds
Recursive self-improvement has not yet provided a bailout for large AI labs.
Holds
Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches.
Holds
A layer should learn a residual with reference to its own input rather than an unreferenced function, which is what makes great depth trainable.
Holds
A project-level agent memory file works like a neural net, steering the agent much as a fine-tuned LoRA steers a model.
Holds
A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it.
Holds
Scaling the 2019-2023 recipe — same architecture, bigger model, more data — is not enough; further progress depends on new architectural ideas.
Holds
A PHP vulnerability found with GPT-5.6 Sol will cause havoc this week.
Holds
A startup should not begin life as a nonprofit and bolt a for-profit arm on later, whatever OpenAI's own history suggests.
Holds
Most computer scientists did not anticipate deep learning's rapid success scaling to its current level.
Holds
"Scaling" was powerful because it was one word: naming a research direction is what tells a whole field what to do next.
Holds
A benchmark result should be reported under a stated budget, or as a curve against test-time compute — never as a single number.
Holds