Tap a claim on the ring to put it at the centre.
← Linear probes are very strong classifiers, and attention-head probes are even stronger.
12 connected korrents · 11 moments from 12 Jun 2017 to 23 Oct 2026.
Everything filed under neural networks
neural networks
Everything filed under transformers
transformers
Everything filed under cybersecurity
cybersecurity
Everything filed under AI and science
AI and science
Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject Same subject
Read this korrent: Linear probes are very strong classifiers, and attention-head probes are even stronger.
Linear probes are very strong classifiers, and attention-head probes are even stronger.
Last stated 11 months ago
20 Dec 2025
NN
Neel Nanda — holds since 2025-12-20 — tap for who they are
Same subject: Crude, high-level interpretability probes might get us pretty far without a herculean bottom-up understanding of models. — tap to centre the map on it
Crude, high-level interpretability probes might get us pretty far without a herculean bottom-up understanding of models.
Last stated 4 years ago
12 May 2023
AC
Ajeya Cotra — holds since 2023-05-12 — tap for who they are
Same subject: AI will make it easier to probe essentially everything. — tap to centre the map on it
AI will make it easier to probe essentially everything.
Last stated 4 weeks ago
23 Oct 2026
ZM
Zvi Mowshowitz — holds since 2026-10-23 — tap for who they are
Same subject: High-quality data collection depends more on careful human execution than on machine learning techniques alone. — tap to centre the map on it
High-quality data collection depends more on careful human execution than on machine learning techniques alone.
Last stated 3 years ago
5 Feb 2024
LW
Lilian Weng — holds since 2024-02-05 — tap for who they are
Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it
A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution.
Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it
Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough.
Last stated 2 months ago
9 Sept 2026
SR
Sebastian Raschka — holds since 2026-09-09 — tap for who they are
Same subject: Self-attention can yield more interpretable models. — tap to centre the map on it
Self-attention can yield more interpretable models.
Last stated 9 years ago
12 Jun 2017
IP
Illia Polosukhin — holds since 2017-06-12 — tap for who they are
JU
Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are
NP
Niki Parmar — holds since 2017-06-12 — tap for who they are
AV
Ashish Vaswani — holds since 2017-06-12 — tap for who they are
NS
Noam Shazeer — holds since 2017-06-12 — tap for who they are
AG
Aidan N. Gomez — holds since 2017-06-12 — tap for who they are
+2
2 more
Same subject: The transformer was close to the last answer in architecture: nothing about the core design has changed since 2017. — tap to centre the map on it
The transformer was close to the last answer in architecture: nothing about the core design has changed since 2017.
Last stated 2 years ago
19 Jun 2024
AS
Aravind Srinivas — holds since 2024-06-19 — tap for who they are
Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it
A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017.
Last stated 2 years ago
6 Jan 2025
AR
Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are
Same subject: As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context. — tap to centre the map on it
As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context.
Last stated 6 months ago
16 May 2026
SR
Sebastian Raschka — holds since 2026-05-16 — tap for who they are
Same subject: A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. — tap to centre the map on it
A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it.
Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: A winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work. — tap to centre the map on it
A winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work.
Last stated 9 years ago
9 Mar 2018
MC
Michael Carbin — holds since 2018-03-09 — tap for who they are
JF
Jonathan Frankle — holds since 2018-03-09 — tap for who they are
Same subject: Intelligence is the execution of small, simplicity-weighted programs found by search over smooth loss landscapes in overparameterized networks, approaching Bayes-optimal reasoning as scale increases. — tap to centre the map on it
Intelligence is the execution of small, simplicity-weighted programs found by search over smooth loss landscapes in overparameterized networks, approaching Bayes-optimal reasoning as scale increases.
Last stated 5 years ago
11 Jun 2021
GW
Gwern — holds since 2021-06-11 — tap for who they are
same subject or similar wording a cloud: claims about one subject, named for it bar: when it was last stated, on a scale from 2015 to today — full is today a face: someone who holds the claim — tap it for who they are