korrents

On the map

Tap a claim on the ring to put it at the centre.

← Linear probes are very strong classifiers, and attention-head probes are even stronger.

12 connected korrents · 11 moments from 12 Jun 2017 to 23 Oct 2026.

Everything filed under neural networks neural networks Everything filed under transformers transformers Everything filed under cybersecurity cybersecurity Everything filed under AI and science AI and science Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Linear probes are very strong classifiers, and attention-head probes are even stronger. Linear probes are very strong classifiers,and attention-head probes are evenstronger. Last stated 11 months ago 20 Dec 2025 NN Neel Nanda — holds since 2025-12-20 — tap for who they are Same subject: Crude, high-level interpretability probes might get us pretty far without a herculean bottom-up understanding of models. — tap to centre the map on it Crude, high-level interpretabilityprobes might get us pretty farwithout a herculean bottom-upunderstanding of models. Last stated 4 years ago 12 May 2023 AC Ajeya Cotra — holds since 2023-05-12 — tap for who they are Same subject: AI will make it easier to probe essentially everything. — tap to centre the map on it AI will make it easier to probeessentially everything. Last stated 4 weeks ago 23 Oct 2026 ZM Zvi Mowshowitz — holds since 2026-10-23 — tap for who they are Same subject: High-quality data collection depends more on careful human execution than on machine learning techniques alone. — tap to centre the map on it High-quality data collection dependsmore on careful human execution thanon machine learning techniquesalone. Last stated 3 years ago 5 Feb 2024 LW Lilian Weng — holds since 2024-02-05 — tap for who they are Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can relyentirely on self-attention for inputand output representations withoutsequence-aligned RNNs orconvolution. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it Looped/recurrent-depth transformerarchitectures can improve modelquality at a fixed compute budgetwhen the model is large enough. Last stated 2 months ago 9 Sept 2026 SR Sebastian Raschka — holds since 2026-09-09 — tap for who they are Same subject: Self-attention can yield more interpretable models. — tap to centre the map on it Self-attention can yield moreinterpretable models. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: The transformer was close to the last answer in architecture: nothing about the core design has changed since 2017. — tap to centre the map on it The transformer was close to thelast answer in architecture: nothingabout the core design has changedsince 2017. Last stated 2 years ago 19 Jun 2024 AS Aravind Srinivas — holds since 2024-06-19 — tap for who they are Same subject: A single AI paper rarely changes everything; the last one that did was 'Attention Is All You Need' in 2017. — tap to centre the map on it A single AI paper rarely changeseverything; the last one that didwas 'Attention Is All You Need' in2017. Last stated 2 years ago 6 Jan 2025 AR Aishwarya Naresh Reganti — holds since 2025-01-06 — tap for who they are Same subject: As reasoning models and agents retain more tokens for longer, KV-cache size and attention cost become the main constraint on scaling context. — tap to centre the map on it As reasoning models and agentsretain more tokens for longer,KV-cache size and attention costbecome the main constraint onscaling context. Last stated 6 months ago 16 May 2026 SR Sebastian Raschka — holds since 2026-05-16 — tap for who they are Same subject: A small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it. — tap to centre the map on it A small enough winning ticket learnsfaster than the network it was cutout of, and ends up more accuratethan it. Last stated 9 years ago 9 Mar 2018 MC Michael Carbin — holds since 2018-03-09 — tap for who they are JF Jonathan Frankle — holds since 2018-03-09 — tap for who they are Same subject: A winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work. — tap to centre the map on it A winning ticket wins on its initialweights: the connections it keepsstarted at values that happen tomake training work. Last stated 9 years ago 9 Mar 2018 MC Michael Carbin — holds since 2018-03-09 — tap for who they are JF Jonathan Frankle — holds since 2018-03-09 — tap for who they are Same subject: Intelligence is the execution of small, simplicity-weighted programs found by search over smooth loss landscapes in overparameterized networks, approaching Bayes-optimal reasoning as scale increases. — tap to centre the map on it Intelligence is the execution ofsmall, simplicity-weighted programsfound by search over smooth losslandscapes in overparameterizednetworks, approaching Bayes-optimalreasoning as scale increases. Last stated 5 years ago 11 Jun 2021 GW Gwern — holds since 2021-06-11 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 2015 to today — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Linear probes are very strong classifiers, and attention-head probes are even stronger. Last stated 20 Dec 2025 · 11 months ago Holds Neel Nanda Read this korrent →