korrents

On the map

Tap a claim on the ring to put it at the centre.

← Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions.

14 connected korrents · 12 moments from 1 Jan 1998 to 24 Sept 2026.

Everything filed under transformers transformers Everything filed under neural networks neural networks Everything filed under innovation innovation Everything filed under design design Everything filed under LLMs LLMs Same subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subjectSame subject Read this korrent: Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. Sequence transduction can use a simplearchitecture based solely on attention,without recurrence or convolutions. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are LJ Llion Jones — holds since 2017-06-12 — tap for who they are ŁK Łukasz Kaiser — holds since 2017-06-12 — tap for who they are Same subject: A transduction model can rely entirely on self-attention for input and output representations without sequence-aligned RNNs or convolution. — tap to centre the map on it A transduction model can relyentirely on self-attention for inputand output representations withoutsequence-aligned RNNs orconvolution. Last stated 9 years ago 12 Jun 2017 IP Illia Polosukhin — holds since 2017-06-12 — tap for who they are JU Jakob Uszkoreit — holds since 2017-06-12 — tap for who they are NP Niki Parmar — holds since 2017-06-12 — tap for who they are AV Ashish Vaswani — holds since 2017-06-12 — tap for who they are NS Noam Shazeer — holds since 2017-06-12 — tap for who they are AG Aidan N. Gomez — holds since 2017-06-12 — tap for who they are +2 2 more Same subject: Current techniques restrict pre-trained representation power because standard language models are unidirectional. — tap to centre the map on it Current techniques restrictpre-trained representation powerbecause standard language models areunidirectional. Last stated 8 years ago 11 Oct 2018 KT Kristina Toutanova — holds since 2018-10-11 — tap for who they are MC Ming-Wei Chang — holds since 2018-10-11 — tap for who they are KL Kenton Lee — holds since 2018-10-11 — tap for who they are JD Jacob Devlin — holds since 2018-10-11 — tap for who they are Same subject: Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches. — tap to centre the map on it Research on transformer-styleattention mechanisms has not yetreached diminishing returns comparedto alternative approaches. Last stated 5 years ago 3 Jun 2021 GW Gwern — holds since 2021-06-03 — tap for who they are Same subject: Looped/recurrent-depth transformer architectures can improve model quality at a fixed compute budget when the model is large enough. — tap to centre the map on it Looped/recurrent-depth transformerarchitectures can improve modelquality at a fixed compute budgetwhen the model is large enough. Last stated 2 months ago 9 Sept 2026 SR Sebastian Raschka — holds since 2026-09-09 — tap for who they are Same subject: Graph transformer networks allow multimodule document recognition systems to be trained globally with gradient methods to minimize overall performance. — tap to centre the map on it Graph transformer networks allowmultimodule document recognitionsystems to be trained globally withgradient methods to minimize overallperformance. Last stated 29 years ago 1 Jan 1998 YB Yoshua Bengio — holds since 1998-01-01 — tap for who they are Same subject: Standard recurrent neural networks are theoretically capable of representing long-term dependencies but fail to learn them from data in practice. — tap to centre the map on it Standard recurrent neural networksare theoretically capable ofrepresenting long-term dependenciesbut fail to learn them from data inpractice. Last stated 11 years ago 27 Aug 2015 CO Christopher Olah — holds since 2015-08-27 — tap for who they are Same subject: Pre-trained representations reduce the need for many heavily-engineered task-specific architectures. — tap to centre the map on it Pre-trained representations reducethe need for many heavily-engineeredtask-specific architectures. Last stated 8 years ago 11 Oct 2018 KT Kristina Toutanova — holds since 2018-10-11 — tap for who they are MC Ming-Wei Chang — holds since 2018-10-11 — tap for who they are KL Kenton Lee — holds since 2018-10-11 — tap for who they are JD Jacob Devlin — holds since 2018-10-11 — tap for who they are Same subject: We have almost no automated techniques for making a system transfer what it learns, and none of the few we have are used in modern deep learning. — tap to centre the map on it We have almost no automatedtechniques for making a systemtransfer what it learns, and none ofthe few we have are used in moderndeep learning. Last stated a year ago 26 Sept 2025 RS Richard Sutton — holds since 2025-09-26 — tap for who they are Same subject: "Lots of anonymous people worked it out by trial and error" is never the real story of an invention: on closer investigation, identifiable people always turn out to be responsible. — tap to centre the map on it "Lots of anonymous people worked itout by trial and error" is never thereal story of an invention: oncloser investigation, identifiablepeople always turn out to beresponsible. Last stated 3 years ago 25 Oct 2023 AH Anton Howes — holds since 2023-10-25 — tap for who they are Same subject: A business can have market fit, real advantages, and sustained effort behind a hard product and still fail if it doesn't give customers a strong enough reason to switch from what they already use. — tap to centre the map on it A business can have market fit, realadvantages, and sustained effortbehind a hard product and still failif it doesn't give customers astrong enough reason to switch fromwhat they already use. Last stated 5 months ago 2 Jun 2026 JJ Justin Jackson — holds since 2026-06-02 — tap for who they are Same subject: A capability existing is not the same as a capability being absorbed: how fast a technology changes the world is not set by how fast the technology improves. — tap to centre the map on it A capability existing is not thesame as a capability being absorbed:how fast a technology changes theworld is not set by how fast thetechnology improves. Last stated 3 months ago 21 Jul 2026 RT Ruxandra Teslo — holds since 2026-07-21 — tap for who they are Same subject: A $1,299, 100-gram pair of slip-on glasses seems far more enticing than a $3,699, 700-gram headset strapped to the face. — tap to centre the map on it A $1,299, 100-gram pair of slip-onglasses seems far more enticing thana $3,699, 700-gram headset strappedto the face. Last stated a month ago 24 Sept 2026 MS M.G. Siegler — holds since 2026-09-24 — tap for who they are Same subject: A baby-step version of eval-driven design is to sketch out 10 to 20 hero flows and run the feature or interaction through them to see where it falls apart. — tap to centre the map on it A baby-step version of eval-drivendesign is to sketch out 10 to 20hero flows and run the feature orinteraction through them to seewhere it falls apart. Last stated 2 months ago 9 Sept 2026 EB Ed Bayes — holds since 2026-09-09 — tap for who they are Same subject: A blank chat box is lazy: it violates the first rule of a good user experience, that it is obvious what you can do. — tap to centre the map on it A blank chat box is lazy: itviolates the first rule of a gooduser experience, that it is obviouswhat you can do. Last stated a year ago 12 May 2025 JZ Julie Zhuo — holds since 2025-05-12 — tap for who they are
same subject or similar wordinga cloud: claims about one subject, named for itbar: when it was last stated, on a scale from 1998 to today (stretched back to the oldest claim here) — full is todaya face: someone who holds the claim — tap it for who they are

At the centre Sequence transduction can use a simple architecture based solely on attention, without recurrence or convolutions. Last stated 12 Jun 2017 · 9 years ago Holds Illia PolosukhinJakob UszkoreitNiki ParmarAshish VaswaniNoam ShazeerAidan N. GomezLlion JonesŁukasz Kaiser Read this korrent →