korrents

Our readingWhat is a korrent?

Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches.

Drawn from what Gwern said

What these subjects mean

transformers The architecture that dropped recurrence in favour of attention, and that almost every large model since is built on.

innovation Where new things come from and how they spread: invention against diffusion, and being near the factory floor.

S-curves and diminishing returns

What Gwern actually said

  1. Gwern Pseudonymous essayist
    if the marginal return on this research direction had fallen below that of competitors, we would see those neglected directions invade Transformer topics-while we continue to see the reverse, and many applications as yet untouched by all the new approaches, suggesting that we still don't pay enough attention

Added to korrents 25 Sept 2026 · How quotes work · Something wrong? Tell us

Do you hold this korrent?Do you also believe this?

I hold this too

Related korrents

Shared subject first, then similar wording. They may agree or disagree.

See this claim on the map