korrents

Our readingWhat is a korrent?

Explicit positional methods are useful or essential because they provide attention with a direct signal about positions.

Drawn from what Sebastian Raschka said

What this subject means

transformers The architecture that dropped recurrence in favour of attention, and that almost every large model since is built on.

What Sebastian Raschka actually said

  1. Sebastian Raschka AI research engineer working on large language models
    Nonetheless, GPT-2 and most newer LLMs include explicit positional information because it gives attention a direct signal about positions. That’s because even though the causal mask determines which tokens are accessible and provides implicit order information, it doesn’t distinguish a separation of one position from a separation of ten, for example. So, explicit positional methods are still useful (or even essential), since they supply that information.

Added to korrents 15 Nov 2026 · How quotes work · Something wrong? Tell us

Do you hold this korrent?Do you also believe this?

I hold this too

Related korrents

Shared subject first, then similar wording. They may agree or disagree.

See this claim on the map