korrents

Our readingWhat is a korrent?

Unidirectional restrictions are sub-optimal for sentence-level tasks and harmful for token-level tasks that need bidirectional context.

Drawn from what Kristina Toutanova, Kenton Lee and 2 others said

SubjectstransformersLLMs
What this subject means

transformers The architecture that dropped recurrence in favour of attention, and that almost every large model since is built on.

What they actually said

  1. Kristina Toutanova One of the four authors of the 2018 paper "BERT:… Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions. BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org · 11 Oct 2018All 16 korrents from this piece
    Their wordsSuch restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

    Where this was said

    ↗BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org

    11 Oct 2018

    All 16 korrents from this piecethis one is 8th

  2. Kenton Lee One of the four authors of the 2018 paper "BERT:… Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions. BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org · 11 Oct 2018All 16 korrents from this piece
    Their wordsSuch restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

    Where this was said

    ↗BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org

    11 Oct 2018

    All 16 korrents from this piecethis one is 7th

  3. Ming-Wei Chang One of the four authors of the 2018 paper "BERT:… Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions. BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org · 11 Oct 2018All 16 korrents from this piece
    Their wordsSuch restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

    Where this was said

    ↗BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org

    11 Oct 2018

    All 16 korrents from this piecethis one is 6th

  4. Jacob Devlin One of the four authors of the 2018 paper "BERT:… Such restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions. BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org · 11 Oct 2018All 16 korrents from this piece
    Their wordsSuch restrictions are sub-optimal for sentence-level tasks, and could be very harmful when applying fine-tuning based approaches to token-level tasks such as question answering, where it is crucial to incorporate context from both directions.

    Where this was said

    ↗BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingarxiv.org

    11 Oct 2018

    All 16 korrents from this piecethis one is 5th

Added to korrents 9 Sept 2026 · How quotes work · Something wrong? Tell us

Do you hold this korrent?Do you also believe this?

I hold this too

Related korrents

Shared subject first, then similar wording. They may agree or disagree.

See this claim on the map