korrents
Sebastian Ruder

Sebastian Ruder

@sebastian-ruder · 23 positions · On ppll ↗

Research scientist at Meta working on multilingual models and evaluation; led the multilingual team at Cohere and was at Google DeepMind before that.

Sebastian Ruder did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

2 of Sebastian Ruder's claims fit 2 mental models. Most often: Bottlenecks, Less, but better.

Models we see in what they say

Our reading: their claim applies the idea without naming it. The claim is theirs; filing it here is ours.

Bottlenecks

A system moves only as fast as its narrowest point; speed up anything else and nothing changes.

Used by 91 others

Available online text data is becoming a limiting factor for AI training, and repeating it yields diminishing returns.

  1. Sebastian Ruder Research scientist at Meta working on multilingual models and evaluation recent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns NLP Research in the Era of LLMsruder.io · 19 Dec 2023All korrents from this piece
    Their wordsrecent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns

    ↗NLP Research in the Era of LLMsruder.io

    19 Dec 2023

Less, but better

Find the essence and cut the rest; simplicity is hard work, not the absence of effort.

Used by 59 others

In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset.

  1. Sebastian Ruder Research scientist at Meta working on multilingual models and evaluation training on a small set of high-quality data outperforms instruction-tuning on larger, noisier data. An Overview of Instruction Tuning Dataruder.io · 15 Nov 2023All korrents from this piece
    Their wordstraining on a small set of high-quality data outperforms instruction-tuning on larger, noisier data.

    ↗An Overview of Instruction Tuning Dataruder.io

    15 Nov 2023