korrents

← People and their mental models

Sebastian Ruder's mental models

2 claims Sebastian Ruder made fit 2 mental models. Most often: Bottlenecks, Less, but better. Everything they said here.

Models we see in what they say

Our reading: their claim applies the idea without naming it. The claim is theirs; filing it here is ours.

Bottlenecks

A system moves only as fast as its narrowest point; speed up anything else and nothing changes.

Used by 88 others

Available online text data is becoming a limiting factor for AI training, and repeating it yields diminishing returns.

  1. Sebastian Ruder Research scientist at Meta working on multilingual models and evaluation recent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns NLP Research in the Era of LLMsruder.io · 19 Dec 2023All korrents from this piece
    Their wordsrecent LLMs are reaching the limits of text data online and repeating data eventually leads to diminishing returns

    ↗NLP Research in the Era of LLMsruder.io

    19 Dec 2023

Less, but better

Find the essence and cut the rest; simplicity is hard work, not the absence of effort.

Used by 59 others

In instruction tuning, a smaller set of high-quality training examples outperforms a larger, noisier dataset.

  1. Sebastian Ruder Research scientist at Meta working on multilingual models and evaluation training on a small set of high-quality data outperforms instruction-tuning on larger, noisier data. An Overview of Instruction Tuning Dataruder.io · 15 Nov 2023All korrents from this piece
    Their wordstraining on a small set of high-quality data outperforms instruction-tuning on larger, noisier data.

    ↗An Overview of Instruction Tuning Dataruder.io

    15 Nov 2023