korrents

korrents · papers

Language models are few-shot learners (with 30 co-authors)

Dario Amodei · 28 May 2020 · arxiv.org

3 korrents from this paper

In plain words

Larger language models can do new tasks after seeing just a few examples in plain text, with no extra training, sometimes matching systems that got thousands of examples. This reduces the need to collect huge labeled datasets for every new task. It reports results on a 175 billion parameter model tested this way, building on prior pre-training work that still required task-specific extra training.

Our summary of the paper, not the authors' words — written to be readable without the field's vocabulary, from the stored copy of the paper and nothing else. Drafted with xai:grok-4.5 and checked by a person. The authors' own sentences are the quotes below.

Near this, by wording

Papers whose claims are worded most like this one's, found by the same hourly pass that draws the map. It is a measure of LANGUAGE, not of agreement or of citation: two papers can be near each other here and flatly contradict one another.

Dario Amodei did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

  1. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do.
  2. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches.
  3. Ultimately, however, one-shot, or even sometimes zero-shot, seem like the fairest comparisons to human performance, and are important targets for future work.