korrents
Gwern

What Gwern thinks about LLMs

@gwern · 15 positions · On ppll ↗

Pseudonymous essayist. Writes at gwern.net on AI scaling, statistics, genetics and long-running internet research, with heavy citation and a habit of revising old essays in place rather than leaving them wrong.

Gwern did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

3 dated positions, 2021, in their own words. Our reading of what Gwern has said — not written or endorsed by them.

3 positions so far — this page is not yet offered to search engines.

  1. if the marginal return on this research direction had fallen below that of competitors, we would see those neglected directions invade Transformer topics-while we continue to see the reverse, and many applications as yet untouched by all the new approaches, suggesting that we still don't pay enough attention

    ↗April 2021 newslettergwern.substack.com 1st of 3 in this piece

  2. 4 months earlier
  3. the mixture-of-experts approach, while scaling stably, starts showing its limits

    ↗Jan 2021 Gwern.net Newslettergwern.substack.com 1st of 3 in this piece

  4. The fact that the prefix-tuning, by directly optimizing the prompt embeddings, yields better performance than even single optimized text prompts, suggests so.

    ↗Jan 2021 Gwern.net Newslettergwern.substack.com 3rd of 3 in this piece