Language models surprised us

Ajeya Cotra · 29 Aug 2023

Others here said this too

AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.

Their wordsThis rapid scaleup will probably drive another qualitative leap forward in capability like what we saw over the last 18 months.

Also said by Scott Alexander, Miles Brundage

Read the piece on planned-obsolescence.org ↗

4 korrents from this piece ↓

Ajeya Cotra did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

Our reading
Machine-learning researchers and superforecasters alike were surprised by how fast language models improved in 2022 and 2023.

Their wordsMost experts were surprised by progress in language models in 2022 and 2023.
Our reading
Forecasters underestimate AI spending even more than they underestimate AI capability, and the spending is what drives the next leap.

Their wordsMost importantly, ML experts and superforecasters both seem to be massively underestimating future spending on training runs.
Our reading
AI progress is faster than people expect, and ordinary scaling can be enough to solve problems that looked very hard.

Their wordsThis rapid scaleup will probably drive another qualitative leap forward in capability like what we saw over the last 18 months.
Our reading
Underestimating near-term AI progress is itself a danger, which is why researchers should register their forecasts in public.

Their wordsMassively underestimating near-future progress could be very risky.

Keep reading

The other pieces are the ones whose claims are worded most like this one's. That measures language, not agreement: they may argue the opposite.