Gwern Pseudonymous essayist if the marginal return on this research direction had fallen below that of competitors, we would see those neglected directions invade Transformer topics-while we continue to see the reverse, and many applications as yet untouched by all the new approaches, suggesting that we still don't pay enough attention April 2021 newslettergwern.substack.com · 3 Jun 2021All korrents from this piece
Their wordsif the marginal return on this research direction had fallen below that of competitors, we would see those neglected directions invade Transformer topics-while we continue to see the reverse, and many applications as yet untouched by all the new approaches, suggesting that we still don't pay enough attention
Gwern
Pseudonymous essayist. Writes at gwern.net on AI scaling, statistics, genetics and long-running internet research, with heavy citation and a habit of revising old essays in place rather than leaving them wrong.
Gwern did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
2 of Gwern's claims fit 2 mental models. Most often: S-curves and diminishing returns, Second-order thinking.
Models we see in what they say
S-curves and diminishing returns
Most growth slows as it matures; the question is whether the curve is flattening or something new is restocking it.
Second-order thinking
Ask "and then what?": follow a decision past its first effect to the ones that come after.
Gwern Pseudonymous essayist there's nothing so expensive as demanding research be cheap. April 2021 newslettergwern.substack.com · 3 Jun 2021All korrents from this piece
Their wordsthere's nothing so expensive as demanding research be cheap.