korrents

← People and their mental models

Gwern's mental models

2 claims Gwern made fit 2 mental models. Most often: S-curves and diminishing returns, Second-order thinking. Everything they said here.

Models we see in what they say

Our reading: their claim applies the idea without naming it. The claim is theirs; filing it here is ours.

S-curves and diminishing returns

Most growth slows as it matures; the question is whether the curve is flattening or something new is restocking it.

Used by 22 others

Research on transformer-style attention mechanisms has not yet reached diminishing returns compared to alternative approaches.

  1. Gwern Pseudonymous essayist if the marginal return on this research direction had fallen below that of competitors, we would see those neglected directions invade Transformer topics-while we continue to see the reverse, and many applications as yet untouched by all the new approaches, suggesting that we still don't pay enough attention April 2021 newslettergwern.substack.com · 3 Jun 2021All korrents from this piece
    Their wordsif the marginal return on this research direction had fallen below that of competitors, we would see those neglected directions invade Transformer topics-while we continue to see the reverse, and many applications as yet untouched by all the new approaches, suggesting that we still don't pay enough attention

    ↗April 2021 newslettergwern.substack.com

    3 Jun 2021

Second-order thinking

Ask "and then what?": follow a decision past its first effect to the ones that come after.

Used by 86 others