korrents

korrents · piece

May 2021 Gwern.net Newsletter

Gwern · 11 Jun 2021

Read the piece on gwern.substack.com ↗

3 korrents from this piece ↓

Gwern did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

  1. The more people tweet about how MLP-Mixer is overhyped because is −X% worse than the ultra-hand-optimized baseline or requires Y× more FLOPS, the more they demonstrate precisely why this sort of research is so important!
  2. The Master Synthesis: intelligence is execution of small simplicity-weighted programs, best discovered by search over smooth loss landscapes like that of highly-overparameterized differentiable networks containing lottery-ticket subnetworks which are ensembled/averaged over, approaching Bayes-optimal reasoning in the limit (as nearest-neighbors-like high dimensional interpolation / memorization gives way to algorithmic generalization / interpolation on a more abstract level)
  3. human intelligence's g is real but is the overall 'pool' of neural resources which derives from overall body integrity because the number of neurons, their density, their myelination, resistance to damage and infection etc, is causally downstream of all body and developmental systems, creating a huge mutational target

Keep reading

The other pieces are the ones whose claims are worded most like this one's. That measures language, not agreement: they may argue the opposite.