korrents

korrents · piece

Calculus on Computational Graphs: Backpropagation

Christopher Olah · 31 Aug 2015 · colah.github.io ↗

2 korrents from this piece

Christopher Olah did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

  1. Our reading

    An idea can look obvious in hindsight only because the insight that makes it obvious was itself the hard discovery.

    Their wordsIt also wasn't obvious that derivatives were the right way to train them. Those are only obvious once you realize you can quickly calculate derivatives. There was a circular dependency.

  2. Our reading

    Once a problem is correctly framed, the hardest part of solving it is already done.

    Their wordsThat's the benefit of hindsight. Once you've framed the question, the hardest work is already done.