korrents
Christopher Olah

Christopher Olah

@chris-olah · 11 positions · 0 changes of mind

Interpretability researcher at Anthropic, which he co-founded; before that at OpenAI and Google Brain.

Everything they publish, on ppll ↗

Christopher Olah did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

  1. Our reading

    Sharing credit generously benefits everyone because it encourages future collaboration.

    Their wordsCredit isn't zero sum. It is often in everyone's benefit to be generous with credit, because it creates an incentive for others to help in the future.

    ↗Collaboration and Credit Principlescolah.github.io 1st of 3 in this piece

  3. Our reading

    Failing to credit someone who deserves it does more harm than mistakenly crediting someone who doesn't.

    Their wordsthe harm of not crediting someone who deserves it is often greater than the harm of crediting someone who doesn't deserve it.

    ↗Collaboration and Credit Principlescolah.github.io 2nd of 3 in this piece

  5. Our reading

    People tend to overestimate their own contributions to collaborative work.

    Their wordsRemember that you are likely overestimating your own contributions.

    ↗Collaboration and Credit Principlescolah.github.io 3rd of 3 in this piece

  6. 4 years earlier

  7. Our reading

    Deep learning will be understood very differently in thirty years than it is today.

    Their wordsI think it is very likely that, reflecting back in 30 years, we will see deep learning very differently.

    ↗Neural Networks, Types, and Functional Programmingcolah.github.io 1st of 2 in this piece

  9. Our reading

    The connection between representations and types may be fundamentally significant, akin to links among geometry, logic, topology and functional programming.

    Their wordsThe known connection between geometry, logic, topology, and functional programming suggests that the connections between representations and types may be of fundamental significance.

    ↗Neural Networks, Types, and Functional Programmingcolah.github.io 2nd of 2 in this piece

  10. 3 days earlier

  11. Our reading

    An idea can look obvious in hindsight only because the insight that makes it obvious was itself the hard discovery.

    Their wordsIt also wasn't obvious that derivatives were the right way to train them. Those are only obvious once you realize you can quickly calculate derivatives. There was a circular dependency.

    ↗Calculus on Computational Graphs: Backpropagationcolah.github.io 1st of 2 in this piece

  13. Our reading

    Once a problem is correctly framed, the hardest part of solving it is already done.

    Their wordsThat's the benefit of hindsight. Once you've framed the question, the hardest work is already done.

    ↗Calculus on Computational Graphs: Backpropagationcolah.github.io 2nd of 2 in this piece

  14. 4 days earlier

  15. Our reading

    Standard recurrent neural networks are theoretically capable of representing long-term dependencies but fail to learn them from data in practice.

    neural networks

    Their wordsIn theory, RNNs are absolutely capable of handling such "long-term dependencies." A human could carefully pick parameters for them to solve toy problems of this form. Sadly, in practice, RNNs don't seem to be able to learn them.

    ↗Understanding LSTM Networkscolah.github.io 1st of 2 in this piece

  17. Our reading

    LSTM architectures are designed so that retaining information over long time spans is their default behavior rather than something they must struggle to learn.

    neural networks

    Their wordsLSTMs are explicitly designed to avoid the long-term dependency problem. Remembering information for long periods of time is practically their default behavior, not something they struggle to learn!

    ↗Understanding LSTM Networkscolah.github.io 2nd of 2 in this piece

  18. 7 months earlier

  19. Our reading

    In a dimensionality-reduced visualization of high-dimensional data, the raw position of points carries no meaning on its own.

    Their wordsThe problem is that in dimensionality reduced plots of high-dimensional data, position doesn't explain the data points.

    ↗Visualizing Representations: Deep Learning and Human Beingscolah.github.io 1st of 2 in this piece

  21. Our reading

    Understanding a machine learning model and understanding the data it operates on are inseparable tasks.

    Their wordsUnderstanding data and understanding models that work on that data are intimately linked.

    ↗Visualizing Representations: Deep Learning and Human Beingscolah.github.io 2nd of 2 in this piece