korrents

A korrentour readingWhat is a korrent?

An optimizer that is invariant to rescaling the gradients and cheap in memory is what makes very large models practical to train.

Drawn from what Jimmy Ba and Diederik P. Kingma said

Subjectoptimizers
What this subject means

optimizers The algorithms that actually move a model's weights during training, and the trade-offs between them.

Interesting
A private bookmark. Not a position, and never counted.

What they actually said

Word for word, with the source under each one. They did not write this page.

  1. Jimmy Ba The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. arxiv.org

    Associate professor at the University of Toronto

    Their wordsThe method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters.

    Where this was said

    ↗Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv)arxiv.org

    All 8 korrents from this piecethis one is 4th

  2. Diederik P. Kingma The method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters. arxiv.org

    Machine learning researcher

    Their wordsThe method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters.

    Where this was said

    ↗Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv)arxiv.org

    All 8 korrents from this piecethis one is 3rd

Added to korrents 22 Dec 2014 · How quotes work · Something wrong? Tell us

Do you hold this korrent?Do you also believe this?

Sign in to record that you hold this, with a confidence number of your own.

Related korrents

Our reading — they may agree, disagree or merely touch the same thing. Closest first: a shared subject counts for most, then how near the wording is.

On the map

Loading the map… or open it on its own page

Open the map on its own page →