Jimmy Ba
Associate professor at the University of Toronto, and co-author of the Adam optimizer.
Everything they publish, on ppll ↗
Jimmy Ba did not write this page.
We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.
-
Their wordsWe introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions, based on adaptive estimates of lower-order moments.
↗Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv)arxiv.org 2nd of 8 in this piece
-
Their wordsThe method is straightforward to implement, is computationally efficient, has little memory requirements, is invariant to diagonal rescaling of the gradients, and is well suited for problems that are large in terms of data and/or parameters.
↗Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv)arxiv.org 4th of 8 in this piece
-
Their wordsThe method is also appropriate for non-stationary objectives and problems with very noisy and/or sparse gradients.
↗Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv)arxiv.org 6th of 8 in this piece
-
Their wordsThe hyper-parameters have intuitive interpretations and typically require little tuning.
↗Kingma & Ba, "Adam: A Method for Stochastic Optimization" (arXiv)arxiv.org 8th of 8 in this piece