What Yoshua Bengio thinks about neural networks

@yoshua-bengio · 12 positions · 1 change of mind

Deep-learning researcher and Turing Award laureate; founder of the Mila institute.

Everything they publish, on ppll ↗

Yoshua Bengio did not write this page.

We collected these quotes from things they published elsewhere, and every quote links to where it was said. They have no account here and have not endorsed this site. Quotes are word for word; the short line under each one is our own restatement, not their wording. Their own site. Is this you? Claim it or ask us to remove it. Or tell us what is wrong here.

6 dated positions, 1998 to 2023, in their own words. Our reading of what Yoshua Bengio has said — not written or endorsed by them.

1. 24 Jun 2023

   Our readingAccess to large-scale generalist AI systems that could be weaponized should be limited, meaning their code and neural network parameters should not be released open-source.

   Their wordsAt the same time, in order to reduce the probability of someone intentionally or unintentionally bringing about a rogue AI, we need to increase governance and we should consider limiting access to the large-scale generalist AI systems that could be weaponized, which would mean that the code and neural net parameters would not be shared in open-source and some of the important engineering tricks to make them work would not be shared either.

   ↗FAQ on Catastrophic AI Risksyoshuabengio.org

2. 9 years earlier

3. 10 Jun 2014

   Our readingDeep generative models have had less impact than discriminative models because of intractable probabilistic computations and difficulty using piecewise linear units.

   Their wordsDeep generative models have had less of an impact, due to the difficulty of approximating many intractable probabilistic computations that arise in maximum likelihood estimation and related strategies, and due to difficulty of leveraging the benefits of piecewise linear units in the generative context.

   ↗Generative Adversarial Networks (with 7 co-authors)arxiv.org 2nd of 2 in this piece

4. 16 years earlier

5. 1 Jan 1998

   Our readingMultilayer neural networks trained with backpropagation are the best example of successful gradient-based learning.

   Their wordsMultilayer neural networks trained with the back-propagation algorithm constitute the best example of a successful gradient based learning technique.

   ↗Gradient-based learning applied to document recognition (with 3 co-authors)doi.org 1st of 4 in this piece

7. 1 Jan 1998

   Our readingWith suitable architecture, gradient-based learning can form decision surfaces that classify high-dimensional patterns such as handwritten characters with minimal preprocessing.

   Their wordsGiven an appropriate network architecture, gradient-based learning algorithms can be used to synthesize a complex decision surface that can classify high-dimensional patterns, such as handwritten characters, with minimal preprocessing.

   ↗Gradient-based learning applied to document recognition (with 3 co-authors)doi.org 2nd of 4 in this piece

9. 1 Jan 1998

   Our readingConvolutional neural networks are specifically designed to handle variability in two-dimensional shapes.

   Their wordsConvolutional neural networks, which are specifically designed to deal with the variability of 2D shapes, are shown to outperform all other techniques.

   ↗Gradient-based learning applied to document recognition (with 3 co-authors)doi.org 3rd of 4 in this piece

11. 1 Jan 1998

    Our readingGraph transformer networks allow multimodule document recognition systems to be trained globally with gradient methods to minimize overall performance.

    Their wordsA new learning paradigm, called graph transformer networks (GTN), allows such multimodule systems to be trained globally using gradient-based methods so as to minimize an overall performance measure.

    ↗Gradient-based learning applied to document recognition (with 3 co-authors)doi.org 4th of 4 in this piece