Modular Deep Learning

Sebastian Ruder · 23 Feb 2023 · ruder.io ↗

2 korrents from this piece

Sebastian Ruder did not write this page.

Every claim below was made in this piece, quoted word for word and numbered in the order the piece makes them, so you can read it there rather than take our word for it. The sentence above each quote is our reading of the claim, not their wording. Each quote was checked against a stored copy of the page at build time; where the two differ, the quote is the fact.

Our reading
As models keep scaling up, modularity will become essential to how they are developed.

Their wordsGiven the trend of pre-training larger and larger models, we believe modularity will be crucial.
Our reading
Modular AI development could shift model development away from a few dominant institutions and toward the wider community.

Their wordsBut modularity may also facilitate a shift away from a concentration of model development in a few institutions and to distributing the development of modular components across the community.