korrents

pruning

Cutting connections that are not doing the work — in a trained network, and in a developing brain.

What people on korrents have said about pruning, newest first — 12 positions from 4 people.

FilterEveryone, all time
Who Everyone JFJonathan Frankle 5 MCMichael Carbin 5 CHCasey Handmer 1 TWTommy Wood 1
When All time Past 1 yearPast 3 yearsPast 5 years
  1. TW

    Tommy Wood quoted

    Our readingNeuroplasticity is as much the removal of connections as the growth of new ones, so pruning is a form of learning rather than a form of decay.

    Their wordsA lot of people don't realize that particularly brain development, but then also the refining of functions in the brain through neuroplasticity includes the pruning or removal of signapses. Often we think about it's all about growth and new connections, but actually it also involves the refining or removal of of other connections that aren't needed.

    ↗Accelerate Learning & Increase Cognitive Capacity | Dr. Tommy Woodyoutube.com 1st of 26 in this recording

    Interesting
  2. 3 years earlier
  3. CH

    Casey Handmer quoted

    Our readingBatteries will replace transmission as the cheapest way to keep the lights on, and the grid will shrink rather than grow.

    Their wordswe will see batteries aggressively displace transmission as the most cost-effective way to ensure continuity of electricity supply, leading to an eventual pruning of the grid and drastic reduction in the average distance that power travels between production and consumption, even as the average time that power is stored increases.

    ↗Radical Energy Abundancecaseyhandmer.wordpress.com 4th of 7 in this piece

    energybatteries

    Interesting
  4. 6 years earlier
  5. JF

    Jonathan Frankle quoted

    Our readingPruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free.

    Their wordsNeural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 1st of 10 in this piece

    neural networks

    Interesting
  7. MC

    Michael Carbin quoted

    Our readingPruning cuts the parameters of a trained network by over 90% without costing accuracy, so inference gets smaller and faster for free.

    Their wordsNeural network pruning techniques can reduce the parameter counts of trained networks by over 90%, decreasing storage requirements and improving computational performance of inference without compromising accuracy.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 2nd of 10 in this piece

    neural networks

    Interesting
  9. JF

    Jonathan Frankle quoted

    Our readingThe saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start.

    Their wordsHowever, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 3rd of 10 in this piece

    Interesting
  11. MC

    Michael Carbin quoted

    Our readingThe saving pruning offers is only ever at inference, because the sparse architectures it produces are hard to train from the start.

    Their wordsHowever, contemporary experience is that the sparse architectures produced by pruning are difficult to train from the start, which would similarly improve training performance.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 4th of 10 in this piece

    Interesting
  13. JF

    Jonathan Frankle quoted

    Our readingPruning does not merely shrink a network; it uncovers subnetworks whose original initialization was what made them trainable at all.

    Their wordsWe find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 5th of 10 in this piece

    Interesting
  15. MC

    Michael Carbin quoted

    Our readingPruning does not merely shrink a network; it uncovers subnetworks whose original initialization was what made them trainable at all.

    Their wordsWe find that a standard pruning technique naturally uncovers subnetworks whose initializations made them capable of training effectively.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 6th of 10 in this piece

    Interesting
  17. JF

    Jonathan Frankle quoted

    Our readingA winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work.

    Their wordsThe winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 7th of 10 in this piece

    Interesting
  19. MC

    Michael Carbin quoted

    Our readingA winning ticket wins on its initial weights: the connections it keeps started at values that happen to make training work.

    Their wordsThe winning tickets we find have won the initialization lottery: their connections have initial weights that make training particularly effective.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 8th of 10 in this piece

    Interesting
  21. JF

    Jonathan Frankle quoted

    Our readingA small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it.

    Their wordsAbove this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 9th of 10 in this piece

    Interesting
  23. MC

    Michael Carbin quoted

    Our readingA small enough winning ticket learns faster than the network it was cut out of, and ends up more accurate than it.

    Their wordsAbove this size, the winning tickets that we find learn faster than the original network and reach higher test accuracy.

    ↗Frankle & Carbin, "The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks" (arXiv)arxiv.org 10th of 10 in this piece

    Interesting