Skip to content

Rethinking the Value of Network Pruning (Pytorch) (ICLR 2019)

License

Notifications You must be signed in to change notification settings

lovepan1/rethinking-network-pruning

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

13 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Rethinking the Value of Network Pruning

This repository contains the code for reproducing the results, and trained ImageNet models, in the following paper:

Rethinking the Value of Network Pruning

Zhuang Liu*, Mingjie Sun*, Tinghui Zhou, Gao Huang, Trevor Darrell (* equal contribution).

ICLR 2019. Also Best Paper Award at NIPS 2018 Workshop on Compact Deep Neural Networks with industrial applications.

This paper shows that training the small pruned model from scratch can almost always achieve comparable or higher level of accuracy than the model obtained from the typical "training, pruning and fine-tuning" procedure. Our results suggest:

  1. Training a large, over-parameterized model is not absolutely necessary to obtain an efficient final model.
  2. Learned “important” weights of the large model are not necessarily useful for the small pruned model.
  3. The pruned architecture itself, rather than a set of inherited "important" weights, is what leads to the efficiency benefit in the final model, which suggests that some pruning algorithms could be seen as performing network architecture search.

Citation

@inproceedings{liu2018rethinking,
  title={Rethinking the Value of Network Pruning},
  author={Zhuang Liu and Mingjie Sun and Tinghui Zhou and Gao Huang and Trevor Darrell},
  booktitle={ICLR},
  year={2019}
}

Implementation

We evaluated the following six pruning methods:

  1. L1-norm based channel pruning
  2. ThiNet
  3. Regression based feature reconstruction
  4. Network Slimming
  5. Sparse Structure Selection
  6. Non-structured weight-level pruning

For CIFAR, our code is based on pytorch-classification and network-slimming. For ImageNet, we use the official Pytorch ImageNet training code. The instructions and models are in each subfolder.

Our experiment environment is Python 3.6 & PyTorch 0.3.1.

Contact

sunmj15 at gmail.com
liuzhuangthu at gmail.com

About

Rethinking the Value of Network Pruning (Pytorch) (ICLR 2019)

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Python 100.0%