https://github.com/jack-willturner/batchnorm-pruning
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers https://arxiv.org/abs/1802.00124
https://github.com/jack-willturner/batchnorm-pruning
batchnorm deep-learning lasso pruning sgd
Last synced: about 1 month ago
JSON representation
Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers https://arxiv.org/abs/1802.00124
- Host: GitHub
- URL: https://github.com/jack-willturner/batchnorm-pruning
- Owner: jack-willturner
- Created: 2018-02-13T16:56:47.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-11-14T11:41:10.000Z (almost 7 years ago)
- Last Synced: 2025-04-03T20:34:58.837Z (6 months ago)
- Topics: batchnorm, deep-learning, lasso, pruning, sgd
- Language: Python
- Homepage:
- Size: 41.6 MB
- Stars: 71
- Watchers: 7
- Forks: 16
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# [Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers](https://arxiv.org/abs/1802.00124)
Tensorflow implementation from original author [here](https://github.com/bobye/batchnorm_prune).
A PyTorch implementation of [this paper](https://arxiv.org/abs/1802.00124).
To do list:
- [ ] Extend to MobileNet and VGG
- [ ] Fix MAC op calculation for strided convolution
- [ ] Include training scheme from paper
## Usage
I haven't included any code for transfer learning/ using pretrained models, so everything here must be done from scratch.
You will have to rewrite your models to use my extended version of batch normalization, so any occurences of `nn.BatchNorm2d`
should be replaced with `bn.BatchNorm2dEx`. I have included a few examples in the `models` folder. Note that in the forward pass
you need to provide the `weight` from the last convolution to the batchnorm (e.g. `out = self.bn1(self.conv1(x), self.conv1.weight)`.
I will add command line support for hyperparameters soon, but for now they will have to be altered in the `main` script itself. Currently the default is set to train ResNet-18; this can easily be swapped out for another model.
```bash
python main.py
```
## Results on CIFAR-10
| Model | Size | MAC ops | Inf. time | Accuracy |
|----------------------|-------|---------|-----------|----------|
| ResNet-18 | | | | |
| ResNet-18-Compressed | | | | |
| VGG-16 | | | | |
| VGG-16-Compressed | | | | |
| MobileNet | | | | |
| MobileNet-Compressed | | | | |
## Citing
Now accepted to ICLR 2018, will update bibtex soon:
```
@article{ye2018rethinking,
title={Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers},
author={Ye, Jianbo and Lu, Xin and Lin, Zhe and Wang, James Z},
journal={arXiv preprint arXiv:1802.00124},
year={2018}
}
```