{"id":18886237,"url":"https://github.com/eric-mingjie/network-slimming","last_synced_at":"2025-04-13T07:48:40.440Z","repository":{"id":39381917,"uuid":"139824749","full_name":"Eric-mingjie/network-slimming","owner":"Eric-mingjie","description":"Network Slimming (Pytorch) (ICCV 2017)","archived":false,"fork":false,"pushed_at":"2020-11-06T07:25:09.000Z","size":24,"stargazers_count":917,"open_issues_count":61,"forks_count":212,"subscribers_count":13,"default_branch":"master","last_synced_at":"2025-04-13T07:48:37.755Z","etag":null,"topics":["channel-pruning","convolutional-neural-networks","deep-learning","pytorch","sparsity"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Eric-mingjie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-07-05T09:14:05.000Z","updated_at":"2025-04-03T15:17:42.000Z","dependencies_parsed_at":"2022-07-12T19:30:39.086Z","dependency_job_id":null,"html_url":"https://github.com/Eric-mingjie/network-slimming","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eric-mingjie%2Fnetwork-slimming","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eric-mingjie%2Fnetwork-slimming/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eric-mingjie%2Fnetwork-slimming/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Eric-mingjie%2Fnetwork-slimming/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Eric-mingjie","download_url":"https://codeload.github.com/Eric-mingjie/network-slimming/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248681494,"owners_count":21144700,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["channel-pruning","convolutional-neural-networks","deep-learning","pytorch","sparsity"],"created_at":"2024-11-08T07:26:01.505Z","updated_at":"2025-04-13T07:48:40.409Z","avatar_url":"https://github.com/Eric-mingjie.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Network Slimming (Pytorch)\n\nThis repository contains an official pytorch implementation for the following paper  \n[Learning Efficient Convolutional Networks Through Network Slimming](http://openaccess.thecvf.com/content_iccv_2017/html/Liu_Learning_Efficient_Convolutional_ICCV_2017_paper.html) (ICCV 2017).  \n[Zhuang Liu](https://liuzhuang13.github.io/), [Jianguo Li](https://sites.google.com/site/leeplus/), [Zhiqiang Shen](http://zhiqiangshen.com/), [Gao Huang](http://www.cs.cornell.edu/~gaohuang/), [Shoumeng Yan](https://scholar.google.com/citations?user=f0BtDUQAAAAJ\u0026hl=en), [Changshui Zhang](http://bigeye.au.tsinghua.edu.cn/english/Introduction.html).  \n\nOriginal implementation: [slimming](https://github.com/liuzhuang13/slimming) in Torch.    \nThe code is based on [pytorch-slimming](https://github.com/foolwood/pytorch-slimming). We add support for ResNet and DenseNet.  \n\nCitation:\n```\n@InProceedings{Liu_2017_ICCV,\n    author = {Liu, Zhuang and Li, Jianguo and Shen, Zhiqiang and Huang, Gao and Yan, Shoumeng and Zhang, Changshui},\n    title = {Learning Efficient Convolutional Networks Through Network Slimming},\n    booktitle = {The IEEE International Conference on Computer Vision (ICCV)},\n    month = {Oct},\n    year = {2017}\n}\n```\n\n\n## Dependencies\ntorch v0.3.1, torchvision v0.2.0\n\n## Channel Selection Layer\nWe introduce `channel selection` layer to help the  pruning of ResNet and DenseNet. This layer is easy to implement. It stores a parameter `indexes` which is initialized to an all-1 vector. During pruning, it will set some places to 0 which correspond to the pruned channels.\n\n## Baseline \n\nThe `dataset` argument specifies which dataset to use: `cifar10` or `cifar100`. The `arch` argument specifies the architecture to use: `vgg`,`resnet` or\n`densenet`. The depth is chosen to be the same as the networks used in the paper.\n```shell\npython main.py --dataset cifar10 --arch vgg --depth 19\npython main.py --dataset cifar10 --arch resnet --depth 164\npython main.py --dataset cifar10 --arch densenet --depth 40\n```\n\n## Train with Sparsity\n\n```shell\npython main.py -sr --s 0.0001 --dataset cifar10 --arch vgg --depth 19\npython main.py -sr --s 0.00001 --dataset cifar10 --arch resnet --depth 164\npython main.py -sr --s 0.00001 --dataset cifar10 --arch densenet --depth 40\n```\n\n## Prune\n\n```shell\npython vggprune.py --dataset cifar10 --depth 19 --percent 0.7 --model [PATH TO THE MODEL] --save [DIRECTORY TO STORE RESULT]\npython resprune.py --dataset cifar10 --depth 164 --percent 0.4 --model [PATH TO THE MODEL] --save [DIRECTORY TO STORE RESULT]\npython denseprune.py --dataset cifar10 --depth 40 --percent 0.4 --model [PATH TO THE MODEL] --save [DIRECTORY TO STORE RESULT]\n```\nThe pruned model will be named `pruned.pth.tar`.\n\n## Fine-tune\n\n```shell\npython main.py --refine [PATH TO THE PRUNED MODEL] --dataset cifar10 --arch vgg --depth 19 --epochs 160\n```\n\n## Results\n\nThe results are fairly close to the original paper, whose results are produced by Torch. Note that due to different random seeds, there might be up to ~0.5%/1.5% fluctation on CIFAR-10/100 datasets in different runs, according to our experiences.\n### CIFAR10\n|  CIFAR10-Vgg  | Baseline |  Sparsity (1e-4) | Prune (70%) | Fine-tune-160(70%) |\n| :---------------: | :------: | :--------------------------: | :-----------------: | :-------------------: |\n| Top1 Accuracy (%) |  93.77   |            93.30            |        32.54        |         93.78         |\n|    Parameters     |  20.04M  |            20.04M            |        2.25M        |         2.25M         |\n\n|  CIFAR10-Resnet-164  | Baseline |    Sparsity (1e-5) | Prune(40%) | Fine-tune-160(40%) |   Prune(60%)     |  Fine-tune-160(60%)       |\n| :---------------: | :------: | :--------------------------: | :-----------------: | :-------------------: |  :----------------:| :--------------------:|\n| Top1 Accuracy (%) |  94.75   |            94.76             |        94.58       |         95.05         |      47.73       |     93.81     |\n|    Parameters     |  1.71M  |             1.73M            |        1.45M        |         1.45M         |      1.12M          |   1.12M           |\n\n|  CIFAR10-Densenet-40  | Baseline |  Sparsity (1e-5) | Prune (40%) | Fine-tune-160(40%) |       Prune(60%)   | Fine-tune-160(60%) |\n| :---------------: | :------: | :--------------------------: | :-----------------: | :-------------------: | :--------------------: | :-----------------:|\n| Top1 Accuracy (%) |  94.11   |           94.17             |        94.16       |         94.32         |      89.46       |     94.22     |\n|    Parameters     |  1.07M  |            1.07M            |        0.69M       |         0.69M         |       0.49M      |    0.49M     |\n\n### CIFAR100\n|  CIFAR100-Vgg  | Baseline |   Sparsity (1e-4) | Prune (50%) | Fine-tune-160(50%) |\n| :---------------: | :------: | :--------------------------: | :-----------------: | :-------------------: |\n| Top1 Accuracy (%) |   72.12   |            72.05             |         5.31        |         73.32         |\n|    Parameters     |  20.04M  |            20.04M            |        4.93M        |         4.93M         |\n\n|  CIFAR100-Resnet-164  | Baseline |   Sparsity (1e-5) | Prune (40%) | Fine-tune-160(40%) |    Prune(60%)  | Fine-tune-160(60%) |\n| :---------------: | :------: | :--------------------------: | :-----------------: | :-------------------: |:--------------------: | :-----------------:|\n| Top1 Accuracy (%) |  76.79   |            76.87             |        48.0        |         77.36        |  ---       |     ---     |\n|    Parameters     |  1.73M  |            1.73M            |        1.49M        |         1.49M         |---       |     ---     |\n\nNote: For results of pruning 60% of the channels for resnet164-cifar100, in this implementation, sometimes some layers are all pruned and there would be error. However, we also provide a [mask implementation](https://github.com/Eric-mingjie/network-slimming/tree/master/mask-impl) where we apply a mask to the scaling factor in BN layer. For mask implementaion, when pruning 60% of the channels in resnet164-cifar100, we can also train the pruned network.\n\n|  CIFAR100-Densenet-40  | Baseline |    Sparsity (1e-5) | Prune (40%) | Fine-tune-160(40%) | Prune(60%)  | Fine-tune-160(60%) |\n| :---------------: | :------: | :--------------------------: | :-----------------: | :-------------------: |:--------------------: | :-----------------:|\n| Top1 Accuracy (%) |  73.27   |          73.29            |        67.67        |         73.76         |   19.18       |     73.19     |\n|    Parameters     |  1.10M  |            1.10M            |        0.71M        |         0.71M         |  0.50M       |     0.50M    |\n\n## Contact\nsunmj15 at gmail.com \nliuzhuangthu at gmail.com  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feric-mingjie%2Fnetwork-slimming","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feric-mingjie%2Fnetwork-slimming","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feric-mingjie%2Fnetwork-slimming/lists"}