{"id":20936143,"url":"https://github.com/he-y/filter-pruning-geometric-median","last_synced_at":"2025-04-04T21:08:46.155Z","repository":{"id":49314242,"uuid":"177763869","full_name":"he-y/filter-pruning-geometric-median","owner":"he-y","description":"Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration (CVPR 2019 Oral)","archived":false,"fork":false,"pushed_at":"2023-08-31T06:50:25.000Z","size":2274,"stargazers_count":606,"open_issues_count":12,"forks_count":113,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-03-28T20:08:15.014Z","etag":null,"topics":["model-compression","pruning","pytorch"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/1811.00250","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/he-y.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-03-26T10:17:15.000Z","updated_at":"2025-03-15T03:15:53.000Z","dependencies_parsed_at":"2022-07-31T02:18:27.122Z","dependency_job_id":"9eb51622-5802-407a-9081-d0a3749471e3","html_url":"https://github.com/he-y/filter-pruning-geometric-median","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/he-y%2Ffilter-pruning-geometric-median","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/he-y%2Ffilter-pruning-geometric-median/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/he-y%2Ffilter-pruning-geometric-median/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/he-y%2Ffilter-pruning-geometric-median/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/he-y","download_url":"https://codeload.github.com/he-y/filter-pruning-geometric-median/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247249526,"owners_count":20908212,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["model-compression","pruning","pytorch"],"created_at":"2024-11-18T22:18:04.356Z","updated_at":"2025-04-04T21:08:46.135Z","avatar_url":"https://github.com/he-y.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration\n\n![i1](https://github.com/he-y/filter-pruning-geometric-median/blob/master/functions/explain.png)\n\n**[CVPR 2019 Oral](http://openaccess.thecvf.com/content_CVPR_2019/html/He_Filter_Pruning_via_Geometric_Median_for_Deep_Convolutional_Neural_Networks_CVPR_2019_paper.html)**. \n\nImplementation with PyTorch. This implementation is based on [soft-filter-pruning](https://github.com/he-y/soft-filter-pruning).\n\n\n## What's New\nFPGM has been re-implemented in [Pytorch](https://github.com/pytorch/pytorch) and [NNI](https://github.com/microsoft/nni).\n\n### Usage in [Pytorch](https://github.com/pytorch/pytorch)\n\n```\nfrom torch.ao.sparsity.pruning._experimental.pruner import FPGM_pruner\n\n# set network-level sparsity: all layers have a sparsity level of 30%\npruner = FPGMPruner(sparsity_level = 0.3)\n\n# set layer-level sparsity: sparsity_level of conv2d1 = 30%, sparsity_level of conv2d2 = 50%\nconfig = [\n    {\"tensor_fqn\": \"conv2d1.weight\"},\n    {\"tensor_fqn\": \"conv2d2.weight\", \"sparsity_level\": 0.5}\n]\n\npruner.prepare(model, config)\npruner.enable_mask_update = True\npruner.step()\n\n# Get real pruned models (without zeros)\npruned_model = pruner.prune()\n```\n\nSee source code [here](https://github.com/pytorch/pytorch/blob/main/torch/ao/pruning/_experimental/pruner/FPGM_pruner.py) and official test code [here](https://github.com/pytorch/pytorch/blob/main/test/ao/sparsity/test_structured_sparsifier.py#L921-L1041).\n\n### Usage in [NNI](https://github.com/microsoft/nni)\n```\nfrom nni.algorithms.compression.pytorch.pruning import FPGMPruner\nconfig_list = [{\n    'sparsity': 0.5,\n    'op_types': ['Conv2d']\n}]\npruner = FPGMPruner(model, config_list)\npruner.compress()\n```\n\nSee explanation [here](https://nni.readthedocs.io/en/v2.1/Compression/Pruner.html#fpgm-pruner).\n\n\n## Table of Contents\n\n- [Requirements](#requirements)\n- [Models and log files](#models-and-log-files)\n- [Training ResNet on ImageNet](#training-resnet-on-imagenet)\n  - [Usage of Pruning Training](#usage-of-pruning-training)\n  - [Usage of Normal Training](#usage-of-normal-training)\n  - [Inference the pruned model with zeros](#inference-the-pruned-model-with-zeros)\n  - [Inference the pruned model without zeros](#inference-the-pruned-model-without-zeros)\n  - [Scripts to reproduce the results in our paper](#scripts-to-reproduce-the-results-in-our-paper)\n- [Training ResNet on Cifar-10](#training-resnet-on-cifar-10)\n- [Training VGGNet on Cifar-10](#training-vggnet-on-cifar-10)\n- [Notes](#notes)\n  - [Torchvision Version](#torchvision-version)\n  - [Why use 100 epochs for training](#why-use-100-epochs-for-training)\n  - [Process of ImageNet dataset](#process-of-imagenet-dataset)\n  - [FLOPs Calculation](#flops-calculation)\n- [Citation](#citation)\n\n\n## Requirements\n- Python 3.6\n- PyTorch 0.3.1\n- TorchVision 0.3.0\n\n## Models and log files\nThe trained models with log files can be found in [Google Drive](https://drive.google.com/drive/folders/1w_Max8L5ICJZSrlha8UybHfICik-iX95?usp=sharing).\nSpecifically:\n\n[models for pruning ResNet on ImageNet](https://drive.google.com/drive/u/1/folders/1DOYiOZGQxr94rWsEw73ezz9a-0hcNf-2)\n\n[models for pruning ResNet on CIFAR-10](https://drive.google.com/drive/u/1/folders/1YLhcY487U0ZdGiDHzJBJZOJLFYBhrBoD)\n\n[models for pruning VGGNet on CIFAR-10](https://drive.google.com/drive/u/1/folders/1hGnULraEbz8IjSRZx_juzZnvDTDqDdt-)\n\n[models for ablation study](https://drive.google.com/drive/u/1/folders/1PZLOw51n8yvdKO0pzAk_9t6It9Awq6GU)\n\nThe pruned model without zeros, refer to [this issue](https://github.com/he-y/filter-pruning-geometric-median/issues/7).\n\n## Training ResNet on ImageNet\n\n#### Usage of Pruning Training\nWe train each model from scratch by default. If you wish to train the model with pre-trained models, please use the options `--use_pretrain --lr 0.01`. \n\nRun Pruning Training ResNet (depth 152,101,50,34,18) on Imagenet:\n\n```bash\npython pruning_imagenet.py -a resnet152 --save_path ./snapshots/resnet152-rate-0.7 --rate_norm 1 --rate_dist 0.4 --layer_begin 0 --layer_end 462 --layer_inter 3  /path/to/Imagenet2012\n\npython pruning_imagenet.py -a resnet101 --save_path ./snapshots/resnet101-rate-0.7 --rate_norm 1 --rate_dist 0.4 --layer_begin 0 --layer_end 309 --layer_inter 3  /path/to/Imagenet2012\n\npython pruning_imagenet.py -a resnet50  --save_path ./snapshots/resnet50-rate-0.7 --rate_norm 1 --rate_dist 0.4 --layer_begin 0 --layer_end 156 --layer_inter 3  /path/to/Imagenet2012\n\npython pruning_imagenet.py -a resnet34  --save_path ./snapshots/resnet34-rate-0.7 --rate_norm 1 --rate_dist 0.4 --layer_begin 0 --layer_end 105 --layer_inter 3  /path/to/Imagenet2012\n\npython pruning_imagenet.py -a resnet18  --save_path ./snapshots/resnet18-rate-0.7 --rate_norm 1 --rate_dist 0.4 --layer_begin 0 --layer_end 57 --layer_inter 3  /path/to/Imagenet2012\n```\nExplanation:\n \nNote1: `rate_norm = 0.9` means pruning 10% filters by norm-based criterion, `rate_dist = 0.2` means pruning 20% filters by distance-based criterion.\n\nNote2: the `layer_begin` and `layer_end` is the index of the first and last conv layer, `layer_inter` choose the conv layer instead of BN layer. \n\n#### Usage of Normal Training\nRun resnet(100 epochs): \n```bash\npython original_train.py -a resnet50 --save_dir ./snapshots/resnet50-baseline  /path/to/Imagenet2012 --workers 36\n```\n\n#### Inference the pruned model with zeros\n```bash\nsh function/inference_pruned.sh\n```\n#### Inference the pruned model without zeros\nThe pruned model without zeros, refer to [this issue](https://github.com/he-y/filter-pruning-geometric-median/issues/7).\n\n\n#### Scripts to reproduce the results in our paper\nTo train the ImageNet model with / without pruning, see the directory `scripts`.\nFull script is [here](https://github.com/he-y/filter-pruning-geometric-median/tree/master/scripts).\n\n\n## Training ResNet on Cifar-10\n```bash\nsh scripts/pruning_cifar10.sh\n```\nPlease be care of the hyper-parameter [`layer_end`](https://github.com/he-y/filter-pruning-geometric-median/blob/master/scripts/pruning_cifar10.sh#L4-L9) for different layer of ResNet.\n\nReproduce ablation study of Cifar-10:\n```bash\nsh scripts/ablation_pruning_cifar10.sh\n```\n\n\n## Training VGGNet on Cifar-10\nRefer to the directory `VGG_cifar`. \n#### Reproduce previous paper [Pruning Filters for Efficient ConvNets](https://arxiv.org/abs/1608.08710)\n```bash\nsh VGG_cifar/scripts/PFEC_train_prune.sh\n```\nFour function included in the script, including [training baseline](https://github.com/he-y/filter-pruning-geometric-median/blob/master/VGG_cifar/scripts/PFEC_train_prune.sh#L3-L12), [pruning from pretrain](https://github.com/he-y/filter-pruning-geometric-median/blob/master/VGG_cifar/scripts/PFEC_train_prune.sh#L14-L43), [pruning from scratch](https://github.com/he-y/filter-pruning-geometric-median/blob/master/VGG_cifar/scripts/PFEC_train_prune.sh#L45-L54), [finetune the pruend](https://github.com/he-y/filter-pruning-geometric-median/blob/master/VGG_cifar/scripts/PFEC_train_prune.sh#L57-L65)\n\n#### Our method\n```bash\nsh VGG_cifar/scripts/pruning_vgg_my_method.sh\n```\nIncluding [pruning the pretrained](https://github.com/he-y/filter-pruning-geometric-median/blob/master/VGG_cifar/scripts/pruning_vgg_my_method.sh#L52-L61), [pruning the scratch](https://github.com/he-y/filter-pruning-geometric-median/blob/master/VGG_cifar/scripts/pruning_vgg_my_method.sh#L62-L66).\n\n## Notes\n\n#### Torchvision Version\nWe use the torchvision of 0.3.0. If the version of your torchvision is 0.2.0, then the `transforms.RandomResizedCrop` should be `transforms.RandomSizedCrop` and the `transforms.Resize` should be `transforms.Scale`.\n\n#### Why use 100 epochs for training\nThis can improve the accuracy slightly.\n\n#### Process of ImageNet dataset\nWe follow the [Facebook process of ImageNet](https://github.com/facebook/fb.resnet.torch/blob/master/INSTALL.md#download-the-imagenet-dataset).\nTwo subfolders (\"train\" and \"val\") are included in the \"/path/to/ImageNet2012\".\nThe correspding code is [here](https://github.com/he-y/filter-pruning-geometric-median/blob/master/pruning_imagenet.py#L136-L137).\n\n#### FLOPs Calculation\nRefer to the [file](https://github.com/he-y/soft-filter-pruning/blob/master/utils/cifar_resnet_flop.py).\n\n\n\n## Citation\n```\n@inproceedings{he2019filter,\n  title     = {Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration},\n  author    = {He, Yang and Liu, Ping and Wang, Ziwei and Hu, Zhilan and Yang, Yi},\n  booktitle = {Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},\n  year      = {2019}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhe-y%2Ffilter-pruning-geometric-median","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhe-y%2Ffilter-pruning-geometric-median","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhe-y%2Ffilter-pruning-geometric-median/lists"}