{"id":17424446,"url":"https://github.com/kumasento/gconv-prune","last_synced_at":"2026-03-03T21:02:04.849Z","repository":{"id":80944407,"uuid":"204706641","full_name":"kumasento/gconv-prune","owner":"kumasento","description":"Code repository for paper \"Efficient Structured Pruning and Architecture Searching for Group Convolution\" https://arxiv.org/abs/1811.09341","archived":false,"fork":false,"pushed_at":"2021-10-04T16:14:02.000Z","size":2405,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-16T01:13:10.325Z","etag":null,"topics":["deep-learning","group-convolution","iccv2019","pruning","pytorch","structured-pruning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kumasento.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-27T13:13:39.000Z","updated_at":"2025-01-21T06:47:37.000Z","dependencies_parsed_at":null,"dependency_job_id":"af9cc142-6b35-48d1-b0f3-5c8e4299745a","html_url":"https://github.com/kumasento/gconv-prune","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/kumasento/gconv-prune","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumasento%2Fgconv-prune","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumasento%2Fgconv-prune/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumasento%2Fgconv-prune/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumasento%2Fgconv-prune/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kumasento","download_url":"https://codeload.github.com/kumasento/gconv-prune/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kumasento%2Fgconv-prune/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30060632,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-03T18:21:05.932Z","status":"ssl_error","status_checked_at":"2026-03-03T18:20:59.341Z","response_time":61,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","group-convolution","iccv2019","pruning","pytorch","structured-pruning"],"created_at":"2024-10-17T05:42:04.903Z","updated_at":"2026-03-03T21:02:04.844Z","avatar_url":"https://github.com/kumasento.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# `gconv-prune`: structural pruning based on group convolution\n\n[![Python Package using Conda](https://github.com/kumasento/gconv-prune/actions/workflows/python-package-conda.yml/badge.svg)](https://github.com/kumasento/gconv-prune/actions/workflows/python-package-conda.yml)\n[![codecov](https://codecov.io/gh/kumasento/gconv-prune/branch/main/graph/badge.svg?token=8WF5R96I1L)](https://codecov.io/gh/kumasento/gconv-prune)\n[![Total alerts](https://img.shields.io/lgtm/alerts/g/kumasento/gconv-prune.svg?logo=lgtm\u0026logoWidth=18)](https://lgtm.com/projects/g/kumasento/gconv-prune/alerts/)\n[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/kumasento/gconv-prune.svg?logo=lgtm\u0026logoWidth=18)](https://lgtm.com/projects/g/kumasento/gconv-prune/context:python)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n\nThis is the codebase for paper [Efficient Structured Pruning and Architecture Searching for Group Convolution](https://arxiv.org/abs/1811.09341v4) that has appeared at the [ICCV'19 NEUARCH workshop](https://neuralarchitects.org/).\n\n![Overall workflow](https://user-images.githubusercontent.com/8768839/135343849-eca7f27a-1181-4381-af4f-b8e3b6fdfe5b.png)\n\n![Pruning objective](https://user-images.githubusercontent.com/8768839/135343917-3c03bba9-2cfe-4774-9c29-038d9c34f288.png)\n\n\n- [`gconv-prune`: structural pruning based on group convolution](#gconv-prune-structural-pruning-based-on-group-convolution)\n  - [Installation](#installation)\n  - [Usage](#usage)\n    - [Training baseline models](#training-baseline-models)\n    - [Pruning by a fixed group configuration](#pruning-by-a-fixed-group-configuration)\n      - [ImageNet Evaluation Results](#imagenet-evaluation-results)\n      - [CondenseNet-86 Evaluation Results](#condensenet-86-evaluation-results)\n    - [Searching for a group configuration](#searching-for-a-group-configuration)\n\n## Installation\n\nWe use Anaconda3 as the package manager. Please run the following script to initialize our required environmental packages.\n\n```shell\nconda env create -f environment.yml\n```\n\nThen use `pip` to install the core package `gumi`, which implements the algorithm mentioned in our paper.\n\n```shell\npython -m pip install -e .\n```\n\n## Usage\n\nThis project is still in its early stage. Scripts are listed under `evaluation/iccv19` mainly.\n\n### Training baseline models\n\nBaseline models serve as starting points for further pruning. They can be treated the same as pre-trained models.\n\nPlease move forward to this [script](evaluation/iccv19/train_baselines.sh) to find commands for training baseline CIFAR-10/100 models used in the paper.\n\nImageNet baseline models are directly download from PyTorch official [releases](https://pytorch.org/docs/stable/torchvision/models.html).\n\n### Pruning by a fixed group configuration\n\nThe script for pruning a model by a fixed group configuration is [prune.py](evaluation/iccv19/prune.py).\n\nWe basically use this _fixed_ approach to gather our ImageNet training results.\n\n#### ImageNet Evaluation Results\n\nTo get results for Table 2., please check out the following commands.\n\nNote that `--perm GRPS` basically sets the optimisation algorithm to use (mentioned in our paper), and `--num-sort-iters` is the `N_S` hyperparameter as mentioned in the paper.\n\n```shell\n# Training ResNet-34 A\npython prune.py --group-cfg config/resnet34_A.json --perm GRPS --num-sort-iters 10 -a resnet34 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --gpu-id $GPU_ID --epochs 30 --lr 1e-2 --schedule 10 20 --checkpoint $RESULTS/resnet34_A --pretrained\n\n# Training ResNet-34 B\npython prune.py --group-cfg config/resnet34_B.json --perm GRPS --num-sort-iters 10 -a resnet34 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --gpu-id $GPU_ID --epochs 30 --lr 1e-2 --schedule 10 20 --checkpoint $RESULTS/resnet34_B --pretrained\n\n# Training ResNet-50\npython prune.py -g 2 --perm GRPS --num-sort-iters 10 -a resnet50 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --gpu-id $GPU_ID --epochs 30 --lr 1e-2 --schedule 10 20 --checkpoint $RESULTS/resnet50_G2 --pretrained\n\n# Training ResNet-101\npython prune.py -g 2 --perm GRPS --num-sort-iters 10 -a resnet101 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --gpu-id $GPU_ID --epochs 30 --lr 1e-2 --schedule 10 20 --checkpoint $RESULTS/resnet101_G2 --pretrained\n```\n\nHere are links to download these pruned models:\n\n- [ResNet-34 (A)](https://gumi-models.s3.amazonaws.com/resnet34_A.tar.gz)\n- [ResNet-34 (B)](https://gumi-models.s3.amazonaws.com/resnet34_B.tar.gz)\n- [ResNet-50 (G=2)](https://gumi-models.s3.amazonaws.com/resnet50_G_2.tar.gz)\n- [ResNet-101 (G=2)](https://gumi-models.s3.amazonaws.com/resnet101_G_2.tar.gz)\n\nTo evaluate these models, you can reuse the `prune.py` script to only validate, instead of pruning, these checkpoint files.\n\n```shell\n# ResNet-34 (A)\ntar xvf resnet34_A.tar.gz\npython prune.py --group-cfg config/resnet34_A.json -a resnet34 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --resume resnet34_A/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --gpu-id $GPU_ID\n\n# ResNet-34 (B)\ntar xvf resnet34_B.tar.gz\npython prune.py --group-cfg config/resnet34_B.json -a resnet34 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --resume resnet34_B/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --gpu-id $GPU_ID\n\n# ResNet-50 (G=2)\ntar xvf resnet50_G_2.tar.gz\npython prune.py -g 2 -a resnet50 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --resume resnet50_G_2/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --gpu-id $GPU_ID\n\n# ResNet-101 (G=2)\ntar xvf resnet101_G_2.tar.gz\npython prune.py -g 2 -a resnet101 -d imagenet --dataset-dir $DATASET_DIR/ILSVRC2012 --resume resnet101_G_2/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --gpu-id $GPU_ID\n```\n\nRunning the script above will first load the model checkpoint and run validation, and then validate again after updating the model for exportation, and finally produce the size of the GConv-based model and the validation accuracy.\n\n#### CondenseNet-86 Evaluation Results\n\nBesides those ResNet variants, we apply our method on CondenseNet-86 as well by taking a fixed group configuration. We have pruned pre-trained CondenseNet-86 into `G=4` (compared with the original CondenseNet paper) and `G=2` (the one labelled by 50% budget).\n\nTo produce these models, first you need to download our pre-trained checkpoints ([CIFAR-10](https://gumi-models.s3.amazonaws.com/densenet86_cifar10.pth.tar) and [CIFAR-100](https://gumi-models.s3.amazonaws.com/densenet86_cifar100.pth.tar)), and then run the `prune.py` script with specification on `-g`.\n\n```shell\n# Download CIFAR-10 checkpoint\nwget https://gumi-models.s3.amazonaws.com/densenet86_cifar10.pth.tar\n# G = 2\npython prune.py -g 2 --perm GRPS --num-sort-iters 10 -a condensenet86 -d cifar10 --epochs 100 --schedule 50 75 --lr 5e-3 --wd 1e-4 --resume densenet86_cifar10.pth.tar --checkpoint $RESULT_DIR\n# G = 4\npython prune.py -g 4 --perm GRPS --num-sort-iters 10 -a condensenet86 -d cifar10 --epochs 100 --schedule 50 75 --lr 5e-3 --wd 1e-4 --resume densenet86_cifar10.pth.tar --checkpoint $RESULT_DIR\n\n# Download CIFAR-100 checkpoint\nwget https://gumi-models.s3.amazonaws.com/densenet86_cifar100.pth.tar\n# G = 2\npython prune.py -g 2 --perm GRPS --num-sort-iters 10 -a condensenet86 -d cifar100 --epochs 100 --schedule 50 75 --lr 5e-3 --wd 1e-4 --resume densenet86_cifar100.pth.tar --checkpoint $RESULT_DIR\n# G = 4\npython prune.py -g 4 --perm GRPS --num-sort-iters 10 -a condensenet86 -d cifar100 --epochs 100 --schedule 50 75 --lr 5e-3 --wd 1e-4 --resume densenet86_cifar100.pth.tar --checkpoint $RESULT_DIR\n```\n\nPruned models can be downloaded from here:\n\n| Model          | G   | Dataset   | Download                                                                            |\n| -------------- | --- | --------- | ----------------------------------------------------------------------------------- |\n| CondenseNet-86 | 2   | CIFAR-10  | [link](https://gumi-models.s3.amazonaws.com/condensenet86_cifar10_G_2_GRPS.tar.gz)  |\n| CondenseNet-86 | 4   | CIFAR-10  | [link](https://gumi-models.s3.amazonaws.com/condensenet86_cifar10_G_4_GRPS.tar.gz)  |\n| CondenseNet-86 | 2   | CIFAR-100 | [link](https://gumi-models.s3.amazonaws.com/condensenet86_cifar100_G_2_GRPS.tar.gz) |\n| CondenseNet-86 | 4   | CIFAR-100 | [link](https://gumi-models.s3.amazonaws.com/condensenet86_cifar100_G_4_GRPS.tar.gz) |\n\nTo evaluate:\n\n```shell\n# CIFAR-10 G=2\nwget https://gumi-models.s3.amazonaws.com/condensenet86_cifar10_G_2_GRPS.tar.gz\ntar xvf condensenet86_cifar10_G_2_GRPS.tar.gz\npython prune.py -g 2 -a condensenet86 -d cifar10 --resume condensenet86_cifar10_G_2/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --condensenet --gpu-id $GPU_ID\n# CIFAR-10 G=4\nwget https://gumi-models.s3.amazonaws.com/condensenet86_cifar10_G_4_GRPS.tar.gz\ntar xvf condensenet86_cifar10_G_4_GRPS.tar.gz\npython prune.py -g 4 -a condensenet86 -d cifar10 --resume condensenet86_cifar10_G_4/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --condensenet --gpu-id $GPU_ID\n# CIFAR-10 G=2\nwget https://gumi-models.s3.amazonaws.com/condensenet86_cifar100_G_2_GRPS.tar.gz\ntar xvf condensenet86_cifar100_G_2_GRPS.tar.gz\npython prune.py -g 2 -a condensenet86 -d cifar100 --resume condensenet86_cifar100_G_2/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --condensenet --gpu-id $GPU_ID\n# CIFAR-10 G=2\nwget https://gumi-models.s3.amazonaws.com/condensenet86_cifar100_G_4_GRPS.tar.gz\ntar xvf condensenet86_cifar100_G_4_GRPS.tar.gz\npython prune.py -g 4 -a condensenet86 -d cifar100 --resume condensenet86_cifar100_G_4/model_best.pth.tar --skip-fine-tune --apply-mask --keep-mask --condensenet --gpu-id $GPU_ID\n```\n\n### Searching for a group configuration\n\nThe other major benefit of using our method is its ability to search efficiently for a group configuration under given constraints.\n\n`gopt.py` under `evaluation/iccv19` provides the corresponding functionality.\n\n```shell\npython gopt.py \\\n    -a presnet164 \\\n    -d cifar100 \\\n    --resume ${PATH_TO_PRETRAINED_MODEL} \\\n    --group-cfg group_cfg.json \\\n    --gpu-id ${GPU_ID} \\\n    --max-num-params ${MAX_NUM_PARAMS} \\\n    --strategy MAX_COST \\\n    --min-factor ${MIN_FACTOR}\n```\n\nAs shown above, when using this script, you need to specify the model architecture, the dataset, and the path to the resumable model file pre-trained on the given dataset. Meanwhile, workload constraints should be provided through `--max-num-params` measured in MB.\n\n`--strategy` and `--min-factor` are hyperparameters that specify how the optimisation should perform.\nStrategy `MAX_COST` implements the _local search_ method mentioned in the paper (Section 3.3).\n`--min-factor` indicates at least how many channels we want to preserve in each layer. We need this number because the estimated cost could be too aggressive.\n\nThis [script](evaluation/iccv19/run_gopt.sh) records the commands to run.\n\nAnd this [script](evaluation/iccv19/run_gopt_comparison_preresnet164_with_network_slimming.sh)\ncompares the performance between our approach and the result from Network Slimming.\n\n```shell\n# Run gopt with 1.44M number of parameter constraints for CIFAR-100 on GPU 0.\n# Each group should have at least 8 channels.\n./run_gopt_comparison_preresnet164_with_network_slimming.sh 0 8 1.44 100\n```\n\nExplored models and configurations can be found at [here](https://gumi-models.s3.amazonaws.com/preresnet164_gopt.tar.gz).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkumasento%2Fgconv-prune","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkumasento%2Fgconv-prune","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkumasento%2Fgconv-prune/lists"}