{"id":13788701,"url":"https://github.com/JiahuiYu/slimmable_networks","last_synced_at":"2025-05-12T03:30:39.034Z","repository":{"id":50265485,"uuid":"158487526","full_name":"JiahuiYu/slimmable_networks","owner":"JiahuiYu","description":"Slimmable Networks, AutoSlim, and Beyond, ICLR 2019, and ICCV 2019","archived":false,"fork":false,"pushed_at":"2023-03-09T08:37:20.000Z","size":329,"stargazers_count":916,"open_issues_count":11,"forks_count":130,"subscribers_count":29,"default_branch":"master","last_synced_at":"2025-04-12T23:30:07.784Z","etag":null,"topics":["adaptive","automated","edge-devices","efficient","neural-architecture-search","on-demand","slimmable-networks"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JiahuiYu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-11-21T03:47:34.000Z","updated_at":"2025-03-24T08:11:22.000Z","dependencies_parsed_at":"2024-01-07T03:52:01.587Z","dependency_job_id":"0e96ca8a-97c1-46ca-bad8-2537f21103d1","html_url":"https://github.com/JiahuiYu/slimmable_networks","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiahuiYu%2Fslimmable_networks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiahuiYu%2Fslimmable_networks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiahuiYu%2Fslimmable_networks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JiahuiYu%2Fslimmable_networks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JiahuiYu","download_url":"https://codeload.github.com/JiahuiYu/slimmable_networks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253667934,"owners_count":21944941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adaptive","automated","edge-devices","efficient","neural-architecture-search","on-demand","slimmable-networks"],"created_at":"2024-08-03T21:00:52.168Z","updated_at":"2025-05-12T03:30:38.707Z","avatar_url":"https://github.com/JiahuiYu.png","language":"Python","funding_links":[],"categories":["3.) Model Compression \u0026 Acceleration"],"sub_categories":["**[Papers]**"],"readme":"# Slimmable Networks\n\n![version](https://img.shields.io/badge/version-v3.0.0--alpha-green.svg?style=plastic)\n![pytorch](https://img.shields.io/badge/pytorch-v1.0.0-green.svg?style=plastic)\n![license](https://img.shields.io/badge/license-CC_BY--NC-green.svg?style=plastic)\n\n\nAn open source framework for slimmable training on tasks of ImageNet classification and COCO detection, which has enabled numerous projects. \u003csup\u003e[1](#snets), [2](#usnets), [3](#autoslim)\u003c/sup\u003e\n\n\u003cstrong  id=\"snets\"\u003e1. Slimmable Neural Networks\u003c/strong\u003e \u003csub\u003e [ICLR 2019 Paper](https://arxiv.org/abs/1812.08928) | [OpenReview](https://openreview.net/forum?id=H1gMCsAqY7) | [Detection](https://github.com/JiahuiYu/slimmable_networks/tree/detection) | [Model Zoo](#slimmable-model-zoo)\u003c/sub\u003e\n\n\u003cimg src=\"https://user-images.githubusercontent.com/22609465/50390872-1b3fb600-0702-11e9-8034-d0f41825d775.png\" width=95%/\u003e\n\nIllustration of slimmable neural networks. The same model can run at different widths (number of active channels), permitting instant and adaptive accuracy-efficiency trade-offs.\n\u003c/div\u003e\n\n\n\u003cstrong id=\"usnets\"\u003e2. Universally Slimmable Networks and Improved Training Techniques\u003c/strong\u003e \u003csub\u003e [ICCV 2019 Paper](https://arxiv.org/abs/1903.05134) | [Model Zoo](#slimmable-model-zoo)\u003c/sub\u003e\n\n\u003cimg src=\"https://user-images.githubusercontent.com/22609465/54562571-45b5ae00-4995-11e9-8984-49e32d07e325.png\" width=60%/\u003e\n\nIllustration of universally slimmable networks. The same model can run at **arbitrary** widths.\n\n\n\u003cstrong id=\"autoslim\"\u003e3. AutoSlim: Towards One-Shot Architecture Search for Channel Numbers\u003c/strong\u003e \u003csub\u003e [NeurIPS 2019 Workshop Paper](https://arxiv.org/abs/1903.11728) | [Model Zoo](#slimmable-model-zoo)\u003c/sub\u003e\n\n\u003cimg src=\"https://user-images.githubusercontent.com/22609465/54886763-93309000-4e59-11e9-963a-c15bf49af3c0.gif\" width=25%/\u003e\u003cimg src=\"https://user-images.githubusercontent.com/22609465/54886764-9592ea00-4e59-11e9-9541-924bbd9ff727.gif\" width=25%/\u003e\u003cimg src=\"https://user-images.githubusercontent.com/22609465/54886766-97f54400-4e59-11e9-81bb-3b262df7c898.gif\" width=25%/\u003e\u003cimg src=\"https://user-images.githubusercontent.com/22609465/54886768-9a579e00-4e59-11e9-9896-25e7eab7e2e0.gif\" width=25%/\u003e\n\nAutoSlimming MobileNet v1, MobileNet v2, MNasNet and ResNet-50: the optimized number of channels under **each** computational budget (FLOPs).\n\n\n## Run\n\n0. Requirements:\n    * python3, pytorch 1.0, torchvision 0.2.1, pyyaml 3.13.\n    * Prepare ImageNet-1k data following pytorch [example](https://github.com/pytorch/examples/tree/master/imagenet).\n1. Training and Testing:\n    * The codebase is a general ImageNet training framework using yaml config under `apps` dir, based on PyTorch.\n    * To test, download pretrained models to `logs` dir and directly run command.\n    * To train, comment `test_only` and `pretrained` in config file. You will need to manage [visible gpus](https://devblogs.nvidia.com/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/) by yourself.\n    * Command: `python train.py app:{apps/***.yml}`. `{apps/***.yml}` is config file. Do not miss `app:` prefix.\n    * Training and testing of MSCOCO benchmarks are released under branch [detection](https://github.com/JiahuiYu/slimmable_networks/tree/detection).\n2. Still have questions?\n    * If you still have questions, please search closed issues first. If the problem is not solved, please open a new.\n\n\n## Slimmable Model Zoo\n\n**[Slimmable Neural Networks](https://arxiv.org/abs/1812.08928)**\n\n\n| Model | Switches (Widths) | Top-1 Err. | FLOPs | Model ID |\n| :--- | :---: | :---: | ---: | :---: |\n| S-MobileNet v1 | 1.00\u003cbr\u003e0.75\u003cbr\u003e0.50\u003cbr\u003e0.25 | 28.5\u003cbr\u003e30.5\u003cbr\u003e35.2\u003cbr\u003e46.9 | 569M\u003cbr\u003e325M\u003cbr\u003e150M\u003cbr\u003e41M | [a6285db](https://github.com/JiahuiYu/slimmable_networks/files/2709079/s_mobilenet_v1_0.25_0.5_0.75_1.0.pt.zip) |\n| S-MobileNet v2 | 1.00\u003cbr\u003e0.75\u003cbr\u003e0.50\u003cbr\u003e0.35 | 29.5\u003cbr\u003e31.1\u003cbr\u003e35.6\u003cbr\u003e40.3 | 301M\u003cbr\u003e209M\u003cbr\u003e97M\u003cbr\u003e59M | [0593ffd](https://github.com/JiahuiYu/slimmable_networks/files/2709080/s_mobilenet_v2_0.35_0.5_0.75_1.0.pt.zip) |\n| S-ShuffleNet | 2.00\u003cbr\u003e1.00\u003cbr\u003e0.50 | 28.6\u003cbr\u003e34.5\u003cbr\u003e42.8 | 524M\u003cbr\u003e138M\u003cbr\u003e38M | [1427f66](https://github.com/JiahuiYu/slimmable_networks/files/2709082/s_shufflenet_0.5_1.0_2.0.pt.zip) |\n| S-ResNet-50 | 1.00\u003cbr\u003e0.75\u003cbr\u003e0.50\u003cbr\u003e0.25 | 24.0\u003cbr\u003e25.1\u003cbr\u003e27.9\u003cbr\u003e35.0 | 4.1G\u003cbr\u003e2.3G\u003cbr\u003e1.1G\u003cbr\u003e278M | [3fca9cc](https://drive.google.com/open?id=1f6q37OkZaz_0GoOAwllHlXNWuKwor2fC) |\n\n\n**[Universally Slimmable Networks and Improved Training Techniques](https://arxiv.org/abs/1903.05134)**\n\n| Model | Model\u0026#160;ID | Spectrum | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |\n| :- | :-: | :- | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |\n| US\u0026#x2011;MobileNet\u0026#160;v1 | [13d5af2](https://github.com/JiahuiYu/slimmable_networks/files/2979952/us_mobilenet_v1_calibrated.pt.zip) | Width\u003cbr\u003eMFLOPs\u003cbr\u003eTop-1 Err. | 1.0\u003cbr\u003e568 \u003cbr\u003e28.2  | 0.975 \u003cbr\u003e543 \u003cbr\u003e28.3  | 0.95 \u003cbr\u003e517 \u003cbr\u003e28.4  | 0.925 \u003cbr\u003e490 \u003cbr\u003e28.7  | 0.9 \u003cbr\u003e466 \u003cbr\u003e28.7  | 0.875 \u003cbr\u003e443 \u003cbr\u003e29.1  | 0.85 \u003cbr\u003e421 \u003cbr\u003e29.4  | 0.825 \u003cbr\u003e389 \u003cbr\u003e29.7  | 0.8 \u003cbr\u003e366 \u003cbr\u003e30.2  | 0.775 \u003cbr\u003e345 \u003cbr\u003e30.3  | 0.75 \u003cbr\u003e325 \u003cbr\u003e30.5  | 0.725 \u003cbr\u003e306 \u003cbr\u003e30.9  | 0.7 \u003cbr\u003e287 \u003cbr\u003e31.2  | 0.675 \u003cbr\u003e267 \u003cbr\u003e31.7  | 0.65 \u003cbr\u003e249 \u003cbr\u003e32.2  | 0.625 \u003cbr\u003e232 \u003cbr\u003e32.5  | 0.6 \u003cbr\u003e217 \u003cbr\u003e33.2  | 0.575 \u003cbr\u003e201 \u003cbr\u003e33.7  | 0.55 \u003cbr\u003e177 \u003cbr\u003e34.4  | 0.525 \u003cbr\u003e162 \u003cbr\u003e35.0  | 0.5 \u003cbr\u003e149 \u003cbr\u003e35.8  | 0.475 \u003cbr\u003e136 \u003cbr\u003e36.5  | 0.45 \u003cbr\u003e124 \u003cbr\u003e37.3  | 0.425 \u003cbr\u003e114 \u003cbr\u003e38.1  | 0.4 \u003cbr\u003e100 \u003cbr\u003e39.0  | 0.375 \u003cbr\u003e89 \u003cbr\u003e40.0  | 0.35 \u003cbr\u003e80 \u003cbr\u003e41.0  | 0.325 \u003cbr\u003e71 \u003cbr\u003e41.9  | 0.3 \u003cbr\u003e64 \u003cbr\u003e42.7  | 0.275 \u003cbr\u003e48 \u003cbr\u003e44.2  | 0.25\u003cbr\u003e41\u003cbr\u003e44.3 |\n| US\u0026#x2011;MobileNet\u0026#160;v2 | [3880cad](https://github.com/JiahuiYu/slimmable_networks/files/2979953/us_mobilenet_v2_calibrated.pt.zip) | Width\u003cbr\u003eMFLOPs\u003cbr\u003eTop-1 Err. | 1.0 \u003cbr\u003e300 \u003cbr\u003e28.5 | 0.975 \u003cbr\u003e299 \u003cbr\u003e28.5 | 0.95 \u003cbr\u003e284 \u003cbr\u003e28.8 | 0.925 \u003cbr\u003e274 \u003cbr\u003e28.9 | 0.9 \u003cbr\u003e269 \u003cbr\u003e29.1 | 0.875 \u003cbr\u003e268 \u003cbr\u003e29.1 | 0.85 \u003cbr\u003e254 \u003cbr\u003e29.4 | 0.825 \u003cbr\u003e235 \u003cbr\u003e29.9 | 0.8 \u003cbr\u003e222 \u003cbr\u003e30.0 | 0.775 \u003cbr\u003e213 \u003cbr\u003e30.2 | 0.75 \u003cbr\u003e209 \u003cbr\u003e30.4 | 0.725 \u003cbr\u003e185 \u003cbr\u003e30.7 | 0.7 \u003cbr\u003e173 \u003cbr\u003e31.1 | 0.675 \u003cbr\u003e165 \u003cbr\u003e31.4 | 0.65 \u003cbr\u003e161 \u003cbr\u003e31.7 | 0.625 \u003cbr\u003e161 \u003cbr\u003e31.7 | 0.6 \u003cbr\u003e151 \u003cbr\u003e32.4 | 0.575 \u003cbr\u003e150 \u003cbr\u003e32.4 | 0.55 \u003cbr\u003e106 \u003cbr\u003e34.4 | 0.525 \u003cbr\u003e100 \u003cbr\u003e34.6 | 0.5 \u003cbr\u003e97 \u003cbr\u003e34.9 | 0.475 \u003cbr\u003e96 \u003cbr\u003e35.1 | 0.45 \u003cbr\u003e88 \u003cbr\u003e35.8 | 0.425 \u003cbr\u003e88 \u003cbr\u003e35.8 | 0.4 \u003cbr\u003e80 \u003cbr\u003e36.6 | 0.375 \u003cbr\u003e80 \u003cbr\u003e36.7 | 0.35\u003cbr\u003e59\u003cbr\u003e37.7 |\n\n\n**[AutoSlim: Towards One-Shot Architecture Search for Channel Numbers](https://arxiv.org/abs/1903.11728)**\n\n| Model | Top-1 Err. | FLOPs | Model ID |\n| :--- | :---: | ---: | :---: |\n| AutoSlim-MobileNet v1 | 27.0\u003cbr\u003e28.5\u003cbr\u003e32.1 | 572M\u003cbr\u003e325M\u003cbr\u003e150M | [9b0b1ab](https://github.com/JiahuiYu/slimmable_networks/files/5166182/autoslim_mobilenet_v1.pt.zip) |\n| AutoSlim-MobileNet v2 | 24.6\u003cbr\u003e25.8\u003cbr\u003e27.0 | 505M\u003cbr\u003e305M\u003cbr\u003e207M | [a24f1f2](https://github.com/JiahuiYu/slimmable_networks/files/5166194/autoslim_mobilenet_v2.pt.zip) |\n| AutoSlim-MNasNet | 24.6\u003cbr\u003e25.4\u003cbr\u003e26.8 | 532M\u003cbr\u003e315M\u003cbr\u003e217M | [31477c9](https://drive.google.com/file/d/1tEuMYc_F-4MUYPua8KAIKjEd7eJDVSx2) |\n| AutoSlim-ResNet-50 | 24.0\u003cbr\u003e24.4\u003cbr\u003e26.0\u003cbr\u003e27.8 | 3.0G\u003cbr\u003e2.0G\u003cbr\u003e1.0G\u003cbr\u003e570M | [f95f419](https://drive.google.com/file/d/1WOOu6frdfGo1_nyHdaMpRILPATtzVAMT) |\n\n\n## Technical Details\n\nImplementing slimmable networks and slimmable training is straightforward:\n  * Switchable batchnorm and slimmable layers are implemented in [`models/slimmable_ops`](/models/slimmable_ops.py).\n  * Slimmable training is implemented in [these lines](https://github.com/JiahuiYu/slimmable_networks/blob/aeb10c9f437208603145e073ee730f0d7dbfa80f/train.py#L281-L289) in [`train.py`](/train.py).\n\n\n## License\n\nCC 4.0 Attribution-NonCommercial International\n\nThe software is for educaitonal and academic research purpose only.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJiahuiYu%2Fslimmable_networks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJiahuiYu%2Fslimmable_networks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJiahuiYu%2Fslimmable_networks/lists"}