Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/JiahuiYu/slimmable_networks

Slimmable Networks, AutoSlim, and Beyond, ICLR 2019, and ICCV 2019
https://github.com/JiahuiYu/slimmable_networks

adaptive automated edge-devices efficient neural-architecture-search on-demand slimmable-networks

Last synced: 2 months ago
JSON representation

Slimmable Networks, AutoSlim, and Beyond, ICLR 2019, and ICCV 2019

Host: GitHub
URL: https://github.com/JiahuiYu/slimmable_networks
Owner: JiahuiYu
License: other
Created: 2018-11-21T03:47:34.000Z (almost 6 years ago)
Default Branch: master
Last Pushed: 2023-03-09T08:37:20.000Z (over 1 year ago)
Last Synced: 2024-07-28T07:32:01.597Z (2 months ago)
Topics: adaptive, automated, edge-devices, efficient, neural-architecture-search, on-demand, slimmable-networks
Language: Python
Homepage:
Size: 321 KB
Stars: 904
Watchers: 29
Forks: 131
Open Issues: 11
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

awesome-AutoML-and-Lightweight-Models - JiahuiYu/slimmable_networks

README

        # Slimmable Networks

![version](https://img.shields.io/badge/version-v3.0.0--alpha-green.svg?style=plastic)

![pytorch](https://img.shields.io/badge/pytorch-v1.0.0-green.svg?style=plastic)

![license](https://img.shields.io/badge/license-CC_BY--NC-green.svg?style=plastic)

An open source framework for slimmable training on tasks of ImageNet classification and COCO detection, which has enabled numerous projects. ^{[1](#snets), [2](#usnets), [3](#autoslim)}

1. Slimmable Neural Networks _{[ICLR 2019 Paper](https://arxiv.org/abs/1812.08928) | [OpenReview](https://openreview.net/forum?id=H1gMCsAqY7) | [Detection](https://github.com/JiahuiYu/slimmable_networks/tree/detection) | [Model Zoo](#slimmable-model-zoo)}



Illustration of slimmable neural networks. The same model can run at different widths (number of active channels), permitting instant and adaptive accuracy-efficiency trade-offs.

2. Universally Slimmable Networks and Improved Training Techniques _{[ICCV 2019 Paper](https://arxiv.org/abs/1903.05134) | [Model Zoo](#slimmable-model-zoo)}



Illustration of universally slimmable networks. The same model can run at **arbitrary** widths.

3. AutoSlim: Towards One-Shot Architecture Search for Channel Numbers _{[NeurIPS 2019 Workshop Paper](https://arxiv.org/abs/1903.11728) | [Model Zoo](#slimmable-model-zoo)}



AutoSlimming MobileNet v1, MobileNet v2, MNasNet and ResNet-50: the optimized number of channels under **each** computational budget (FLOPs).

## Run

0. Requirements:

    * python3, pytorch 1.0, torchvision 0.2.1, pyyaml 3.13.

    * Prepare ImageNet-1k data following pytorch [example](https://github.com/pytorch/examples/tree/master/imagenet).

1. Training and Testing:

    * The codebase is a general ImageNet training framework using yaml config under `apps` dir, based on PyTorch.

    * To test, download pretrained models to `logs` dir and directly run command.

    * To train, comment `test_only` and `pretrained` in config file. You will need to manage [visible gpus](https://devblogs.nvidia.com/cuda-pro-tip-control-gpu-visibility-cuda_visible_devices/) by yourself.

    * Command: `python train.py app:{apps/***.yml}`. `{apps/***.yml}` is config file. Do not miss `app:` prefix.

    * Training and testing of MSCOCO benchmarks are released under branch [detection](https://github.com/JiahuiYu/slimmable_networks/tree/detection).

2. Still have questions?

    * If you still have questions, please search closed issues first. If the problem is not solved, please open a new.

## Slimmable Model Zoo

**[Slimmable Neural Networks](https://arxiv.org/abs/1812.08928)**

| Model | Switches (Widths) | Top-1 Err. | FLOPs | Model ID |

| :--- | :---: | :---: | ---: | :---: |

| S-MobileNet v1 | 1.00
0.75
0.50
0.25 | 28.5
30.5
35.2
46.9 | 569M
325M
150M
41M | [a6285db](https://github.com/JiahuiYu/slimmable_networks/files/2709079/s_mobilenet_v1_0.25_0.5_0.75_1.0.pt.zip) |

| S-MobileNet v2 | 1.00
0.75
0.50
0.35 | 29.5
31.1
35.6
40.3 | 301M
209M
97M
59M | [0593ffd](https://github.com/JiahuiYu/slimmable_networks/files/2709080/s_mobilenet_v2_0.35_0.5_0.75_1.0.pt.zip) |

| S-ShuffleNet | 2.00
1.00
0.50 | 28.6
34.5
42.8 | 524M
138M
38M | [1427f66](https://github.com/JiahuiYu/slimmable_networks/files/2709082/s_shufflenet_0.5_1.0_2.0.pt.zip) |

| S-ResNet-50 | 1.00
0.75
0.50
0.25 | 24.0
25.1
27.9
35.0 | 4.1G
2.3G
1.1G
278M | [3fca9cc](https://drive.google.com/open?id=1f6q37OkZaz_0GoOAwllHlXNWuKwor2fC) |

**[Universally Slimmable Networks and Improved Training Techniques](https://arxiv.org/abs/1903.05134)**

| Model | Model ID | Spectrum | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |

| :- | :-: | :- | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - | - |

| US‑MobileNet v1 | [13d5af2](https://github.com/JiahuiYu/slimmable_networks/files/2979952/us_mobilenet_v1_calibrated.pt.zip) | Width
MFLOPs
Top-1 Err. | 1.0
568 
28.2  | 0.975 
543 
28.3  | 0.95 
517 
28.4  | 0.925 
490 
28.7  | 0.9 
466 
28.7  | 0.875 
443 
29.1  | 0.85 
421 
29.4  | 0.825 
389 
29.7  | 0.8 
366 
30.2  | 0.775 
345 
30.3  | 0.75 
325 
30.5  | 0.725 
306 
30.9  | 0.7 
287 
31.2  | 0.675 
267 
31.7  | 0.65 
249 
32.2  | 0.625 
232 
32.5  | 0.6 
217 
33.2  | 0.575 
201 
33.7  | 0.55 
177 
34.4  | 0.525 
162 
35.0  | 0.5 
149 
35.8  | 0.475 
136 
36.5  | 0.45 
124 
37.3  | 0.425 
114 
38.1  | 0.4 
100 
39.0  | 0.375 
89 
40.0  | 0.35 
80 
41.0  | 0.325 
71 
41.9  | 0.3 
64 
42.7  | 0.275 
48 
44.2  | 0.25
41
44.3 |

| US‑MobileNet v2 | [3880cad](https://github.com/JiahuiYu/slimmable_networks/files/2979953/us_mobilenet_v2_calibrated.pt.zip) | Width
MFLOPs
Top-1 Err. | 1.0 
300 
28.5 | 0.975 
299 
28.5 | 0.95 
284 
28.8 | 0.925 
274 
28.9 | 0.9 
269 
29.1 | 0.875 
268 
29.1 | 0.85 
254 
29.4 | 0.825 
235 
29.9 | 0.8 
222 
30.0 | 0.775 
213 
30.2 | 0.75 
209 
30.4 | 0.725 
185 
30.7 | 0.7 
173 
31.1 | 0.675 
165 
31.4 | 0.65 
161 
31.7 | 0.625 
161 
31.7 | 0.6 
151 
32.4 | 0.575 
150 
32.4 | 0.55 
106 
34.4 | 0.525 
100 
34.6 | 0.5 
97 
34.9 | 0.475 
96 
35.1 | 0.45 
88 
35.8 | 0.425 
88 
35.8 | 0.4 
80 
36.6 | 0.375 
80 
36.7 | 0.35
59
37.7 |

**[AutoSlim: Towards One-Shot Architecture Search for Channel Numbers](https://arxiv.org/abs/1903.11728)**

| Model | Top-1 Err. | FLOPs | Model ID |

| :--- | :---: | ---: | :---: |

| AutoSlim-MobileNet v1 | 27.0
28.5
32.1 | 572M
325M
150M | [9b0b1ab](https://github.com/JiahuiYu/slimmable_networks/files/5166182/autoslim_mobilenet_v1.pt.zip) |

| AutoSlim-MobileNet v2 | 24.6
25.8
27.0 | 505M
305M
207M | [a24f1f2](https://github.com/JiahuiYu/slimmable_networks/files/5166194/autoslim_mobilenet_v2.pt.zip) |

| AutoSlim-MNasNet | 24.6
25.4
26.8 | 532M
315M
217M | [31477c9](https://drive.google.com/file/d/1tEuMYc_F-4MUYPua8KAIKjEd7eJDVSx2) |

| AutoSlim-ResNet-50 | 24.0
24.4
26.0
27.8 | 3.0G
2.0G
1.0G
570M | [f95f419](https://drive.google.com/file/d/1WOOu6frdfGo1_nyHdaMpRILPATtzVAMT) |

## Technical Details

Implementing slimmable networks and slimmable training is straightforward:

  * Switchable batchnorm and slimmable layers are implemented in [`models/slimmable_ops`](/models/slimmable_ops.py).

  * Slimmable training is implemented in [these lines](https://github.com/JiahuiYu/slimmable_networks/blob/aeb10c9f437208603145e073ee730f0d7dbfa80f/train.py#L281-L289) in [`train.py`](/train.py).

## License

CC 4.0 Attribution-NonCommercial International

The software is for educaitonal and academic research purpose only.