https://github.com/xxradon/IGCV3-pytorch
IGCV3 reimplement by pytorch
https://github.com/xxradon/IGCV3-pytorch
Last synced: 7 months ago
JSON representation
IGCV3 reimplement by pytorch
- Host: GitHub
- URL: https://github.com/xxradon/IGCV3-pytorch
- Owner: xxradon
- Created: 2018-06-21T07:19:00.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-06-21T07:25:22.000Z (over 7 years ago)
- Last Synced: 2024-10-31T17:39:15.513Z (about 1 year ago)
- Language: Python
- Size: 567 KB
- Stars: 19
- Watchers: 4
- Forks: 5
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
- awesome-image-classification - unofficial-pytorch : https://github.com/xxradon/IGCV3-pytorch
- awesome-image-classification - unofficial-pytorch : https://github.com/xxradon/IGCV3-pytorch
README
# MobileNet-V2 and IGCV3
An implementation of `Google MobileNet-V2` and `IGCV3` introduced in PyTorch.
Link to the original paper: [Inverted Residuals and Linear Bottlenecks: Mobile Networks for Classification, Detection and Segmentation](https://arxiv.org/abs/1801.04381),IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks. Ke Sun, Mingjie Li, Dong Liu, and Jingdong Wang.
arXiv preprint [arXIV:1806.00178](https://arxiv.org/pdf/1806.00178.pdf) (2017)
This implementation was made to be an example of a common deep learning software architecture. It's simple and designed to be very modular. All of the components needed for training and visualization are added.
# Only tested on Cifar10 and Cifar100!!!!
## Usage
This project uses Python 3 and PyTorch 0.3.1
### Main Dependencies
```
pytorch 0.3.1
numpy
tqdm
easydict
matplotlib
tensorboardX
```
Install dependencies:
```bash
pip install -r requirements.txt
```
### Train and Test
1. Prepare your data, then create a dataloader class such as `cifar10data.py` and `cifar100data.py`.
2. Create a .json config file for your experiments. Use the given .json config files as a reference.
### Run
```
python main.py config/.json
```
### Experiments
Due to the lack of computational power. I trained on CIFAR-10 dataset as an example to prove correctness, and was able to achieve test top1-accuracy of 90.9%.
#### Tensorboard Visualization
Tensorboard is integrated with the project using `tensorboardX` library which proved to be very useful as there is no official visualization library in pytorch.
You can start it using:
```bash
tensorboard --logdir experimenets//summaries
```
These are the learning curves for the CIFAR-10 experiment.
# IGCV3:Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks.
The codes are based on https://github.com/liangfu/mxnet-mobilenet-v2.
> IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks. Ke Sun, Mingjie Li, Dong Liu, and Jingdong Wang.
arXiv preprint [arXIV:1806.00178](https://arxiv.org/pdf/1806.00178.pdf) (2017)
## Prior Works
### Interleaved Group Convolutions ([IGCV1](https://arxiv.org/pdf/1707.02725.pdf))
Interleaved Group Convolutions use a pair of two successive interleaved group convolutions: primary group convolution and secondary group convolution. The two group convolutions are complementary.
> Illustrating the interleaved group convolution, with L = 2 primary partitions and M = 3 secondary partitions. The convolution for each primary partition in primary group convolution is spatial. The convolution for each secondary partition in secondary group convolution is point-wise (1 × 1).
You can find its code [here](https://github.com/hellozting/InterleavedGroupConvolutions)!
### Interleaved Structured Sparse Convolution ([IGCV2](https://arxiv.org/pdf/1804.06202.pdf))
IGCV2 extends IGCV1 by decomposing the convolution matrix in to more structured sparse matrices, which uses a depth-wise convoultion (3 × 3) to replace the primary group convoution in IGC and uses a series of point-wise group convolutions (1 × 1).
## Interleaved Low-Rank Group Convolutions (IGCV3)
We proposes Interleaved Low-Rank Group Convolutions, named IGCV3, extend IGCV2 by using low-rank group convolutions to replace group convoutions in IGCV2. It consists of a channel-wise spatial convolution, a low-rank group convolution with 
groups that reduces the width and a low-rank group convolution with 
groups which expands the widths back.
> Illustrating the interleaved branches in IGCV3 block. The first group convolution is a group 1 × 1 convolution with =2 groups. The second is a channel-wise spatial convolution. The third is a group 1 × 1 convolution with =2 groups.
## Results
### CIFAR Experiments
We compare our IGCV3 network with other Mobile Networks on CIFAR datasets which illustrated our model' advantages on small dataset.
#### Comparison with Other Mobile Networks
Classification accuracy comparisons of MobileNetV2 and IGCV3 on CIFAR datasets. "Network s×" means reducing the number of parameter in "Network 1.0×" by s times.
#Params (M) CIFAR-10 CIFAR100
MobileNetV2（our impl.） 2.394.56 77.09
IGCV3-D 0.5× 1.294.73 77.29
IGCV3-D 0.7× 1.794.92 77.83
IGCV3-D 1.0× 2.494.96 77.95
IGCV3-D 1.0×(my pytorch impl) 2.494.70 75.96
MobileNetV2（my pytorch impl） 2.394.01 --
#### Comparison with IGCV2
#Params (M) CIFAR-10 CIFAR100
IGCV2 2.494.76 77.45
IGCV3-D 2.494.96 77.95
### ImageNet Experiments
Comparison with MobileNetV2 on ImageNet.
#### Before Retrain
#Params (M) Top-1 Top-5
MobileNetV2 3.470.0 89.0
IGCV3-D 3.570.6 89.7
#### After Retrain
#Params (M) Top-1 Top-5
MobileNetV2 3.471.4 90.1
IGCV3-D 3.5 72.2 90.5
# The code is maily from [IGCV3](https://github.com/homles11/IGCV3) and [MobilenetV2](https://github.com/MG2033/MobileNet-V2).Thanks for their contribution.