https://github.com/lusinlu/bias-loss-skipblocknet

[ICCV 2021]Code for the the bias loss and evaluation of SkipblockNet model on ImageNet validation set
https://github.com/lusinlu/bias-loss-skipblocknet

deep-learning loss-functions machine-learning mobile-networks mobilenet objective-functions pytorch pytorch-implementation

Last synced: 10 days ago
JSON representation

[ICCV 2021]Code for the the bias loss and evaluation of SkipblockNet model on ImageNet validation set

• Host: GitHub
• URL: https://github.com/lusinlu/bias-loss-skipblocknet
• Owner: lusinlu
• Created: 2020-11-20T10:15:51.000Z (almost 4 years ago)
• Default Branch: main
• Last Pushed: 2022-09-09T09:40:40.000Z (about 2 years ago)
• Last Synced: 2023-10-20T01:13:16.991Z (about 1 year ago)
• Topics: deep-learning, loss-functions, machine-learning, mobile-networks, mobilenet, objective-functions, pytorch, pytorch-implementation
• Language: Python
• Homepage:
• Size: 19.8 MB
• Stars: 33
• Watchers: 3
• Forks: 6
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Bias Loss & SkipblockNet

[![report](https://img.shields.io/badge/arxiv-report-red)](https://arxiv.org/pdf/2107.11170.pdf)

[![PWC](https://img.shields.io/badge/PWC-report-blue)](https://paperswithcode.com/paper/bias-loss-for-mobile-neural-networks)

[ICCV 2021]Demo for the bias loss and SkipblockNet architecture presented in the [paper](https://openaccess.thecvf.com/content/ICCV2021/papers/Abrahamyan_Bias_Loss_for_Mobile_Neural_Networks_ICCV_2021_paper.pdf) "Bias Loss for Mobile Neural Networks".

## Requirements

for installing required packages run

` pip install -r requirements.txt`

## Usage (SkipblockNet)

Pretrained SkipblockNet-m is available from [Google Drive](https://drive.google.com/drive/folders/1G3UR8wtTFB8S-9Fp6sRtfn9Vtfb6XcTU?usp=sharing). For the testing please download and place the model in the same directory as the validation script.

`python validate.py --data path/to/the/dataset`

## Usage (Bias loss)

Training and testing codes are available for DenseNet121, ShuffleNet V2 0.5x and ResNet18. To test the pretrained models please download corresponding model from the [Google Drive](https://drive.google.com/drive/u/0/folders/1G3UR8wtTFB8S-9Fp6sRtfn9Vtfb6XcTU) and run the testing script in the bias loss directory

`python test.py --checkpoint 'path to the checkpoint' --model 'name of the model' --data_path 'path to the cifar-100 dataset'`

To train the models run the training script in the bias loss directory as follows:

`python train.py --model 'name of the model to be trained' --data_path 'path to the cifar-100 dataset'`

## Introduction

"Bias Loss for Mobile Neural Networks"

By Lusine Abrahamyan, Valentin Ziatchin, Yiming Chen and Nikos Deligiannis.

### Approach (SkipblockNet)



### Performance (SkipblockNet)

SkipNet beats other SOTA lightweight CNNs such as MobileNetV3 and FBNet.

 |



### Approach (Bias loss)

The bias loss is a dynamically scaled cross-entropy loss, where the scale decays as the variance of data point decreases.



### Performance (Bias loss)

Bellow is the results of the pretrained models that can be found in the [Google Drive](https://drive.google.com/drive/u/0/folders/1G3UR8wtTFB8S-9Fp6sRtfn9Vtfb6XcTU)

| Model         | Top-1 bias loss | Top-1 CE |

| :------------ |:---------------:| -----:|

| ResNet18            | 75.51%    |   74.33% |

| DenseNet121         | 77.83%    |   75.98% |

| ShuffleNet V2 0.5x  | 72.00%    |   71.55% |

## Citation

If you find the code useful for your research, please consider citing our works

```

@inproceedings{abrahamyan2021bias,

  title={Bias Loss for Mobile Neural Networks},

  author={Abrahamyan, Lusine and Ziatchin, Valentin and Chen, Yiming and Deligiannis, Nikos},

  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},

  pages={6556--6566},

  year={2021}

}

```

## Acknowledgement

Codes is heavily modified from [pytorch-vision](https://github.com/pytorch/vision/blob/master/torchvision/models/resnet.py) and [pytorch-cifar100](https://github.com/weiaicunzai/pytorch-cifar100).