https://github.com/reshalfahsi/image-classification-augmentation

Image Classification Using Swin Transformer With RandAugment, CutMix, and MixUp
https://github.com/reshalfahsi/image-classification-augmentation

caltech256 cutmix cutmix-augmentation cutmix-mixup image-classification mixup mixup-cutmix pytorch pytorch-lightning randaugment swin-transformer transfer-learning

Last synced: about 6 hours ago
JSON representation

Image Classification Using Swin Transformer With RandAugment, CutMix, and MixUp

• Host: GitHub
• URL: https://github.com/reshalfahsi/image-classification-augmentation
• Owner: reshalfahsi
• Created: 2024-02-28T10:41:56.000Z (9 months ago)
• Default Branch: master
• Last Pushed: 2024-02-29T00:59:17.000Z (9 months ago)
• Last Synced: 2024-02-29T11:49:00.080Z (9 months ago)
• Topics: caltech256, cutmix, cutmix-augmentation, cutmix-mixup, image-classification, mixup, mixup-cutmix, pytorch, pytorch-lightning, randaugment, swin-transformer, transfer-learning
• Language: Jupyter Notebook
• Homepage:
• Size: 3.32 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Image Classification Using Swin Transformer With RandAugment, CutMix, and MixUp

 


    

    


 


In this project, we will explore three distinct Swin Transformers, i.e., without augmentation, with augmentation, and without using the pre-trained weight (or from scratch). Here, the augmentation is undertaken with RandAugment, CutMix, and MixUp. We are about to witness the consequences of utilizing augmentation and pre-trained weight (transfer learning) on the models on the imbalanced dataset, i.e., Caltech-256. The dataset is split per category with a ratio of ``81``:``9``:``10`` for the training, validation, and testing sets. For the from scratch model, each category is truncated to ``100`` instances. Applying the augmentation and pre-trained weight clearly boosts the performance of the model. Not to mention the pre-trained weight insanely pushes the model to effectively predict the right label in the top-1 and top-5.

## Experiment

Check out this [notebook](https://github.com/reshalfahsi/image-classification-augmentation/blob/master/Image_Classification_Using_Swin_Transformer_With_RandAugment_CutMix_and_MixUp.ipynb) to see and ponder the full implementation.

## Result

## Quantitative Result

The result below shows the performance of three different Swin Transformer models: without augmentation, with augmentation, and from scratch, quantitatively.

Model | Loss | Top-1 Acc. | Top-5 Acc. |

------------ | ------------- | ------------- | ------------- |

No Augmentation |  0.369 | 90.17% | 97.68% |

Augmentation | 0.347 | 91.57% | 98.75% |

From Scratch | 4.544 | 11.58% | 27.09% |

## Validation Accuracy and Loss Curve

  
 Accuracy curves of the models on the validation set. 


  
 Loss curves of the models on the validation set. 


## Qualitative Result

The following collated pictures visually delineate the quality of the prediction of the three models.

  
 The prediction result of Swin Transformer without augmentation. 


  
 The prediction result of Swin Transformer with augmentation. 


  
 The prediction result of Swin Transformer from scratch (no pre-trained). 


## Credit

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://arxiv.org/pdf/2103.14030.pdf)

- [TorchVision's Swin Transformer](https://github.com/pytorch/vision/blob/main/torchvision/models/swin_transformer.py)

- [Image classification with Swin Transformers](https://keras.io/examples/vision/swin_transformers/)

- [Caltech-256 Object Category Dataset](https://authors.library.caltech.edu/records/5sv1j-ytw97)

- [TorchVision's Caltech256 Dataset](https://github.com/pytorch/vision/blob/main/torchvision/datasets/caltech.py)

- [RandAugment: Practical automated data augmentation with a reduced search space](https://arxiv.org/pdf/1909.13719.pdf)

- [RandAugment for Image Classification for Improved Robustness](https://keras.io/examples/vision/randaugment/)

- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://arxiv.org/pdf/1905.04899.pdf)

- [CutMix data augmentation for image classification](https://keras.io/examples/vision/cutmix/)

- [mixup: Beyond Empirical Risk Minimization](https://arxiv.org/pdf/1710.09412.pdf)

- [MixUp augmentation for image classification](https://keras.io/examples/vision/mixup/)

- [Multi-head or Single-head? An Empirical Comparison for Transformer Training](https://arxiv.org/pdf/2106.09650.pdf)

- [Getting 95% Accuracy on the Caltech101 Dataset using Deep Learning](https://debuggercafe.com/getting-95-accuracy-on-the-caltech101-dataset-using-deep-learning/)

- [How to use CutMix and MixUp](https://pytorch.org/vision/main/auto_examples/transforms/plot_cutmix_mixup.html)

- [PyTorch Lightning](https://lightning.ai/docs/pytorch/latest/)