https://github.com/vfdev-5/vit-pytorch

Introduction and experimentations with ViT
https://github.com/vfdev-5/vit-pytorch

Last synced: 11 days ago
JSON representation

Introduction and experimentations with ViT

• Host: GitHub
• URL: https://github.com/vfdev-5/vit-pytorch
• Owner: vfdev-5
• Created: 2021-01-22T21:41:31.000Z (almost 4 years ago)
• Default Branch: master
• Last Pushed: 2021-03-12T00:44:42.000Z (over 3 years ago)
• Last Synced: 2024-10-13T12:12:41.235Z (26 days ago)
• Language: Jupyter Notebook
• Size: 1.47 MB
• Stars: 0
• Watchers: 4
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Introduction to Vision Transformers with PyTorch

Table of Content:

- [ViT in PyTorch](vit_pytorch.ipynb)

- [Convolution vs Attention](conv_vs_attention.ipynb)

- [Script and logs for ViT training on CIFAR10](cifar10)

Refs:

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://arxiv.org/abs/2010.11929)

- [Training data-efficient image transformers & distillation through attention](https://arxiv.org/abs/2012.12877)

- [Deit code](https://github.com/facebookresearch/deit)