{"id":17670724,"url":"https://github.com/sayannath/vit-tf-hub-application","last_synced_at":"2025-05-12T14:18:59.142Z","repository":{"id":46764860,"uuid":"410542428","full_name":"sayannath/ViT-TF-Hub-Application","owner":"sayannath","description":"Build and fine-tune your Image Classifier using a Vision Transformer Model from TensorFlow Hub","archived":false,"fork":false,"pushed_at":"2021-09-27T13:32:33.000Z","size":13935,"stargazers_count":10,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-26T12:22:55.766Z","etag":null,"topics":["fine-tuning","jax","keras","tensorflow","tf2","tfhub","transformers","vision-transformer","vit"],"latest_commit_sha":null,"homepage":"https://tfhub.dev/sayakpaul/collections/vision_transformer/1","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sayannath.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-09-26T12:12:30.000Z","updated_at":"2023-12-20T15:48:04.000Z","dependencies_parsed_at":"2022-09-14T22:54:49.809Z","dependency_job_id":null,"html_url":"https://github.com/sayannath/ViT-TF-Hub-Application","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FViT-TF-Hub-Application","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FViT-TF-Hub-Application/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FViT-TF-Hub-Application/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayannath%2FViT-TF-Hub-Application/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sayannath","download_url":"https://codeload.github.com/sayannath/ViT-TF-Hub-Application/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243150959,"owners_count":20244488,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fine-tuning","jax","keras","tensorflow","tf2","tfhub","transformers","vision-transformer","vit"],"created_at":"2024-10-24T02:42:13.855Z","updated_at":"2025-03-12T03:31:17.960Z","avatar_url":"https://github.com/sayannath.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"![GitHub forks](https://img.shields.io/github/forks/sayannath/ViT-TF-Hub-Application?style=for-the-badge)\n![GitHub Repo stars](https://img.shields.io/github/stars/sayannath/ViT-TF-Hub-Application?style=for-the-badge)\n![GitHub last commit](https://img.shields.io/github/last-commit/sayannath/ViT-TF-Hub-Application?style=for-the-badge)\n![Twitter Follow](https://img.shields.io/twitter/follow/sayannath2350?style=for-the-badge)\n[![Ask Me Anything !](https://img.shields.io/badge/Ask%20me-anything-1abc9c.svg?style=for-the-badge)](https://gitHub.com/sayannath)\n\n# Vision Transformer TF-Hub Application\n![PngItem_3011351 (1)](https://user-images.githubusercontent.com/72073401/134901679-918e04ca-2e70-4847-8e15-98003ff878ae.png)\n\n## Description\nThis repositories show how to `fine-tune` a Vision Transformer model from [TensorFlow Hub](https://www.tfhub.dev) on the Image Scene Detection dataset.\n\n## Dataset Used\nA newly collected Camera Scene Classification dataset consisting of images belonging to 30 different classes. This dataset is the part of the competition which is [Mobile AI Workshop @ CVPR 2021](https://competitions.codalab.org/competitions/28113).\nYou can find the dataset details [here](https://competitions.codalab.org/competitions/28113#participate).\n\n## Models\n\nThese models are available on [TensorFlow Hub](https://www.tfhub.dev) for Vision Transformer.\n\n### Image Classifiers\n\n* [ViT-S16](https://tfhub.dev/sayakpaul/vit_s16_classification/1)\n* [ViT-B8](https://tfhub.dev/sayakpaul/vit_b8_classification/1)\n* [ViT-B16](https://tfhub.dev/sayakpaul/vit_b16_classification/1)\n* [ViT-B32](https://tfhub.dev/sayakpaul/vit_b32_classification/1)\n* [ViT-L16](https://tfhub.dev/sayakpaul/vit_l16_classification/1)\n* [ViT-R26-S32 (light augmentation)](https://tfhub.dev/sayakpaul/vit_r26_s32_lightaug_classification/1)\n* [ViT-R26-S32 (medium augmentation)](https://tfhub.dev/sayakpaul/vit_r26_s32_medaug_classification/1)\n* [ViT-R50-L32](https://tfhub.dev/sayakpaul/vit_r50_l32_classification/1)\n\n### Feature Extractors\n\n* [ViT-S16](https://tfhub.dev/sayakpaul/vit_s16_fe/1)\n* [ViT-B8](https://tfhub.dev/sayakpaul/vit_b8_fe/1)\n* [ViT-B16](https://tfhub.dev/sayakpaul/vit_b16_fe/1)\n* [ViT-B32](https://tfhub.dev/sayakpaul/vit_b32_fe/1)\n* [ViT-L16](https://tfhub.dev/sayakpaul/vit_l16_fe/1)\n* [ViT-R26-S32 (light augmentation)](https://tfhub.dev/sayakpaul/vit_r26_s32_lightaug_fe/1)\n* [ViT-R26-S32 (medium augmentation)](https://tfhub.dev/sayakpaul/vit_r26_s32_medaug_fe/1)\n* [ViT-R50-L32](https://tfhub.dev/sayakpaul/vit_r50_l32_fe/1)\n\n\u003e Note: As we want to fine-tune our model so we used the feature-extractor model and build the image classifier.\n\n## Benchmark Results\n\n| Sl No |          Models          | No of Parameters | Accuracy | Validation Accuracy |\n|:-----:|:------------------------:|:----------------:|:--------:|:-------------------:|\n|   1   |         ViT-S/16         |    21,677,214    |  99.73%  |        96.87%       |\n|   2   |  ViT R26-S/32(light aug) |    36,058,462    |  99.70%  |        96.67%       |\n|   3   | ViT R26-S/32(medium aug) |    36,058,462    |  99.80%  |        97.17%       |\n|   4   |         ViT B/32         |    87,478,302    |  99.43%  |        96.87%       |\n|   5   |     MobileNetV3Small     |     2,070,158    |  95.20%  |        92.73%       |\n|   6   |        MobileNetV2       |     2,929,246    |  95.06%  |        88.89%       |\n|   7   |     BigTransfer (BiT)    |                  |  99.53%  |        96.97%       |\n\n\u003e Note: Last three results are benchmarked during thr CVPR Competition. You can find the repository [here](https://github.com/sayannath/Image-Scene-Classification).\n\n## Notebooks\n:white_check_mark: **ViT** **S/16** \u003cbr\u003e\n:white_check_mark: **ViT** **R26-S/32 (Light Augmentation)** \u003cbr\u003e\n:white_check_mark: **ViT** **R26-S/32 (Medium Augmentation)** \u003cbr\u003e\n:white_check_mark: **ViT** **B/32** \u003cbr\u003e\n:white_large_square: **ViT R50-L/32** \u003cbr\u003e\n:white_large_square: **ViT B/16** \u003cbr\u003e\n:white_large_square: **ViT L/16** \u003cbr\u003e\n:white_large_square: **ViT B/8** \u003cbr\u003e\n\n## Links\n| Sl No | Models                   | Colab Notebook | TensorBoard |\n|----|--------------------------|----------------|-------------|\n| 1  | ViT-S/16                 | [Link](https://colab.research.google.com/drive/1ISB3E5_wjojRjhbCjRLaKLCPxUHqtxd1?usp=sharing)       | [Link](https://tensorboard.dev/experiment/m9OMnYIzTw66LWXvyXCYgg/)    |\n| 2  | ViT R26-S/32(light aug)  | [Link](https://colab.research.google.com/drive/14Ms__eAJOD0jdDLlHxmIawcQET_GyQjz?usp=sharing)       | [Link](https://tensorboard.dev/experiment/myd5IEZtRjWEmAQQ9lSolA/)    |\n| 3  | ViT R26-S/32(medium aug) | [Link](https://colab.research.google.com/drive/1xuQTvl5lYqR3tn_17d7_WeDrdj76ieIl?usp=sharing)       | [Link](https://tensorboard.dev/experiment/35bwOLWxQLqO0E11sdveDQ/)    |\n| 4  | ViT B/32                 | [Link](https://colab.research.google.com/drive/1-9mo1H8tOHOjqunF317a-I1B4vbX6yeC?usp=sharing)       | [Link](https://tensorboard.dev/experiment/H2QSxurmQt6YNVVWSlTaUA/)    |\n\n\u003e Each directory of model contains the particular notebook, python script, metric graph, train-logs(in .csv) and TensorBoard callbacks.\n\n## References\n\n[1] [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale by Dosovitskiy et al.](https://arxiv.org/abs/2010.11929)\n\n[2] [How to train your ViT? Data, Augmentation, and Regularization in Vision Transformers by Steiner et al.](https://arxiv.org/abs/2106.10270)\n\n[3] [Vision Transformer GitHub](https://github.com/google-research/vision_transformer)\n\n[4] [jax2tf tool](https://github.com/google/jax/tree/main/jax/experimental/jax2tf/)\n\n[5] [Image Classification with Vision Transformer in Keras](https://keras.io/examples/vision/image_classification_with_vision_transformer/)\n\n[6] [ViT-jax2tf](https://github.com/sayakpaul/ViT-jax2tf)\n\n[7] [Vision Transformers are Robust Learners](https://arxiv.org/abs/2105.07581), [Repository](https://github.com/sayakpaul/robustness-vit)\n\n[8] [Vision Transformer TF-Hub Model Collection](https://tfhub.dev/sayakpaul/collections/vision_transformer/1)\n\n## Acknowledgements\n\n* Thanks to [Sayak Paul](https://sayak.dev) for building the models of ViT so that we can use Vision Transformer in a straight way.\n* Thanks to the authors of Vision Transformers for their efforts put into open-sourcing the models.\n\n## Contributors\n\n\u003ca href=\"https://github.com/sayannath/ViT-TF-Hub-Application/graphs/contributors\"\u003e\n  \u003cimg src=\"https://contrib.rocks/image?repo=sayannath/ViT-TF-Hub-Application\" /\u003e\n\u003c/a\u003e\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayannath%2Fvit-tf-hub-application","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsayannath%2Fvit-tf-hub-application","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayannath%2Fvit-tf-hub-application/lists"}