{"id":15661023,"url":"https://github.com/sayakpaul/big_vision_experiments","last_synced_at":"2025-05-05T21:17:11.528Z","repository":{"id":106647716,"uuid":"491083154","full_name":"sayakpaul/big_vision_experiments","owner":"sayakpaul","description":"Contains my experiments with the `big_vision` repo to train ViTs on ImageNet-1k.","archived":false,"fork":false,"pushed_at":"2023-01-16T15:43:35.000Z","size":246,"stargazers_count":22,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-05-05T21:17:06.531Z","etag":null,"topics":["computer-vision","google-cloud","image-recognition","jax","large-scale-pretraining","tpu"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sayakpaul.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-11T11:34:57.000Z","updated_at":"2024-01-04T17:08:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"bd72e5a6-d6f0-4763-a29d-057455e251f2","html_url":"https://github.com/sayakpaul/big_vision_experiments","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2Fbig_vision_experiments","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2Fbig_vision_experiments/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2Fbig_vision_experiments/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sayakpaul%2Fbig_vision_experiments/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sayakpaul","download_url":"https://codeload.github.com/sayakpaul/big_vision_experiments/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252577027,"owners_count":21770721,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","google-cloud","image-recognition","jax","large-scale-pretraining","tpu"],"created_at":"2024-10-03T13:25:26.257Z","updated_at":"2025-05-05T21:17:11.506Z","avatar_url":"https://github.com/sayakpaul.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Experiments with `big_vision`\n\nContains my experiments with the [`big_vision`](https://github.com/google-research/big_vision) repository to train ViTs on ImageNet-1k.\n\n## What is `big_vision`?\n\nFrom the repository:\n\n\u003e This codebase is designed for training large-scale vision models on Cloud TPU VMs. It is based on Jax/Flax libraries, and uses tf.data and TensorFlow Datasets for scalable input pipelines in the Cloud.\n\n\u003e big_vision aims to support research projects at Google. We are unlikely to work on feature requests or accept external contributions, unless they were pre-approved (ask in an issue first). \n\n## Why this repository?\n\n* I really like how `big_vision` is organized into composable modules.\n* I wanted to reproduce some of the ImageNet-1k results reported by the `big_vision` authors.\n* `big_vision` not only reports scores for ImageNet-1k validation set but also reports\nscores for ImageNet-V2 and ImageNet-Real.\n* I wanted to run the entire training using Cloud TPUs and at the same time I wanted to \nimprove my JAX skills.\n* I wanted to improve my chops in large-scale pre-training. Large-scale pretraining is a goldmine of\ndeep learning that continues to benefit downstream applications. Programs like [TRC](https://sites.research.google/trc) make it possible for\nthe community to learn the nitty-gritty of large-scale pre-training by providing\nTPU support.\n* For the sheer joy of training models to SoTA.\n\nThis repository will also contain the trained checkpoints and the training logs. Additionally, \nthis Colab Notebook ([`notebooks/analyze-metrics.ipynb`](https://colab.research.google.com/github/sayakpaul/big_vision_experiments/blob/main/notebooks/analyze-metrics.ipynb)) takes the raw training logs and generates a plot for reporting accuracies\nacross three benchmarks: ImageNet-1k validation set, ImageNetV2, ImageNet-Real.\n\nHere's one such plot I generated from ViT S/16 checkpoints (that get to 76.23% top-1 accuracy on ImageNet-1k validation set within 90 epochs of training):\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"./assets/imagenet1k_s16.png\" width=500/\u003e\n\u003c/div\u003e\n\n\u003csup\u003eTraining was performed on a TPU v3-8 VM that took 7 hours 22 minutes to complete.\u003c/sup\u003e\n\nThe performance of this model is also quite inline with what's reported in [1].\n \n## Checkpoints and training logs\n\n* [`vit_s16_imagenet_1k`](https://github.com/sayakpaul/big_vision_experiments/releases/tag/v0.1.0)\n\n\n## Setup\n\nEven though the `big_vision` repository provides instructions for setting things up I found them a bit incomplete.\nHence, I developed another one. Find it here - [`setup.md`](https://github.com/sayakpaul/big_vision_experiments/blob/main/setup.md).\n\n## References\n\n[1] Better plain ViT baselines for ImageNet-1k: https://arxiv.org/abs/2205.01580\n\n## Acknowledgements\n\n* [TRC (TPU Research Cloud)](https://sites.research.google/trc) for providing TPU access.\n* [ML-GDE program](https://developers.google.com/programs/experts/) for providing GCP credits.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayakpaul%2Fbig_vision_experiments","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsayakpaul%2Fbig_vision_experiments","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsayakpaul%2Fbig_vision_experiments/lists"}