{"id":16270213,"url":"https://github.com/shoufachen/adaptformer","last_synced_at":"2025-04-09T07:10:12.413Z","repository":{"id":37535439,"uuid":"496634448","full_name":"ShoufaChen/AdaptFormer","owner":"ShoufaChen","description":"[NeurIPS 2022] Implementation of \"AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition\"","archived":false,"fork":false,"pushed_at":"2022-09-16T00:55:50.000Z","size":2801,"stargazers_count":348,"open_issues_count":21,"forks_count":21,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-04-02T02:40:00.944Z","etag":null,"topics":["adapter","neurips-2022","recognition","vision-transformer","visual-adapter"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2205.13535","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ShoufaChen.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-05-26T13:37:49.000Z","updated_at":"2025-04-01T15:48:12.000Z","dependencies_parsed_at":"2023-01-17T21:45:50.976Z","dependency_job_id":null,"html_url":"https://github.com/ShoufaChen/AdaptFormer","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShoufaChen%2FAdaptFormer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShoufaChen%2FAdaptFormer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShoufaChen%2FAdaptFormer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ShoufaChen%2FAdaptFormer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ShoufaChen","download_url":"https://codeload.github.com/ShoufaChen/AdaptFormer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247994122,"owners_count":21030050,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["adapter","neurips-2022","recognition","vision-transformer","visual-adapter"],"created_at":"2024-10-10T18:09:52.763Z","updated_at":"2025-04-09T07:10:12.388Z","avatar_url":"https://github.com/ShoufaChen.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n## [NeurIPS 2022] AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition\n\n### [Project Page](http://www.shoufachen.com/adaptformer-page/) |  [arXiv](https://arxiv.org/abs/2205.13535)\n\n![teaser](figs/teaser.gif)\n\u003c/div\u003e\n\n\nThis is a PyTorch implementation of the paper [AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition](https://arxiv.org/abs/2205.13535).\n\n[Shoufa Chen](https://www.shoufachen.com/)\u003csup\u003e1\u003c/sup\u003e\\*,\n[Chongjian Ge](https://chongjiange.github.io/)\u003csup\u003e1\u003c/sup\u003e\\*,\n[Zhan Tong](https://scholar.google.com/citations?user=6FsgWBMAAAAJ)\u003csup\u003e2\u003c/sup\u003e,\n[Jiangliu Wang](https://laura-wang.github.io/)\u003csup\u003e2,3\u003c/sup\u003e,\n[Yibing Song](https://ybsong00.github.io/)\u003csup\u003e2\u003c/sup\u003e,\n[Jue Wang](http://juewang725.github.io/)\u003csup\u003e2\u003c/sup\u003e,\n[Ping Luo](http://luoping.me/)\u003csup\u003e1\u003c/sup\u003e \u003cbr\u003e\n\u003csup\u003e1\u003c/sup\u003eThe University of Hong Kong, \u003csup\u003e2\u003c/sup\u003eTencent AI Lab, \u003csup\u003e3\u003c/sup\u003eThe Chinese University of Hong Kong  \n\\*denotes equal contribution\n\n### Catalog\n\n- [x] Video code\n- [x] Image code\n\n### Usage\n\n#### Install\n* Tesla V100 (32G): CUDA 10.1 + PyTorch 1.6.0 + torchvision 0.7.0\n* timm 0.4.8\n* einops\n* easydict\n\n#### Data Preparation\nSee [DATASET.md](DATASET.md).\n\n#### Training\nStart\n```bash\n# video\nOMP_NUM_THREADS=1 python3 -m torch.distributed.launch \\\n    --nproc_per_node=8 --nnodes=8 \\\n    --node_rank=$1 --master_addr=$2 --master_port=22234 \\\n    --use_env main_video.py \\\n    --finetune /path/to/pre_trained/checkpoints \\\n    --output_dir /path/to/output \\\n    --batch_size 16 --epochs 90 --blr 0.1 --weight_decay 0.0 --dist_eval \\\n    --data_path /path/to/SSV2 --data_set SSV2 \\\n    --ffn_adapt\n```\non each of 8 nodes. `--master_addr` is set as the ip of the node 0. and `--node_rank` is 0, 1, ..., 7 for each node.\n\n```bash\n# image\npython3 -m torch.distributed.launch --nproc_per_node=8 --use_env main_image.py \\\n    --batch_size 128 --cls_token \\\n    --finetune /path/to/pre_trained/mae_pretrain_vit_b.pth \\\n    --dist_eval --data_path /path/to/data \\\n    --output_dir /path/to/output  \\\n    --drop_path 0.0  --blr 0.1 \\\n    --dataset cifar100 --ffn_adapt\n```\n\nTo obtain the pre-trained checkpoint, see [PRETRAIN.md](PRETRAIN.md).\n### Acknowledgement\n\nThe project is based on [MAE](https://github.com/facebookresearch/mae), [VideoMAE](https://github.com/MCG-NJU/VideoMAE), [timm](https://github.com/rwightman/pytorch-image-models), and [MAM](https://github.com/jxhe/unify-parameter-efficient-tuning).\nThanks for their awesome works.\n\n### Citation\n```\n@article{chen2022adaptformer,\n      title={AdaptFormer: Adapting Vision Transformers for Scalable Visual Recognition},\n      author={Chen, Shoufa and Ge, Chongjian and Tong, Zhan and Wang, Jiangliu and Song, Yibing and Wang, Jue and Luo, Ping},\n      journal={arXiv preprint arXiv:2205.13535},\n      year={2022}\n}\n```\n\n### License\n\nThis project is under the MIT license. See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshoufachen%2Fadaptformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshoufachen%2Fadaptformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshoufachen%2Fadaptformer/lists"}