{"id":20305562,"url":"https://github.com/vitae-transformer/vitae-transformer","last_synced_at":"2025-04-09T13:09:59.132Z","repository":{"id":43880436,"uuid":"479654027","full_name":"ViTAE-Transformer/ViTAE-Transformer","owner":"ViTAE-Transformer","description":"The official repo for [NeurIPS'21] \"ViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias\" and [IJCV'22] \"ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond\"","archived":false,"fork":false,"pushed_at":"2023-04-05T02:02:47.000Z","size":23240,"stargazers_count":268,"open_issues_count":11,"forks_count":29,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-02T08:36:26.148Z","etag":null,"topics":["ade20k","deep-learning","imagenet","imagenet-classification","mscoco","object-detection","semantic-segmentation","vision-transformer","vitae-transformer"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ViTAE-Transformer.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-09T07:36:31.000Z","updated_at":"2025-03-31T10:40:18.000Z","dependencies_parsed_at":"2025-01-01T21:11:25.574Z","dependency_job_id":"3a556dfb-bb52-4ddc-984e-44aaf9a8a7a7","html_url":"https://github.com/ViTAE-Transformer/ViTAE-Transformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FViTAE-Transformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FViTAE-Transformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FViTAE-Transformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ViTAE-Transformer%2FViTAE-Transformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ViTAE-Transformer","download_url":"https://codeload.github.com/ViTAE-Transformer/ViTAE-Transformer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248045245,"owners_count":21038554,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ade20k","deep-learning","imagenet","imagenet-classification","mscoco","object-detection","semantic-segmentation","vision-transformer","vitae-transformer"],"created_at":"2024-11-14T17:08:57.365Z","updated_at":"2025-04-09T13:09:59.107Z","avatar_url":"https://github.com/ViTAE-Transformer.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"left\"\u003eViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond\u003ca href=\"https://arxiv.org/abs/2202.10108\"\u003e\u003cimg src=\"https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg\" \u003e\u003c/a\u003e\u003c/h1\u003e \n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"#Updates\"\u003eUpdates\u003c/a\u003e |\n  \u003ca href=\"#introduction\"\u003eIntroduction\u003c/a\u003e |\n  \u003ca href=\"#statement\"\u003eStatement\u003c/a\u003e |\n\u003c/p\u003e\n\n## Current applications\n\n\u003e **Image Classification**: Please see \u003ca href=\"https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Image-Classification\"\u003eViTAE-Transformer for image classification\u003c/a\u003e;\n\n\u003e **Object Detection**: Please see \u003ca href=\"https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Object-Detection\"\u003eViTAE-Transformer for object detection\u003c/a\u003e;\n\n\u003e **Sementic Segmentation**: Please see \u003ca href=\"https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Semantic-Segmentation\"\u003eViTAE-Transformer for semantic segmentation\u003c/a\u003e;\n\n\u003e **Animal Pose Estimation**: Please see \u003ca href=\"https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Animal-Pose-Estimation\"\u003eViTAE-Transformer for animal pose estimation\u003c/a\u003e;\n\n\u003e **Matting**: Please see \u003ca href=\"https://github.com/ViTAE-Transformer/ViTAE-Transformer-Matting\"\u003eViTAE-Transformer for matting\u003c/a\u003e;\n\n\u003e **Remote Sensing**: Please see \u003ca href=\"https://github.com/ViTAE-Transformer/ViTAE-Transformer-Remote-Sensing\"\u003eViTAE-Transformer for Remote Sensing\u003c/a\u003e;\n\n\n## Updates\n\n***09/04/2021***\n- The pretrained models for ViTAE on [matting](https://github.com/ViTAE-Transformer/ViTAE-Transformer-Matting) and [remote sensing](https://github.com/ViTAE-Transformer/ViTAE-Transformer-Remote-Sensing) are released! Please try and have fun!\n\n***24/03/2021***\n- The pretrained models for both ViTAE and ViTAEv2 are released. The code for downstream tasks are also provided for reference.\n\n***07/12/2021***\n- The code is released!\n\n***19/10/2021***\n- The paper is accepted by Neurips'2021! The code will be released soon!\n  \n***06/08/2021***\n- The paper is post on arxiv! The code will be made public available once cleaned up.\n\n## Introduction\n\n\u003cp align=\"left\"\u003eThis repository contains the code, models, test results for the paper \u003ca href=\"https://arxiv.org/pdf/2106.03348.pdf\"\u003eViTAE: Vision Transformer Advanced by Exploring Intrinsic Inductive Bias\u003c/a\u003e. It contains several reduction cells and normal cells to introduce scale-invariance and locality into vision transformers. In \u003ca href=\"https://arxiv.org/pdf/2202.10108.pdf\"\u003eViTAEv2\u003c/a\u003e, we explore the usage of window attentions without shift operations to obtain a better balance between memory footprint, speed, and performance. We also stack the proposed RC and NC in a multi-stage manner to faciliate the learning on other vision tasks including detection, segmentation, and pose.\n\n\u003cfigure\u003e\n\u003cimg src=\"figs/NetworkStructure.png\"\u003e\n\u003cfigcaption align = \"center\"\u003e\u003cb\u003eFig.1 - The details of RC and NC design in ViTAE.\u003c/b\u003e\u003c/figcaption\u003e\n\u003c/figure\u003e\n\n\u003cfigure\u003e\n\u003cimg src=\"figs/ViTAEv2.png\"\u003e\n\u003cfigcaption align = \"center\"\u003e\u003cb\u003eFig.2 - The multi-stage design of ViTAEv2.\u003c/b\u003e\u003c/figcaption\u003e\n\u003c/figure\u003e\n\n\n## Statement\nThis project is for research purpose only. For any other questions please contact [yufei.xu at outlook.com](mailto:yufei.xu@outlook.com) [qmzhangzz at hotmail.com](mailto:qmzhangzz@hotmail.com) .\n\n## Citing ViTAE and ViTAEv2\n```\n@article{xu2021vitae,\n  title={Vitae: Vision transformer advanced by exploring intrinsic inductive bias},\n  author={Xu, Yufei and Zhang, Qiming and Zhang, Jing and Tao, Dacheng},\n  journal={Advances in Neural Information Processing Systems},\n  volume={34},\n  year={2021}\n}\n@article{zhang2022vitaev2,\n  title={ViTAEv2: Vision Transformer Advanced by Exploring Inductive Bias for Image Recognition and Beyond},\n  author={Zhang, Qiming and Xu, Yufei and Zhang, Jing and Tao, Dacheng},\n  journal={arXiv preprint arXiv:2202.10108},\n  year={2022}\n}\n```\n\n## Other Links\n\n\u003e **Image Classification**: See [ViTAE for Image Classification](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Image-Classification)\n\n\u003e **Object Detection**: See [ViTAE for Object Detection](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Object-Detection).\n\n\u003e **Semantic Segmentation**: See [ViTAE for Semantic Segmentation](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Semantic-Segmentation).\n\n\u003e **Animal Pose Estimation**: See [ViTAE for Animal Pose Estimation](https://github.com/ViTAE-Transformer/ViTAE-Transformer/tree/main/Animal-Pose-Estimation).\n\n\u003e **Matting**: See [ViTAE for Matting](https://github.com/ViTAE-Transformer/ViTAE-Transformer-Matting).\n\n\u003e **Remote Sensing**: See [ViTAE for Remote Sensing](https://github.com/ViTAE-Transformer/ViTAE-Transformer-Remote-Sensing).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvitae-transformer%2Fvitae-transformer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvitae-transformer%2Fvitae-transformer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvitae-transformer%2Fvitae-transformer/lists"}