{"id":74257,"url":"https://github.com/murufeng/awesome_vision_transformer","name":"awesome_vision_transformer","description":"Implementation of vision transformer. ⭐⭐⭐","projects_count":200,"last_synced_at":"2026-10-01T20:00:25.841Z","repository":{"id":47123883,"uuid":"366267911","full_name":"murufeng/Awesome_vision_transformer","owner":"murufeng","description":"Implementation of vision transformer. ⭐⭐⭐","archived":false,"fork":false,"pushed_at":"2021-10-26T05:54:27.000Z","size":203,"stargazers_count":34,"open_issues_count":0,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2026-09-12T00:06:37.585Z","etag":null,"topics":["deep-learning","paper","transformer","transformer-task","vision-transformers"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/murufeng.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-05-11T05:40:57.000Z","updated_at":"2026-04-14T09:17:08.000Z","dependencies_parsed_at":"2022-08-29T10:52:10.946Z","dependency_job_id":null,"html_url":"https://github.com/murufeng/Awesome_vision_transformer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/murufeng/Awesome_vision_transformer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/murufeng%2FAwesome_vision_transformer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/murufeng%2FAwesome_vision_transformer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/murufeng%2FAwesome_vision_transformer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/murufeng%2FAwesome_vision_transformer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/murufeng","download_url":"https://codeload.github.com/murufeng/Awesome_vision_transformer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/murufeng%2FAwesome_vision_transformer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":342085742,"owners_count":37908647,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-08-22T15:14:58.755Z","status":"online","status_checked_at":"2026-10-01T02:00:06.691Z","response_time":59,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"created_at":"2024-10-15T16:56:20.007Z","updated_at":"2026-10-01T20:00:25.842Z","primary_language":null,"list_of_lists":false,"displayable":true,"categories":["Table of Contents"],"sub_categories":["DETR变种","微软Transformer霸榜模型","Awesome Survey","Paper(最新，最受关注的)","[Papers](#paper)","MLP","论文解读"],"readme":"# Awesome vision transformer\nA curated list of vision transformer related resources, including survey, paper, source code, etc.\nMaintainer: Murufeng\n\nWe are looking for a maintainer! Let me know if interested.\n\nPlease feel free to pull requests or open an issue to add papers and source codes.\n\n## Table of Contents\n\n### [Awesome Survey](#Survey)\n### [优秀论文解读](#论文解读)\n### [Papers](#paper) \n#### [ViT系列变种](#ViT系列变种)\n- [魔改算子](#operator)\n- [Local \u0026 Hierarchical \u0026 multi-scale](#Local)\n- [Transformer+卷积结合](#CNN)\n- [Transformer模型压缩轻量化处理](#compress)\n- [DETR变种](#detr)\n\n#### MLP系列\n\n- [MLP-Mixer](https://github.com/murufeng/Awesome_vision_transformer/blob/main/model/MLP/MLP-Mixer.py)\n\n- [ResMLP](https://github.com/murufeng/Awesome_vision_transformer/blob/main/model/MLP/ResMLP-fb.py)\n\n- [RepMLP](https://arxiv.org/abs/2105.01883)\n\n- [gMLP](https://arxiv.org/abs/2105.08050) \n\n- [Spatial shift($S^{2}$) MLP V1](https://arxiv.org/abs/2106.07477)\n- [S^2-MLPv2: Improved Spatial-Shift MLP Architecture for Vision](https://arxiv.org/abs/2108.01072)\n- [ViP(Vision Permute-MLP)](https://arxiv.org/abs/2106.12368) \n- [GFNet | MLP领域再发力，清华大学提出将FFT思想用于空间信息交互](https://arxiv.org/abs/2107.00645) \n- [CycleMLP: A MLP-like Architecture for Dense Prediction](https://arxiv.org/abs/2107.10224)  [Github](https://github.com/ShoufaChen/CycleMLP)\n- [AS-MLP](https://arxiv.org/abs/2107.08391)\n\n#### [Transformer+各类task迁移](#task)\n- [1.目标检测（Object-Detection）](#1.目标检测)\n- [2.超分辨率（Super-Resolution）](#2.超分辨率)\n- [3.图像分割、语义分割(Segmentation)](#3.图像分割)\n- [4.GAN/生成式/对抗式(GAN/Generative/Adversarial)](#4.GAN)\n- [5.track](#5.track)\n- [6.video](#6.video)\n- [7.多模态结合](#7.multimodel)\n- [8.人体姿态估计](#8.姿态估计)\n- [9.神经网络架构搜索NAS](#9.NAS)\n- [10.人脸识别](#10.Face)\n- [11.行人重识别](#11.ReID)\n- [12.密集人群检测](#12.Crowd)\n- [13.医学图像处理](#13.medical)\n- [14.图像风格迁移](#14.imagestyle)\n- [15.low level vision(去噪，去雨，复原，去模糊等等)](#15.low_level_vision)\n\n#### [其它](#其它)\n\n### [模型代码复现](#code)\n- [MLP](#mlp)\n- [Attention](#attention)\n- [Transformer](#code_transformer)\n\n\n\u003ca name=\"Survey\"\u003e\u003c/a\u003e\n### Awesome Survey\n  - [A Survey of Transformers](https://arxiv.org/abs/2106.04554)\n    - 论文作者\u0026单位: 复旦大学邱锡鹏团队; Tianyang Lin, Yuxin Wang, Xiangyang Liu, Xipeng Qiu\n    - 时间: 2021.6.08\n  - [A Survey on Visual Transformer](https://arxiv.org/abs/2012.12556)  \n    - 论文作者\u0026单位: 华为诺亚方舟; Kai Han, Yunhe Wang, Hanting Chen, etc\n    - 2021.1.30\n  - [Transformers in Vision: A Survey](https://arxiv.org/abs/2101.01169) \n    - 论文作者:Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, Mubarak Shah\n    - 时间: 2021.1.4\n\n\u003ca name=\"论文解读\"\u003e\u003c/a\u003e\n\n### 论文解读\n- [一文总结微软研究院Transformer霸榜模型三部曲](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247511722\u0026idx=1\u0026sn=8e73570336b2d67948a6bce5b55d9c0d\u0026chksm=fc4d14e8cb3a9dfeac9f8322d250d1e043075df0717b2737ff8bc0f67c96ec5b029f7b876ce6\u0026token=460758099\u0026lang=zh_CN#rd)\n- [颜水成团队提出VOLO屠榜CV任务，无需额外训练数据，首次在ImageNet 上达到87.1%](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247510980\u0026idx=1\u0026sn=884b30ccdff2c6e7f44f0bc65c262199\u0026chksm=fc4d1186cb3a989008b5cd276c6438edd39c36a82b7575d08ac2b9bf367be759d3fae16fd075\u0026token=1173831238\u0026lang=zh_CN#rd)\n- [分层级联Transformer！苏黎世联邦提出TransCNN: 显著降低了计算/空间复杂度！](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247509814\u0026idx=2\u0026sn=4f83ae4f12cf95b61b346b0effe45ea5\u0026chksm=fc4d1d74cb3a9462a7cfdee53c5d9be59666ece6f1d836005d9f3d9da51a8dd113a66eff92f0\u0026token=2050016138\u0026lang=zh_CN#rd)\n- [登上更高峰！颜水成、程明明团队开源ViP，引入三维信息编码机制，无需卷积与注意力](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247510615\u0026idx=3\u0026sn=d0f60016ed9dc27624ffe092c3181a52\u0026chksm=fc4d1015cb3a990371d8a9da87c93ed6711952b8cb14db9fc0e85f8ffc947465cf229d6e97c7\u0026token=1173831238\u0026lang=zh_CN#rd)\n- [清华鲁继文团队提出DynamicViT：一种高效的动态稀疏化Token的ViT](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247509696\u0026idx=2\u0026sn=1fe82423987a0519413782b830d469d4\u0026chksm=fc4d1c82cb3a95943bee993c5363c47830b3d8e292d31aa93380b6f2721094740fe97c47679b\u0026token=2050016138\u0026lang=zh_CN#rd)\n- [并非所有图像都值16x16个词--- 清华\u0026华为提出一种自适应序列长度的动态ViT](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247509514\u0026idx=1\u0026sn=c30c5514a76185038da274b4c6876289\u0026chksm=fc4d1c48cb3a955eef07b72043e38a18828d6c7c784bd7560f97c66ede9fb7f67bc7e652712a\u0026token=2050016138\u0026lang=zh_CN#rd)\n- [注意力可以使MLP完全替代CNN吗？ 未来有哪些研究方向？](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247509411\u0026idx=1\u0026sn=20e63c8f160bc7929bc052455ea80603\u0026chksm=fc4d1fe1cb3a96f73501568926cc5d7e7442f3bc466fb52ee606c27e316c3d0538f083908d7e\u0026token=2050016138\u0026lang=zh_CN#rd)\n- [超越Swin Transformer！谷歌提出了收敛更快、鲁棒性更强、性能更强的NesT](https://mp.weixin.qq.com/s?__biz=MzU2NDExMzE5Nw==\u0026mid=2247509059\u0026idx=3\u0026sn=1e0af034d262e7b24105dab2b6e348d2\u0026chksm=fc4d1e01cb3a9717ceb25f7dc35cc1e91f1642862abfd0884699f007da902683457f54f8994c\u0026token=2050016138\u0026lang=zh_CN#rd)\n\n\u003ca name=\"paper\"\u003e\u003c/a\u003e\n### Paper(最新，最受关注的)\n- Visual Parser: Representing Part-whole Hierarchies with Transformers -- 牛津大学\n- Crossformer: A versatile vision Transformer based on cross-scale attention 浙大\u0026哥大\n- Contextual Transformer Networks for Visual Recognition - 京东AI梅涛团队\n- Rethinking and Improving Relative Position Encoding for Vision Transformer - 中山大学\u0026微软\n- Local-to-Global Self-Attention in Vision Transformers -Inception Institute of Artificial Intelligence(IIAI)\n- Polarized Self-Attention: Towards High-quality Pixel-wise Regression -- 新的自注意力机制，屠榜分割任务\n- A Unified Efficient Pyramid Transformer for Semantic Segmentation - 李沐 Amazon \u0026 复旦\n- 华为诺亚提出CMT：卷积神经网络遇见视觉Transformer \n   - paper : [CMT: Convolutional Neural Networks Meet Vision Transformers](https://arxiv.org/abs/2107.06263)\n- 超越Swin，Transformer屠榜三大视觉任务！微软推出新作：Focal Self-Attention [paper](https://arxiv.org/abs/2107.00641)\n- Multi-Scale Densenet续作？搞定Transformer降采样，清华联合华为开源动态ViT！ [paper](https://arxiv.org/abs/2105.15075)\n- CSWin Transformer：具有十字形窗口的视觉Transformer主干 [paper](https://arxiv.org/abs/2107.00652)\n-  Faceboo提出 [Early Convolutions Help Transformers See Better](https://arxiv.org/abs/2106.14881)\n- VOLO: Vision Outlooker for Visual Recognition\n  - 论文链接: [https://arxiv.org/abs/2106.13112](https://arxiv.org/abs/2106.13112)\n  - 代码地址: [https://github.com/sail-sg/volo](https://github.com/sail-sg/volo)\n  - 作者团队:颜水成，冯佳时  无需任何额外训练数据，首次在ImageNet数据集上实现87.1%精度\n- Vision Permutator: A Permutable MLP-Like Architecture for Visual Recognition\n    - paper链接：[https://arxiv.org/abs/2106.12368](https://arxiv.org/abs/2106.12368)\n    - 代码地址：[https://github.com/Andrew-Qibin/VisionPermutator](https://github.com/Andrew-Qibin/VisionPermutator)\n- [Scaling Vision Transformers](https://arxiv.org/abs/2106.04560)\n- [CAT: Cross Attention in Vision Transformer](https://arxiv.org/abs/2106.05786)\n- [CoAtNet: Marrying Convolution and Attention for All Data Sizes](https://arxiv.org/abs/2106.04803)\n  - 作者单位：谷歌大佬;Zihang Dai, Hanxiao Liu, Quoc V. Le, Mingxing Tan\n- [Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer](https://arxiv.org/abs/2106.03650)\n- [Container: Context Aggregation Network](https://arxiv.org/abs/2106.01401)\n- [Aggregating Nested Transformers](https://arxiv.org/abs/2105.12723)\n- [X-volution: On the unification of convolution and self-attention](https://arxiv.org/abs/2106.02253)\n- [Video Swin Transformer](https://arxiv.org/abs/2106.13230)\n- [Dynamic Head: Unifying Object Detection Heads with Attentions](https://arxiv.org/abs/2106.08322)\n\n### 微软Transformer霸榜模型\n\n- **超越Swin，Transformer屠榜三大视觉任务！微软推出新作：Focal Self-Attention** \n   - [paper](https://arxiv.org/abs/2107.00641)\n   - [code]()\n- **CSWin Transformer：具有十字形窗口的视觉Transformer主干(Swin Transformer 的进阶版) *\n   - [paper](https://arxiv.org/abs/2107.00652)\n   - [code]()\n- **[Dynamic Head: Unifying Object Detection Heads with Attentions](检测霸榜模型)**\n   - [paper](https://arxiv.org/abs/2106.08322)\n   - [code](https://github.com/microsoft/DynamicHead)\n- **[Swin Transformer]** Swin Transformer: Hierarchical Vision Transformer using Shifted Windows\n  - [paper](https://arxiv.org/abs/2103.14030)\n  - [code](https://github.com/microsoft/Swin-Transformer)\n\n\n\u003ca name=\"ViT系列变种\"\u003e\u003c/a\u003e\n#### ViT系列变种\n\u003ca name=\"operator\"\u003e\u003c/a\u003e\n#### 魔改算子\n- **[VTs]**: [Visual Transformers: Token-based Image Representation and Processing for Computer Vision](https://arxiv.org/abs/2006.03677)]\n- **[So-ViT]**: So-ViT: Mind Visual Tokens for Vision Transformer \n    - [paper](https://arxiv.org/abs/2104.10935)\n    - [code](https://github.com/jiangtaoxie/So-ViT)\n- **[Token Labeling]** Token Labeling: Training a 85.5% Top-1 Accuracy Vision Transformer with 56M Parameters on ImageNet \n   - [paper](https://arxiv.org/abs/2104.10858)\n   - [code](https://github.com/zihangJiang/TokenLabeling)\n- **[LeViT]** [LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference](https://arxiv.org/abs/2104.01136)]\n- **[CrossViT]** [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://arxiv.org/abs/2103.14899)\n- **[CeiT]** [Incorporating Convolution Designs into Visual Transformers](https://arxiv.org/abs/2103.11816)\n- **[DeepViT]** [DeepViT: Towards Deeper Vision Transformer](https://arxiv.org/abs/2103.11886)\n- **[TNT]** Transformer in Transformer \n   - [paper](https://arxiv.org/abs/2103.00112)\n   - [code](https://github.com/huawei-noah/noah-research/tree/master/TNT)\n- **[T2T-ViT]** Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet \n   - [paper](https://arxiv.org/abs/2101.11986)\n   - [code](https://github.com/yitu-opensource/T2T-ViT)\n- **[BoTNet]** [Bottleneck Transformers for Visual Recognition](https://arxiv.org/abs/2101.11605)]\n- **[Visformer]** Visformer: The Vision-friendly Transformer \n   - [paper](https://arxiv.org/abs/2104.12533)\n   - [code](https://github.com/danczs/Visformer)\n- **[ConTNet]** ConTNet: Why not use convolution and transformer at the same time? \n   - [paper](https://arxiv.org/abs/2104.13497)\n   - [code](https://github.com/yan-hao-tian/ConTNet)\n- **[DeiT]**: Training data-efficient image transformers \u0026 distillation through attention \n   - [paper](https://arxiv.org/abs/2012.12877)\n   - [code](https://github.com/facebookresearch/deit)\n\n\u003ca name=\"Local\"\u003e\u003c/a\u003e\n#### Local \u0026 Hierarchical \u0026 multi-scale\n- **[Twins]** Twins: Revisiting Spatial Attention Design in Vision Transformers\n  - [paper](https://arxiv.org/abs/2104.13840)\n  - [code](https://github.com/Meituan-AutoML/Twins)\n- [Scaling Vision Transformers](https://arxiv.org/abs/2106.04560)\n- **[GasHis-Transformer]** [GasHis-Transformer: A Multi-scale Visual Transformer Approach for Gastric Histopathology Image Classification](https://arxiv.org/abs/2104.14528) \n- **[Vision Transformer]** An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (**ICLR**)\n  - [paper](https://arxiv.org/abs/2010.11929)\n  - [code](https://github.com/google-research/vision_transformer)\n- **[RegionViT]** [Regional-to-Local Attention for Vision Transformers](https://arxiv.org/abs/2106.02689)\n- **[PVT]** [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://arxiv.org/abs/2102.12122)] \n   - [code](https://github.com/whai362/PVT)\n- **[FPT]** Feature Pyramid Transformer (**CVPR**) \n  - [paper](https://arxiv.org/abs/2007.09451)\n  - [code](https://github.com/ZHANGDONG-NJUST/FPT)\n- **[PiT]** Rethinking Spatial Dimensions of Vision Transformers \n  - [[paper](https://arxiv.org/abs/2103.16302)\n  - [code](https://github.com/naver-ai/pit)\n- **[CoaT]** Co-Scale Conv-Attentional Image Transformers\n  - [paper](https://arxiv.org/abs/2104.06399)\n  - [code](https://github.com/mlpc-ucsd/CoaT)\n- **[LocalViT]** LocalViT: Bringing Locality to Vision Transformers\n  - [paper](https://arxiv.org/abs/2104.05707)\n  - [code](https://github.com/ofsoundof/LocalViT)\n- **[Swin Transformer]** Swin Transformer: Hierarchical Vision Transformer using Shifted Windows\n  - [paper](https://arxiv.org/abs/2103.14030)\n  - [code](https://github.com/microsoft/Swin-Transformer)\n- **[DPT]** Vision Transformers for Dense Prediction \n  - [paper](https://arxiv.org/abs/2103.13413)\n  - [code](https://github.com/intel-isl/DPT)\n- **[MViT]** [Mask Vision Transformer for Facial Expression Recognition in the wild](https://arxiv.org/abs/2106.04520)]\n\n\n\u003ca name=\"CNN\"\u003e\u003c/a\u003e\n#### Transformer+卷积结合\n- **[Shuffle Transformer]** [Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformer](https://arxiv.org/abs/2106.03650)\n- **[TransCNN]** Transformer in Convolutional Neural Networks\\\n  - [paper](https://arxiv.org/abs/2106.03180)\n  - [code](https://github.com/yun-liu/TransCNN)\n- **[ResT]** ResT: An Efficient Transformer for Visual Recognition\n  - [paper](https://arxiv.org/abs/2105.13677)\n  - [code](https://github.com/wofmanaf/ResT)\n- **[CPVT]** Do We Really Need Explicit Position Encodings for Vision Transformers? \n   - [paper](https://arxiv.org/abs/2102.10882)\n   - [code](https://github.com/Meituan-AutoML/CPVT)\n- **[ConViT]** [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://arxiv.org/abs/2103.10697) \n- **[CoaT]** Co-Scale Conv-Attentional Image Transformers\n  - [paper](https://arxiv.org/abs/2104.06399)\n  - [code](https://github.com/mlpc-ucsd/CoaT)\n- **[CvT]** CvT: Introducing Convolutions to Vision Transformers \n  - [paper](https://arxiv.org/abs/2103.15808)\n  - [code](https://github.com/leoxiaobin/CvT)\n- **[ConTNet]** ConTNet: Why not use convolution and transformer at the same time?\n  - [paper](https://arxiv.org/abs/2104.13497)\n  - [code](https://github.com/yan-hao-tian/ConTNet)\n- **[CeiT]** [Incorporating Convolution Designs into Visual Transformers](https://arxiv.org/abs/2103.11816)\n- **[BoTNet]** [Bottleneck Transformers for Visual Recognition](https://arxiv.org/abs/2101.11605)\n- **[CPTR]** [CPTR: Full Transformer Network for Image Captioning](https://arxiv.org/abs/2101.10804)\n\n\u003ca name=\"compress\"\u003e\u003c/a\u003e\n#### Transformer模型压缩轻量化处理\n- **[DynamicViT]**: Efficient Vision Transformers with Dynamic Token Sparsification\n  - [paper](https://arxiv.org/abs/2106.02034)\n  - [code](https://dynamicvit.ivg-research.xyz/)\n- **[DVT]** [Not All Images are Worth 16x16 Words: Dynamic Vision Transformers with Adaptive Sequence Length](https://arxiv.org/abs/2105.15075)\n- **[LeViT]** [LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inference](https://arxiv.org/abs/2104.01136)\n\n\u003ca name=\"detr\"\u003e\u003c/a\u003e\n### DETR变种\n- **[UP-DETR]** [UP-DETR: Unsupervised Pre-training for Object Detection with Transformers (**CVPR**)](https://arxiv.org/abs/2011.09094)\n- **[Deformable DETR]** Deformable DETR: Deformable Transformers for End-to-End Object Detection (**ICLR**)\n  - [paper](https://arxiv.org/abs/2010.04159)\n  - [code](https://github.com/fundamentalvision/Deformable-DETR)\n- **[DETR]** End-to-End Object Detection with Transformers (**ECCV**) \n  - [paper](https://arxiv.org/abs/2005.12872)\n  - [code](https://github.com/facebookresearch/detr)\n- **[Meta-DETR]** Meta-DETR: Few-Shot Object Detection via Unified Image-Level Meta-Learning \n  - [paper](https://arxiv.org/abs/2103.11731)\n  - [code](https://github.com/ZhangGongjie/Meta-DETR)\n- **[DA-DETR]** [DA-DETR: Domain Adaptive Detection Transformer by Hybrid Attention](https://arxiv.org/abs/2103.17084)\n- **[DETReg]** Unsupervised Pretraining with Region Priors for Object Detection\n  - [paper](https://arxiv.org/abs/2106.04550)\n  - [code](https://amirbar.net/detreg)\n  \n\u003ca name=\"task\"\u003e\u003c/a\u003e\n#### Transformer+各类task迁移\n\u003ca name=\"1.目标检测\"\u003e\u003c/a\u003e\n#### Transformer+目标检测\n- **[Pointformer]** [3D Object Detection with Pointformer](https://arxiv.org/abs/2012.11409) \n- **[ViT-FRCNN]** [Toward Transformer-Based Object Detection](https://arxiv.org/abs/2012.09958)\n- [Oriented Object Detection with Transformer](https://arxiv.org/abs/2106.03146)\n- **[YOLOS]** You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection \n  - [paper](https://arxiv.org/abs/2106.00666)\n  - [code](https://github.com/hustvl/YOLOS)\n- **[COTR]** [COTR: Convolution in Transformer Network for End to End Polyp Detection](https://arxiv.org/abs/2105.10925)\n- **[TransVOD]** End-to-End Video Object Detection with Spatial-Temporal Transformers \n  - [paper](https://arxiv.org/abs/2105.10920)\n  - [code](https://github.com/SJTU-LuHe/TransVOD)\n- **[CAT]** [CAT: Cross-Attention Transformer for One-Shot Object Detection](https://arxiv.org/abs/2104.14984)\n- **[M2TR]** [M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection](https://arxiv.org/abs/2104.09770)\n- [Transformer Transforms Salient Object Detection and Camouflaged Object Detection](https://arxiv.org/abs/2104.10127)\n- **[SSTN]** [SSTN: Self-Supervised Domain Adaptation Thermal Object Detection for Autonomous Driving](https://arxiv.org/abs/2103.03150) \n- **[TSP-FCOS]** [Rethinking Transformer-based Set Prediction for Object Detection](https://arxiv.org/abs/2011.10881)\n- **[ACT]** [End-to-End Object Detection with Adaptive Clustering Transformer](https://arxiv.org/abs/2011.09315)\n- **[PED]** [DETR for Pedestrian Detection](https://arxiv.org/abs/2012.06785)\n- **[DPT]** Vision Transformers for Dense Prediction\n  - [paper](https://arxiv.org/abs/2103.13413)\n  - [code](https://github.com/intel-isl/DPT)\n\n\u003ca name=\"2.超分辨率\"\u003e\u003c/a\u003e\n#### 2.超分辨率（Super-Resolution)\n- **[TTSR]** Learning Texture Transformer Network for Image Super-Resolution (**CVPR**)\n  - [paper](https://arxiv.org/abs/2006.04139)\n  - [code](https://github.com/researchmm/TTSR)\n\n\u003ca name=\"3.图像分割\"\u003e\u003c/a\u003e\n\n#### 3. 图像分割、语义分割(Segmentation)\n- [Fully Transformer Networks for Semantic ImageSegmentation](https://arxiv.org/abs/2106.04108)\n- **[TransVOS]**  [TransVOS: Video Object Segmentation with Transformers](https://arxiv.org/abs/2106.00588)\n- **[SegFormer]** SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers \n  - [paper](https://arxiv.org/abs/2105.15203)\n  - [code](https://github.com/NVlabs/SegFormer)\n- **[VisTR]** [End-to-End Video Instance Segmentation with Transformers (**CVPR**)](https://arxiv.org/abs/2011.14503)\n- **[Trans2Seg]**  Segmenting Transparent Object in the Wild with Transformer \n  - [paper](https://arxiv.org/abs/2101.08461)\n  - [code](https://github.com/xieenze/Trans2Seg)\n- SETR ：Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers\n  - 作者单位：复旦, 牛津大学, 萨里大学, 腾讯优图, Facebook\n  - 主页：[https://fudan-zvg.github.io/SETR/](https://fudan-zvg.github.io/SETR/)\n  - 代码：[https://github.com/fudan-zvg/SETR](https://github.com/fudan-zvg/SETR)\n  - 论文：[https://arxiv.org/abs/2012.15840](https://arxiv.org/abs/2012.15840)\n\n\u003ca name=\"4.GAN\"\u003e\u003c/a\u003e\n#### 4.GAN/生成式/对抗式(GAN/Generative/Adversarial)\n- **[GANsformer]**  Generative Adversarial Transformers\n  - [链接](https://arxiv.org/abs/2103.01209)\n  - [code](https://github.com/dorarad/gansformer)\n\n- **[TransGAN]**: Two Transformers Can Make One Strong GAN\n  - [链接](https://arxiv.org/abs/2102.07074)\n  - [Code](https://github.com/VITA-Group/TransGAN)\n  \n- **[AOT-GAN]** Aggregated Contextual Transformations for High-Resolution Image Inpainting \n  - [paper](https://arxiv.org/abs/2104.01431) \n  - [code](https://github.com/researchmm/AOT-GAN-for-Inpainting)\n  \n  \n\u003ca name=\"5.track\"\u003e\u003c/a\u003e\n#### 5.track\n- **[STGT]** [Spatial-Temporal Graph Transformer for Multiple Object Tracking](https://arxiv.org/abs/2104.00194)\n- [Transformer Tracking](https://arxiv.org/abs/2103.15436)\n- **[TransCenter]** [TransCenter: Transformers with Dense Queries for Multiple-Object Tracking](https://arxiv.org/abs/2103.15145)\n- **[TrackFormer]** [TrackFormer: Multi-Object Tracking with Transformers](https://arxiv.org/abs/2101.02702)\n- **[TransTrack]** TransTrack: Multiple-Object Tracking with Transformer\n  - [paper](https://arxiv.org/abs/2012.15460) \n  - [code](https://github.com/PeizeSun/TransTrack)\n\n\n\u003ca name=\"6.video\"\u003e\u003c/a\u003e\n#### 6.video\n- [Video Swin Transformer](https://arxiv.org/abs/2106.13230)\n- Anticipative Video Transformer\n  - [paper](https://arxiv.org/abs/2106.02036)\n  - [code](http://facebookresearch.github.io/AVT)\n- **[TimeSformer]** [Is Space-Time Attention All You Need for Video Understanding?](https://arxiv.org/abs/2102.05095)\n- **[VidTr]** [VidTr: Video Transformer Without Convolutions](https://arxiv.org/abs/2104.11746)\n- **[ViViT]** [ViViT: A Video Vision Transformer](https://arxiv.org/abs/2103.15691)\n- **[VTN]** [Video Transformer Network](https://arxiv.org/abs/2102.00719)\n- **[VisTR]** [End-to-End Video Instance Segmentation with Transformers (**CVPR**)](https://arxiv.org/abs/2011.14503)\n- **[STTN]** Learning Joint Spatial-Temporal Transformations for Video Inpainting (**ECCV**)\n  - [paper](https://arxiv.org/abs/2007.10247)\n  - [code](https://github.com/researchmm/STTN)\n\n\n\u003ca name=\"7.multimodel\"\u003e\u003c/a\u003e\n#### 7.多模态结合\n- ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision\n  - 论文：https://arxiv.org/abs/2102.03334\n  - 代码：https://github.com/dandelin/ViLT\n\n\u003ca name=\"8.姿态估计\"\u003e\u003c/a\u003e\n#### 8.人体姿态估计\n- **[TransPose]** [TransPose: Towards Explainable Human Pose Estimation by Transformer](https://arxiv.org/abs/2012.14214)\n- **[TFPose]** [TFPose: Direct Human Pose Estimation with Transformers](https://arxiv.org/abs/2103.15320)\n- [Lifting Transformer for 3D Human Pose Estimation in Video](https://arxiv.org/abs/2103.14304)\n\n\u003ca name=\"9.NAS\"\u003e\u003c/a\u003e\n#### 9.神经网络架构搜索NAS\n- **[BossNAS]** BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Search \n  - [paper](https://arxiv.org/abs/2103.12424)\n  - [code](https://github.com/changlin31/BossNAS)\n- [Vision Transformer Architecture Search](https://arxiv.org/abs/2106.13700)\n\n\u003ca name=\"10.Face\"\u003e\u003c/a\u003e\n#### 10.人脸识别\n- **[FaceT]**: [Learning to Cluster Faces via Transformer](https://arxiv.org/abs/2104.11502)\n\n\u003ca name=\"11.ReID\"\u003e\u003c/a\u003e\n#### 11.行人重识别\n- **[TransReID]** [TransReID: Transformer-based Object Re-Identification](https://arxiv.org/abs/2102.04378)\n\n\u003ca name=\"12.Crowd\"\u003e\u003c/a\u003e\n#### 12.密集人群检测\n- **[TransCrowd]** TransCrowd: Weakly-Supervised Crowd Counting with Transformer \n  - [paper](https://arxiv.org/abs/2104.09116)\n  - [code](https://github.com/dk-liang/TransCrowd)\n\n\u003ca name=\"13.medical\"\u003e\u003c/a\u003e\n#### 13.医学图像处理\n- **[SUNETR]** [SUNETR: Transformers for 3D Medical Image Segmentation](https://arxiv.org/abs/2103.10504)] \n- **[U-Transformer]** [U-Net Transformer: Self and Cross Attention for Medical Image Segmentation](https://arxiv.org/abs/2103.06104)\n- **[MedT]** Medical Transformer: Gated Axial-Attention for Medical Image Segmentation \n  - [paper](https://arxiv.org/abs/2102.10662)\n  - [code](https://github.com/jeya-maria-jose/Medical-Transformer)\n- **[TransUNet]** TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation \n  - [paper](https://arxiv.org/abs/2102.04306)\n  - [code](https://github.com/Beckschen/TransUNet)\n\n\u003ca name=\"14.imagestyle\"\u003e\u003c/a\u003e\n#### 14.图像风格迁移\n- StyTr2: Unbiased Image Style Transfer with Transformers\n  - [paper](https://arxiv.org/abs/2105.14576)\n\n\u003ca name=\"15.low_level_vision\"\u003e\u003c/a\u003e\n#### 15.low level vision(去噪，去雨，复原，去模糊等等)\n- **[IPT]** [Pre-Trained Image Processing Transformer (**CVPR**)](https://arxiv.org/abs/2012.00364)\n- **[SDNet]**: mutil-branch for single image deraining using swin\n  - [paper](https://arxiv.org/abs/2105.15077)\n  - [code](https://github.com/H-tfx/SDNet)\n- Uformer: A General U-Shaped Transformer for Image Restoration\n  - [paper](https://arxiv.org/abs/2106.03106)\n  - [code](https://github.com/ZhendongWang6/Uformer)\n\n\u003ca name=\"其它\"\u003e\u003c/a\u003e\n#### 其它\n- [Chasing Sparsity in Vision Transformers:An End-to-End Exploration](https://arxiv.org/abs/2106.04533)\n- [MViT: Mask Vision Transformer for Facial Expression Recognition in the wild](https://arxiv.org/abs/2106.04520)\n- **[CPTR]** [CPTR: Full Transformer Network for Image Captioning](https://arxiv.org/abs/2101.10804)\n- Learn to Dance with AIST++: Music Conditioned 3D Dance Generation\n  - [paper](https://arxiv.org/abs/2101.08779)\n  - [code](https://google.github.io/aichoreographer/)\n- [Deepfake Video Detection Using Convolutional Vision Transformer](https://arxiv.org/abs/2102.11126)\n- [Training Vision Transformers for Image Retrieval](https://arxiv.org/abs/2102.05644)\n\n\n\u003ca name=\"code\"\u003e\u003c/a\u003e\n### 模型代码复现\n\n\u003ca name=\"mlp\"\u003e\u003c/a\u003e\n### MLP\n- [MLP-Mixer](https://github.com/murufeng/Awesome_vision_transformer/blob/main/model/MLP/MLP-Mixer.py)\n\n- [ResMLP](https://github.com/murufeng/Awesome_vision_transformer/blob/main/model/MLP/ResMLP-fb.py)\n\n- RepMLP\n\n- gMLP \n\n- Spatial shift($S^{2}) MLP\n\n- ViP(Vision Permute-MLP)\n\n\u003ca name=\"attention\"\u003e\u003c/a\u003e\n### Attention\n\n- 待更新\n\n\u003ca name=\"code_transformer\"\u003e\u003c/a\u003e\n### Transformer\n\n- 待更新\n","projects_url":"https://awesome.ecosyste.ms/api/v1/lists/murufeng%2Fawesome_vision_transformer/projects"}