Papers

• Transformer original paper

  • Attention is All You Need

• Technical blog

  • English Blog
  • Chinese Blog
  • Chinese Blog

• Survey

  • [paper - 2023.05.11
  • [paper - 2021.11.30
  • [paper - 2021.02.22
  • [paper - 2021.1.30
  • [paper - 2020.6.09

• arXiv papers

  • [paper
  • FocusedDecoder
  • TAG
  • [paper
  • MViTv2
  • [paper
  • [paper
  • DeiT III
  • [paper
  • MaxViT
  • V2X-ViT
  • MemMC-MAE - MAE)]
  • [paper - polyp)]
  • VideoMAE - NJU/VideoMAE)]
  • [paper
  • [paper
  • CrossEfficientViT - coccomini/Combining-EfficientNet-and-Vision-Transformers-for-Video-Deepfake-Detection)]
  • Discrete ViT
  • StyleSwin
  • SReT
  • [paper
  • [paper - research/scenic/tree/main/scenic/projects/token_learner)]
  • [paper
  • ORViT
  • [paper - Transformer)]
  • NViT
  • [paper
  • [paper
  • [paper - CV/CoTNet)]
  • TranSalNet
  • MobileViT
  • [paper
  • 3D-Transformer
  • CCTrans
  • UFO-ViT
  • [paper
  • [paper
  • OH-Former
  • Pix2seq
  • CoAtNet
  • LOTR
  • [paper
  • GraFormer
  • CDTrans
  • [paper - AIR-SUN/PQ-Transformer)]
  • [paper - model/AnchorDETR)]
  • ESRT
  • MaskFormer
  • SwinIR
  • Trans4Trans
  • [paper
  • [paper
  • [paper
  • [paper - Transformer)]
  • [paper
  • [paper - PVT)]
  • [paper
  • TriTransNet
  • PSViT
  • [paper
  • [paper
  • [paper
  • Styleformer - Park/Styleformer)]
  • CMT
  • TransAttUnet
  • [paper
  • ViTGAN
  • [paper
  • Trans4Trans
  • FFVT
  • TransformerFusion
  • [paper
  • [paper
  • [paper
  • XCiT
  • [paper - Transformer)]
  • [paper - Swin-Transformer)]
  • VOLO - sg/volo)]
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • CAT
  • [paper
  • DETReg
  • [paper
  • MViT
  • [paper
  • [paper
  • [paper
  • [paper - Transformer-for-Task-aware-Active-Learning)]
  • [paper
  • [paper
  • [paper
  • Refiner
  • ViTAE
  • [paper
  • [paper - liu/TransCNN)]
  • Uformer
  • [paper
  • RegionViT
  • [paper - x-yang/AOT)]
  • [paper
  • [paper - Transformer)]
  • [paper
  • DynamicViT - research.xyz/)]
  • [paper
  • [paper
  • TransMIL
  • TransVOS
  • KVT
  • MSG-Transformer - Transformer)]
  • SegFormer
  • SDNet - tfx/SDNet)]
  • DVT
  • GazeTR
  • [paper
  • [paper
  • FoveaTer
  • TransDA
  • [paper
  • PTNet
  • ResT
  • CogView
  • NesT
  • TAPG
  • [paper
  • COTR
  • TransVOD - LuHe/TransVOD)]
  • [paper
  • [paper
  • [paper
  • [paper
  • Swin-Unet - Unet)]
  • [paper - SSL)]
  • SCTN
  • RelationTrack
  • VGTR
  • PST
  • TrTr
  • MOTR - model/MOTR)]
  • [paper
  • TransHash
  • ISTR
  • CAT
  • CoSformer
  • [paper
  • PMTrans
  • HandsFormer
  • GasHis-Transformer
  • [paper
  • InTra
  • Twins - AutoML/Twins)]
  • MLMSPT
  • [paper
  • ConTNet - hao-tian/ConTNet)]
  • DTNet
  • [paper
  • [paper
  • M3DeTR
  • Skeletor
  • FaceT
  • MViT
  • VATT
  • So-ViT - ViT)]
  • [paper
  • TransRPPG
  • VideoGPT
  • M2TR
  • [paper
  • TransCrowd - liang/TransCrowd)]
  • [paper
  • [paper
  • [paper
  • TransGAN - Group/TransGAN)]
  • [paper
  • CoaT - ucsd/CoaT)]
  • LocalViT
  • CIT
  • [paper
  • SiT - Ahmed/SiT)]
  • [paper
  • [paper
  • [paper
  • AOT-GAN - GAN-for-Inpainting)]
  • [paper
  • ATAG
  • TubeR
  • AAformer
  • TFill
  • [paper - Free-3D)]
  • STGT
  • [paper
  • Meta-DETR - DETR)]
  • DA-DETR
  • [paper
  • [paper
  • TransUNet
  • CvT
  • TFPose
  • TransCenter
  • [paper
  • [paper
  • [paper
  • [paper
  • GSA-Net
  • [paper
  • DPT - isl/DPT)]
  • TransFG
  • TimeSformer
  • [paper
  • [paper - Transformers-without-Natural-Images/)]
  • [paper
  • CeiT
  • DeepViT
  • [paper
  • [paper
  • SUNETR
  • [paper
  • ConViT
  • TransMed
  • U-Transformer
  • SpecTr - xc-yun/SpecTr)]
  • TransBTS - 1119/TransBTS)]
  • [paper
  • [paper
  • CPVT - AutoML/CPVT)]
  • [paper
  • [paper
  • VTN
  • BoTNet
  • CPTR
  • [paper
  • Trans2Seg
  • [paper
  • Trear
  • VisualSparta
  • TrackFormer
  • TRIQ
  • TransTrack
  • DeiT
  • Pointformer
  • ViT-FRCNN
  • Taming-transformers - transformers/)]
  • SceneFormer
  • PCT
  • PED
  • C-Tran
  • MViTv2
  • [paper
  • [paper
  • DeiT III
  • [paper
  • [paper
  • [paper - CV/CoTNet)]
  • CoAtNet
  • [paper
  • [paper
  • XCiT
  • [paper
  • [paper
  • BossNAS

• 2022

  • P2T
  • X-CLIP - CLIP)]
  • TinyViT
  • AiATrack - Podi/AiATrack)]
  • OSTrack
  • Unicorn - IIAU/Unicorn)]
  • P3AFormer - research/ECCV22-P3AFormer-Tracking-Objects-as-Pixel-wise-Distributions)]
  • MAE
  • [paper - Transformer)]
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper - Transformer)
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • Restormer
  • SAM-DETR - DETR)]
  • BEVT
  • MobileFormer
  • STRM
  • MiniViT
  • DW-ViT - ViT)]
  • TokenFusion
  • CMT
  • [paper
  • TransMix
  • NomMer - NomMer)]
  • SSA - Transformer)]
  • RVT - Vision-Transformer)]
  • LVT - Yang/LVT)]
  • StyTr2 - 2)]
  • [paper
  • CrossFormer
  • [paper - X/UniFormer)]
  • BEVT
  • RelViT
  • DAB-DETR - opensource/DAB-DETR)]
  • MobileFormer
  • CMT
  • RelViT
  • TokenFusion
  • [paper
  • TransMix
  • RVT - Vision-Transformer)]
  • STRM
  • SSA - Transformer)]
  • FastMETRO - ami/FastMETRO)]
  • DW-ViT - ViT)]
  • NomMer - NomMer)]

• 2021

  • T2T-ViT - opensource/T2T-ViT)]
  • THUNDR
  • [paper
  • [paper - ProTo-Program-guided-Transformers-for-Program-guided-Tasks)]
  • Augvit - noah/CV-Backbones/tree/master/augvit_pytorch)]
  • YOLOS
  • CATs
  • Moment-DETR
  • [paper
  • Container
  • TNT - noah/noah-research/tree/master/TNT)]
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper
  • [paper - Transformer)]
  • ICT
  • PoinTr
  • STTR - transformer)]
  • TSP-FCOS
  • [paper
  • [paper
  • PVT
  • [paper
  • GLiT
  • TRAR
  • UniT
  • [paper
  • [paper
  • LocalTrans
  • [paper
  • [paper - transformer)]
  • GroupFormer
  • [paper
  • WB-DETR
  • [paper
  • [paper
  • PPT-Net - Net)]
  • [paper
  • [paper
  • [paper
  • COTR
  • MUSIQ
  • [paper
  • [paper
  • CrackFormer
  • HiT
  • [paper
  • STVGBert
  • HiFT
  • DocFormer
  • LeViT
  • SignBERT
  • VidTr
  • ACTOR
  • Segmenter
  • Visformer
  • PnP-DETR - detr)]
  • **VoTr**
  • TransVG
  • 3DETR
  • Eformer
  • TransFER
  • Oriented RCNN
  • ViViT
  • Stark
  • CT3D
  • VST
  • PiT - ai/pit)]
  • CrossViT
  • PointTransformer
  • TS-CAM - CAM.git)]
  • VTs
  • TransDepth
  • Conditional DETR
  • PIT - Position-Invariant-Transform)]
  • SOTR - cau/SOTR)]
  • SnowflakeNet
  • TransPose
  • TransReID
  • CWT - for-FSS)]
  • [paper
  • [paper
  • [paper - ViT)]
  • SMCA - replication)]
  • AutoFormer
  • [paper
  • HOTR
  • METRO
  • LETR
  • TransFuser
  • [paper
  • [paper
  • LoFTR
  • SETR - zvg.github.io/SETR/)]
  • TransT - dlut/TransT)]
  • [paper
  • VisTR
  • [paper - chefer/Transformer-Explainability)]
  • IPT
  • UP-DETR
  • IQT
  • [paper
  • [paper
  • GANsformer
  • NDT-Transformer
  • VTNet
  • Vision Transformer - research/vision_transformer)]
  • Deformable DETR - DETR)]
  • LAMBDANETWORKS - networks)]
  • [paper
  • HAT
  • [paper - Transformer)]
  • DPT - IVA-Lab/DPT)]
  • UTNet
  • MedT - maria-jose/Medical-Transformer)]
  • MCTrans
  • PNS-Net - Net)]
  • MBT-Net
  • ACT
  • TransFusion - Pose)]
  • [paper
  • DETR3D
  • [paper
  • [paper
  • PTT
  • LSTR
  • [paper
  • [paper
  • [paper
  • AutoFormer
  • TransDepth
  • [paper

• 2020

  • DETR
  • **FPT** - NJUST/FPT)]

• Other resource

  • [Awesome-Transformer-Attention

• Acknowledgement

  • Awesome-Crowd-Counting