awesome-action-recognition

A curated list of action recognition and related area resources
https://github.com/eric-erki/awesome-action-recognition

Last synced: 17 days ago
JSON representation

Action Recognition and Video Understanding
- Spatio-Temporal Action Detection
  - Incremental Tube Construction for Human Action Detection - H. S. Behl et al, arXiv2017.
  - Multi-region two-stream R-CNN for action detection - [X. Peng](http://xjpeng.weebly.com/) and C. Schmid. ECCV2016. [[code]](https://github.com/pengxj/action-faster-rcnn)
  - Spot On: Action Localization from Pointly-Supervised Proposals - P. Mettes et al, ECCV2016.
  - Deep Learning for Detecting Multiple Space-Time Action Tubes in Videos - S. Saha et al, BMVC2016. [[code]](https://bitbucket.org/sahasuman/bmvc2016_code) [[project web]](http://sahasuman.bitbucket.org/bmvc2016/)
  - Finding Action Tubes - G. Gkioxari and J. Malik CVPR2015. [[code]](https://github.com/gkioxari/ActionTubes) [[project web]](https://people.eecs.berkeley.edu/~gkioxari/ActionTubes/)
  - APT: Action localization proposals from dense trajectories - J. Gemert et al, BMVC2015. [[code]](https://github.com/jvgemert/apt)
  - Spatiotemporal deformable part models for action detection - [Y. Tian](http://www.cs.ucf.edu/~ytian/index.html) et al, CVPR2013. [[code]](http://www.cs.ucf.edu/~ytian/sdpm.html)
  - A Better Baseline for AVA - R. Girdhar et al., ActivityNet Workshop, CVPR2018.
  - Real-Time End-to-End Action Detection with Two-Stream Networks - A. El-Nouby and G. Taylor, arXiv2018.
  - Human Action Localization with Sparse Spatial Supervision - P. Weinzaepfel et al., arXiv2017.
  - Spatial-Aware Object Embeddings for Zero-Shot Localization and Classification of Actions - P. Mettes and C. G. M. Snoek, ICCV2017.
  - Action Tubelet Detector for Spatio-Temporal Action Localization - V. Kalogeiton et al, ICCV2017. [[code]](https://github.com/vkalogeiton/caffe/tree/act-detector) [[project web]](http://thoth.inrialpes.fr/src/ACTdetector/)
  - Tube Convolutional Neural Network (T-CNN) for Action Detection in Videos - [R. Hou](http://www.cs.ucf.edu/~rhou/) et al, ICCV2017. [[project web]](http://crcv.ucf.edu/projects/TCNN/)
  - Chained Multi-stream Networks Exploiting Pose, Motion, and Appearance for Action Classification and Detection - M. Zolfaghari et al, ICCV2017. [[project web]](https://lmb.informatik.uni-freiburg.de/projects/action_chain/)
  - Online Real time Multiple Spatiotemporal Action Localisation and Prediction - [G. Singh](http://gurkirt.github.io/) et al, ICCV2017. [[code]](https://github.com/gurkirt/realtime-action-detection)
  - AMTnet: Action-Micro-Tube regression by end-to-end trainable deep architecture - S. Saha et al, ICCV2017.
  - Am I Done? Predicting Action Progress in Videos - F. Becattini et al, BMVC2017.
  - Generic Tubelet Proposals for Action Localization - J. He et al, arXiv2017.
- Ego-Centric Action Recognition
  - Actor and Observer: Joint Modeling of First and Third-Person Videos - G. Sigurdsson et al., CVPR2018. [[code]](https://github.com/gsig/actor-observer)
- Miscellaneous
  - What and How Well You Performed? A Multitask Learning Approach to Action Quality Assessment - P. Parma and B. T. Morris. CVPR2019.
  - CortexNet: a Generic Network Family for Robust Visual Temporal Representations - arXiv2017. [[code]](https://github.com/atcold/pytorch-CortexNet) [[project web]](https://engineering.purdue.edu/elab/CortexNet/)
  - Slicing Convolutional Neural Network for Crowd Video Understanding - J. Shao et al, CVPR2016. [[code]](https://github.com/amandajshao/Slicing-CNN)
  - Two-Stream (RGB and Flow) pretrained model weights
- Action Recognition Datasets
  - Video Dataset Overview from Antoine Miech
  - HACS
  - Moments in Time
  - AVA
  - Kinetics
  - OOPS - A dataset of unintentional action, [paper](https://arxiv.org/abs/1911.11206)
  - COIN - a large-scale dataset for comprehensive instructional video analysis, [paper](https://arxiv.org/abs/1903.02874)
  - 20BN-JESTER - SOMETHING-SOMETHING](https://www.twentybn.com/datasets/something-something)
  - ActivityNet
  - Charades
  - Charades-Ego - First person and third person video aligned dataset
  - EPIC-Kitchens - First person videos recorded in kitchens. Note they provide download scripts and a python library [here](https://github.com/epic-kitchens)
  - Sports-1M - Large scale action recognition dataset.
  - THUMOS14 - 101](http://crcv.ucf.edu/data/UCF101.php) dataset.
  - THUMOS15 - 101](http://crcv.ucf.edu/data/UCF101.php) dataset.
  - UCF-101 - 14](http://crcv.ucf.edu/ICCV13-Action-Workshop/index.files/UCF101_24Action_Detection_Annotations.zip), and [corrupted annotation list](https://github.com/jinwchoi/Jinwoo-Computer-Vision-and-Machine-Learing-papers-to-read/blob/master/UCF101_Spatial_Annotation_Corrupted_file_list), [UCF-101 corrected annotations](https://github.com/gurkirt/corrected-UCF101-Annots) and [different version annotaions](https://github.com/jvgemert/apt). And there are also some pre-computed spatiotemporal action detection [results](https://drive.google.com/drive/folders/0B-LzM05qEdk0aG5pTE94VFI1SUk)
  - UCF-50
  - UCF-Sports
  - J-HMDB
  - MSR Action
  - Sports Videos in the Wild
  - Mixamo Mocap Dataset
  - UWA3D Multiview Activity II Dataset
  - Northwestern-UCLA Dataset
- Video Annotation
  - Efficiently scaling up crowdsourced video annotation - C. Vondrick et. al, IJCV2013. [[code]](https://github.com/cvondrick/vatic)
  - The Design and Implementation of ViPER - D. Mihalcik and D. Doermann, Technical report.
- Skeleton-Based Action Classification
  - An Attention Enhanced Graph Convolutional LSTM Network for Skeleton-Based Action Recognition - C. Si et al., CVPR2019.
  - View Adaptive Neural Networks for High Performance Skeleton-Based Human Action Recognition - P. Zhang et al., TPAMI2019.
  - Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition - S. Yan et al., AAAI2018. [[code]](https://github.com/yysijie/st-gcn)
  - Part-based Graph Convolutional Network for Action Recognition - K. Thakkar et al., BMVC2018.
- Temporal Action Detection
  - Rethinking the Faster R-CNN Architecture for Temporal Action Localization - Yu-Wei Chao et al., CVPR2018
  - End-to-End, Single-Stream Temporal Action Detection in Untrimmed Videos - Shayamal Buch et al., BMVC 2017 [[code]](https://github.com/shyamal-b/ss-tad)
  - Cascaded Boundary Regression for Temporal Action Detection - Jiyang Gao et al., BMVC 2017 [[code](https://github.com/jiyanggao/CBR)]
  - Temporal Action Detection with Structured Segment Networks - Y. Zhao et al., ICCV2017. [[code]](https://github.com/yjxiong/action-detection) [[project web]](http://yjxiong.me/others/ssn/)
  - Temporal Context Network for Activity Localization in Videos - X. Dai et al., ICCV2017.
  - Detecting the Moment of Completion: Temporal Models for Localising Action Completion - F. Heidarivincheh et al., arXiv2017.
  - SST: Single-Stream Temporal Action Proposals - S. Buch et al, CVPR2017. [[code]](https://github.com/shyamal-b/sst)
  - R-C3D: Region Convolutional 3D Network for Temporal Activity Detection - H. Xu et al, arXiv2017. [[code]](https://github.com/VisionLearningGroup/R-C3D) [[project web]](http://ai.bu.edu/r-c3d/) [[PyTorch]](https://github.com/sunnyxiaohu/R-C3D.pytorch)
  - DAPs: Deep Action Proposals for Action Understanding - V. Escorcia et al, ECCV2016. [[code]](https://github.com/escorciav/daps) [[raw data]](https://github.com/escorciav/daps)
  - Online Action Detection using Joint Classification-Regression Recurrent Neural Networks - Y. Li et al, ECCV2016. Noe: RGB-D Action Detection
  - Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs - Z. Shou et al, CVPR2016. [[code]](https://github.com/zhengshou/scnn) Note: Aka S-CNN.
  - Actionness Estimation Using Hybrid Fully Convolutional Networks - L. Wang et al, CVPR2016. [[code]](https://github.com/wanglimin/actionness-estimation/) Note: The code is not a complete verision. It only contains a demo, not training. [[project web]](http://wanglimin.github.io/actionness_hfcn/index.html)
  - Learning Activity Progression in LSTMs for Activity Detection and Early Detection - S. Ma et al, CVPR2016.
  - End-to-end Learning of Action Detection from Frame Glimpses in Videos - S. Yeung et al, CVPR2016. [[code]](https://github.com/syyeung/frameglimpses) [[project web]](http://ai.stanford.edu/~syyeung/frameglimpses.html) Note: This method uses reinforcement learning
  - Bag-of-fragments: Selecting and encoding video fragments for event detection and recounting - P. Mettes et al, ICMR2015.
  - Weakly Supervised Action Localization by Sparse Temporal Pooling Network - Phuc Nguyen et al., CVPR 2018
- Video Representation
  - Learning Correspondence from the Cycle-consistency of Time - X. Wang et al., CVPR2019. [[code]](https://github.com/xiaolonw/TimeCycle) [[project web]](https://ajabri.github.io/timecycle/)
  - Representation Flow for Action Recognition - AJ. Piergiovanni and M. S. Ryoo et al., CVPR2019.
  - Collaborative Spatiotemporal Feature Learning for Video Action Recognition - C. Li et al., CVPR2019.
  - Learning Video Representations from Correspondence Proposals - X. Liu et al., CVPR2019.
  - Timeception for Complex Action Recognition - N. Hussein et al., CVPR2019.
  - The Visual Centrifuge: Model-Free Layered Video Representations - J.-B. Alayrac et al., CVPR2019.
  - Long-Term Feature Banks for Detailed Video Understanding - C.-Y. Wu. et al., CVPR2019. [[code]](https://github.com/facebookresearch/video-long-term-feature-banks/)
  - Temporal Relational Reasoning in Videos - B. Zhou et al., ECCV2018. [[code]](https://github.com/metalbubble/TRN-pytorch) [[project web]](http://relation.csail.mit.edu/)
  - Videos as Space-Time Region Graphs - X. Wang and A. Gupta, ECCV2018.
  - Video Classification with Channel-Separated Convolutional Networks - D. Tran et al., arXiv2019.
  - DistInit: Learning Video Representations without a Single Labeled Video - R. Girdhar et al., arXiv2019.
  - SCSampler: Sampling Salient Clips from Video for Efficient Action Recognition - B. Korbar et al., arXiv2019.
  - Video Action Transformer Network - R. Girdhar et al., CVPR2019. [[project web]](https://rohitgirdhar.github.io/ActionTransformer/)
  - A Closer Look at Spatiotemporal Convolutions for Action Recognition - D. Tran et al., CVPR2018. [[code]](https://github.com/facebookresearch/R2Plus1D) [[PyTorch]](https://github.com/irhumshafkat/R2Plus1D-PyTorch)
  - Attend and Interact: Higher-Order Object Interactions for Video Understanding - CY. Ma et al., CVPR 2018.
  - Non-Local Neural Networks - X. Wang et al., CVPR2018. [[code]](https://github.com/facebookresearch/video-nonlocal-net)
  - Rethinking Spatiotemporal Feature Learning For Video Understanding - S. Xie et al., arXiv2017.
  - ConvNet Architecture Search for Spatiotemporal Feature Learning - D. Tran et al, arXiv2017. Note: Aka Res3D. [[code]](https://github.com/facebook/C3D): In the repository, C3D-v1.1 is the Res3D implementation.
  - Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset - J. Carreira et al, CVPR2017. [[code]](https://github.com/deepmind/kinetics-i3d)[[PyTorch code]](https://github.com/hassony2/kinetics_i3d_pytorch), [[another PyTorch code]](https://github.com/piergiaj/pytorch-i3d)
  - Learning Spatiotemporal Features with 3D Convolutional Networks - D. Tran et al, ICCV2015. [[the official Caffe code]](https://github.com/facebook/C3D) [[project web]](http://vlg.cs.dartmouth.edu/c3d/) Note: Aka C3D. [[Python Wrapper]](https://github.com/chuckcho/C3D/tree/python-wrapper) Note that the official caffe does not support python wrapper. [[TensorFlow]](https://github.com/hx173149/C3D-tensorflow), [[TensorFlow + Keras]](https://github.com/axon-research/c3d-keras), [[Another TensorFlow Implemetation]](https://github.com/frankgu/C3D-tensorflow.git), [[Keras C3D Project web]](https://imatge.upc.edu/web/resources/c3d-model-keras-trained-over-sports-1m): [[Keras code]](https://gist.github.com/albertomontesg/d8b21a179c1e6cca0480ebdf292c34d2), [[Pretrained weights]](https://www.dropbox.com/s/ypiwalgtlrtnw8b/c3d-sports1M_weights.h5?dl=0).
  - Deep Temporal Linear Encoding Networks - A. Diba et al, CVPR2017.
  - Temporal Convolutional Networks: A Unified Approach to Action Segmentation and Detection - C. Lea et al, CVPR 2017. [[code]](https://github.com/colincsl/TemporalConvolutionalNetworks)
  - Long-term Temporal Convolutions - G. Varol et al, TPAMI2017. [[project web]](http://www.di.ens.fr/willow/research/ltc/) [[code]](https://github.com/gulvarol/ltc)
  - Temporal Segment Networks: Towards Good Practices for Deep Action Recognition - L. Wang et al, arXiv 2016. [[code]](https://github.com/yjxiong/temporal-segment-networks)
  - Convolutional Two-Stream Network Fusion for Video Action Recognition - C. Feichtenhofer et al, CVPR2016. [[code]](https://github.com/feichtenhofer/twostreamfusion)
  - Two-Stream Convolutional Networks for Action Recognition in Videos - K. Simonyan and A. Zisserman, NIPS2014.
- Action Classification
  - Temporal 3D ConvNets: New Architecture and Transfer Learning for Video Classification - A. Diba et al., arXiv2017.
  - Attentional Pooling for Action Recognition - R. Girdhar and D. Ramanan, NIPS2017. [[code]](https://github.com/rohitgirdhar/AttentionalPoolingAction)
  - Fully Context-Aware Video Prediction - Byeon et al, arXiv2017.
  - Hidden Two-Stream Convolutional Networks for Action Recognition - Y. Zhu et al, arXiv2017. [[code]](https://github.com/bryanyzhu/Hidden-Two-Stream)
  - Dynamic Image Networks for Action Recognition - H. Bilen et al, CVPR2016. [[code]](https://github.com/hbilen/dynamic-image-nets) [[project web]](http://www.robots.ox.ac.uk/~vgg/publications/2016/Bilen16a/)
  - Describing Videos by Exploiting Temporal Structure - L. Yao et al, ICCV2015. [[code]](https://github.com/yaoli/arctic-capgen-vid) note: from the same group of RCN paper “Delving Deeper into Convolutional Networks for Learning Video Representations"
  - Two-Stream SR-CNNs for Action Recognition in Videos - L. Wang et al, BMVC2016.
  - Real-time Action Recognition with Enhanced Motion Vector CNNs - B. Zhang et al, CVPR2016. [[code]](https://github.com/zbwglory/MV-release)
Object Recognition
- Video Object Detection Datasets
- Object Detection
  - Mask R-CNN - K. He et al, [[Detectron]](https://github.com/facebookresearch/Detectron), [[TensorFlow + Keras]](https://github.com/matterport/Mask_RCNN), [[MXNet]](https://github.com/TuSimple/mx-maskrcnn), [[TensorFlow]](https://github.com/CharlesShang/FastMaskRCNN), [[PyTorch]](https://github.com/felixgwu/mask_rcnn_pytorch) - State-of-the-art object detection/instance segmentation algorithm.
  - Faster R-CNN - S. Ren et al, NIPS2015. [[official MatCaffe code]](https://github.com/ShaoqingRen/faster_rcnn), [[PyCaffe]](https://github.com/rbgirshick/py-faster-rcnn), [[TensorFlow]](https://github.com/smallcorgi/Faster-RCNN_TF), [[Another TF implementation]](https://github.com/CharlesShang/TFFRCNN) [[Keras]](https://github.com/yhenon/keras-frcnn) - State-of-the-art object detector.
  - YOLO - J. Redmon et al, CVPR2016. [[official code]](https://github.com/pjreddie/darknet.git), [[TensorFLow]](https://github.com/gliese581gg/YOLO_tensorflow) - Fast object detector.
  - YOLO9000 - J. Redmon and A. Farhadi, CVPR2017. [[official code]](https://pjreddie.com/darknet/yolo/) - State-of-the-art object detector which can detect 9000 objects in realtime.
  - SSD - W. Liu et al, ECCV2016. [[official PyCaffe code]](https://github.com/weiliu89/caffe/tree/ssd), [[TensorFlow]](https://github.com/balancap/SSD-Tensorflow), [[Keras]](https://github.com/rykov8/ssd_keras) - State-of-the-art object detector with realtime processing speed.
  - RetinaNet - Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He and Piotr Dollár, Facebook AI Research FAIR & ICCV 2017.[[Keras]](https://github.com/fizyr/keras-retinanet) - State-of-the-art object detector with realtime processing speed.
Pose Estimation
- Pose Estimation
  - Detect-and-Track: Efficient Pose Estimation in Videos - R. Girdhar et al., arXiv2017.
  - Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields - Z. Cao et al, CVPR2017. [[code]](https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation) depends on the [[caffe RT pose]](https://github.com/CMU-Perceptual-Computing-Lab/caffe_rtpose.git) - Earlier version of OpenPose from CMU.
  - DensePose - Dense pose human estimation in the wild implemented in the Detectron framework.
  - MultiPoseNet: Fast Multi-Person Pose Estimation using Pose Residual Network - M. Kocabas et al, ECCV2018. [[code]](https://github.com/salihkaragoz/pose-residual-network-pytorch)
Competitions
- Competitions
  - ActEV (Activities in Extended Video - Activity detection in security camera videos. Runs through 2021. Hosted by NIST.
Licenses
- Competitions
  - ![CC0

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

awesome-action-recognition

Action Recognition and Video Understanding

Spatio-Temporal Action Detection

Ego-Centric Action Recognition

Miscellaneous

Action Recognition Datasets

Video Annotation

Skeleton-Based Action Classification

Temporal Action Detection

Video Representation

Action Classification

Object Recognition

Video Object Detection Datasets

Object Detection

Pose Estimation

Pose Estimation

Competitions

Competitions

Licenses

Competitions