https://github.com/eriklindernoren/action-recognition

Exploration of different solutions to action recognition in video, using neural networks implemented in PyTorch.
https://github.com/eriklindernoren/action-recognition

action-recognition pytorch video-classification

Last synced: 5 days ago
JSON representation

Exploration of different solutions to action recognition in video, using neural networks implemented in PyTorch.

• Host: GitHub
• URL: https://github.com/eriklindernoren/action-recognition
• Owner: eriklindernoren
• Created: 2019-04-06T12:25:12.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2019-12-17T19:52:00.000Z (almost 5 years ago)
• Last Synced: 2024-11-06T02:39:17.460Z (10 days ago)
• Topics: action-recognition, pytorch, video-classification
• Language: Python
• Homepage:
• Size: 8.51 MB
• Stars: 181
• Watchers: 9
• Forks: 74
• Open Issues: 19
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Action Recognition in Video

This repo will serve as a playground where I investigate different approaches to solving the problem of action recognition in video.

I will mainly use the [UCF-101 dataset](https://www.crcv.ucf.edu/data/UCF101.php).



    



## Setup

```

$ cd data/              

$ bash download_ucf101.sh     # Downloads the UCF-101 dataset (~7.2 GB)

$ unrar x UCF101.rar          # Unrars dataset

$ unzip ucfTrainTestlist.zip  # Unzip train / test split

$ python3 extract_frames.py   # Extracts frames from the video (~26.2 GB, go grab a coffee for this)

```

## ConvLSTM

The only approach investigated so far. Enables action recognition in video by a bi-directional LSTM operating on frame embeddings extracted by a pre-trained ResNet-152 (ImageNet).

The model is composed of:

* A convolutional feature extractor (ResNet-152) which provides a latent representation of video frames

* A bi-directional LSTM classifier which based on the latent representation of the video predicts the activity depicted

I have made a trained model available [here](https://drive.google.com/open?id=1GlpN0m9uLbI9dg1ARbW9hDEf-VWe4Asl).

### Train  

```

$ python3 train.py  --dataset_path data/UCF-101-frames/ \

                    --split_path data/ucfTrainTestlist \

                    --num_epochs 200 \

                    --sequence_length 40 \

                    --img_dim 112 \

                    --latent_dim 512

```

### Test on Video

```

$ python3 test_on_video.py  --video_path data/UCF-101/SoccerPenalty/v_SoccerPenalty_g01_c01.avi \

                            --checkpoint_model model_checkpoints/ConvLSTM_150.pth

```



    



### Results

The model reaches a classification accuracy of **91.27%** accuracy on a randomly sampled test set, composed of 20% of the total amount of video sequences from UCF-101. Will re-train this model on the offical train / test splits and post results as soon as I have time.