https://github.com/HsinYingLee/OPN
https://github.com/HsinYingLee/OPN
Last synced: 21 days ago
JSON representation
- Host: GitHub
- URL: https://github.com/HsinYingLee/OPN
- Owner: HsinYingLee
- Created: 2017-07-07T21:32:50.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2018-04-26T13:00:18.000Z (over 7 years ago)
- Last Synced: 2024-07-05T15:52:22.212Z (over 1 year ago)
- Language: Python
- Size: 10.7 KB
- Stars: 46
- Watchers: 3
- Forks: 13
- Open Issues: 8
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-self-supervised-learning - [code
- awesome-self-supervised-learning - [code
README
# Unsupervised Representation Learning by Sorting Sequence (ICCV 2017)
[Hsin-Ying Lee](http://vllab1.ucmerced.edu/~hylee/),
[Jia-Bin Huang](https://filebox.ece.vt.edu/~jbhuang/),
[Maneesh Kumar Singh](https://scholar.google.com/citations?user=hdQhiFgAAAAJ),
and [Ming-Hsuan Yang](http://faculty.ucmerced.edu/mhyang/)
IEEE International Conference on Computer Vision, ICCV 2017
### Table of Contents
1. [Introduction](#introduction)
1. [Citation](#citation)
1. [Requirements and Dependencies](#requirements)
1. [Models and Training Data](#models-and-training-data)
1. [Training](#training)
1. [Testing](#testing)
### Introduction
The Order Prediction Network (OPN) is a model that performs representation learning using unlabeled videos. Our method leverage temporal coherence as a supervisory signal by formulating representation learning as a sequence sorting task. The experimental results show that our method compares favorably against state-of-the-art methods on action recognition, image classification and object detection tasks. For more details and evaluation results, please check out our [project webpage](http://vllab.ucmerced.edu/hylee/OPN/) and [paper](http://vllab.ucmerced.edu/hylee/publication/ICCV17_OPN.pdf).
### Citation
If you find the code and data useful in your research, please cite:
@inproceeding{OPN,
author = {Lee, Hsin-Ying and Huang, Jia-Bin and Singh, Maneesh Kumar and Yang, Ming-Hsuan},
title = {Unsupervised Representation Learning by Sorting Sequence},
booktitle = {IEEE International Conference on Computer Vision},
year = {2017}
}
### Requirements
Requirements for `Caffe` and `pycaffe` (see: [Caffe installation instructions](http://caffe.berkeleyvision.org/installation.html))
Note that the Caffe fork needs to support Batch Normalization to run our code.
### Models and Training Data
**Training Data**
[UCF](http://vllab1.ucmerced.edu/~hylee/OPN/results/UCF_train.mat) / [HMDB](http://vllab.ucmerced.edu/hylee/OPN/results/HMDB_train.mat) / [UCF+HMDB+ACT](http://vllab.ucmerced.edu/hylee/OPN/results/UCF_HMDB_ACT.mat)
**Models**
[Model](http://vllab.ucmerced.edu/hylee/OPN/results/UCF_OPN.caffemodel) Unsupervised trained on UCF
[Model](http://vllab.ucmerced.edu/hylee/OPN/results/UCFHMDBACT_nobn.caffemodel) Unsupervised trained on UCF+HMDB+ACT (for Pascal VOC 2007)
### Training
There are a few lines need to be customized in UCF_datalayers.py, including the training data location in L22-L27 and L84-87. The default setting includes all processing like channel splitting and spatial jittering, feel free to comment them out.
$ $CAFFE_ROOT/build/tool/caffe train -solver prototxt/solver_opn.prototxt
### Testing
**Visualization**
$ python visualize.py $MODEL $OUTPUT_FIG
**Action Recognition**
The testing on the UCF-101 and HMDB-51 datasets follows the testing sceme of the [original two-stream ConvNets](https://arxiv.org/pdf/1406.2199.pdf), where we sample 25 RGB frames from each video. From each of the frames we then obtain 10 inputs by cropping and flipping four corners and the center of the frame.
**Pascal VOC 2007 Classification**
The training and testing codes are from [https://github.com/philkr/voc-classification](https://github.com/philkr/voc-classification).
**Pascal VOC 2007 Detection**
The training and testing codes are from [https://github.com/rbgirshick/fast-rcnn](https://github.com/rbgirshick/fast-rcnn)