https://github.com/pathak22/unsupervised-video
[CVPR 2017] Unsupervised deep learning using unlabelled videos on the web
https://github.com/pathak22/unsupervised-video
computer-vision deep-learning feature-learning machine-learning motion-segmentation unsupervised-learning video-processing video-segmentation
Last synced: 15 days ago
JSON representation
[CVPR 2017] Unsupervised deep learning using unlabelled videos on the web
- Host: GitHub
- URL: https://github.com/pathak22/unsupervised-video
- Owner: pathak22
- License: mit
- Created: 2017-04-24T03:31:32.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2019-04-25T05:12:55.000Z (about 6 years ago)
- Last Synced: 2025-05-08T21:43:35.280Z (15 days ago)
- Topics: computer-vision, deep-learning, feature-learning, machine-learning, motion-segmentation, unsupervised-learning, video-processing, video-segmentation
- Language: Lua
- Homepage: https://people.eecs.berkeley.edu/~pathak/unsupervised_video/
- Size: 336 KB
- Stars: 260
- Watchers: 12
- Forks: 51
- Open Issues: 7
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## Learning Features by Watching Objects Move ##
In CVPR 2017. [[Project Website]](http://cs.berkeley.edu/~pathak/unsupervised_video/).
[Deepak Pathak](https://people.eecs.berkeley.edu/~pathak/), [Ross Girshick](http://www.rossgirshick.info/), [Piotr Dollár](https://pdollar.github.io/), [Trevor Darrell](https://people.eecs.berkeley.edu/~trevor/), [Bharath Hariharan](http://home.bharathh.info/)
University of California, Berkeley
Facebook AI Research (FAIR)
This is the code for our [CVPR 2017 paper on Unsupervised Learning using unlabeled videos](http://cs.berkeley.edu/~pathak/unsupervised_video/). This repository contains models trained by the unsupervised motion grouping algorithm both in Caffe and Torch. If you find this work useful in your research, please cite:
@inproceedings{pathakCVPR17learning,
Author = {Pathak, Deepak and Girshick, Ross and Doll\'{a}r,
Piotr and Darrell, Trevor and Hariharan, Bharath},
Title = {Learning Features by Watching Objects Move},
Booktitle = {Computer Vision and Pattern Recognition ({CVPR})},
Year = {2017}
}
### 1) Fetching Models for Unsupervised Transfer
The models below only contains the layer that are used for unsupervised transfer learning. For the full model that contains motion segmentation, see next section.
1. Clone the repository
```Shell
git clone https://github.com/pathak22/unsupervised-video.git
```
2. Fetch caffe models
```Shell
cd unsupervised-video/
bash ./models/download_caffe_models.sh
# This will populate the `./models/` folder with trained models.
```
The models were initially trained in Torch and then converted to caffe. Hence, please include pycaffe based `image_transform_layer.py` in your folder. It converts the scale and mean of the input image as needed.
3. Fetch torch models
```Shell
cd unsupervised-video/
bash ./models/download_torch_models.sh
# This will populate the `./models/` folder with trained models.
```
### 2) Fetching Motion Segmentation models
Follow the instructions below to download full motion segmentation model trained on the automatically selected 205K videos from YFCC100m. I trained it in Torch, but you can train your own model from the full data [available here](https://people.eecs.berkeley.edu/~pathak/unsupervised_video/index.html#data) in any deep learning package using the training details from paper.
```Shell
cd unsupervised-video/
bash ./models/download_torch_motion_model.sh
# This will populate the `./models/` folder with trained model.
cd motionseg/
th load_motionmodel.lua -input ../models/motionSegmenter_fullModel.t7
```
### 3) Additional Software Packages
We are releasing software packages which were developed in the project, but could be generally useful for computer vision research. If you find them useful, please consider citing our work. These include:
(a) uNLC [github]: Implementation of unsupervised bottom-up video segmentation algorithm which is unsupervised adaptation of NLC algorithm by Faktor and Irani, BMVC 2014. For additional details, see section 5.1 in the paper.
(b) PyFlow [github]: This is python wrapper around Ce Liu's C++ implementation of Coarse2Fine Optical Flow. This is used inside uNLC implementation, and also generally useful as an independent package.