https://github.com/o-tawab/Video-Pixel-Networks

Video Pixel Networks in Tensorflow
https://github.com/o-tawab/Video-Pixel-Networks

computer-vision video-pixel-network vpn

Last synced: over 1 year ago
JSON representation

Video Pixel Networks in Tensorflow

• Host: GitHub
• URL: https://github.com/o-tawab/Video-Pixel-Networks
• Owner: o-tawab
• Created: 2017-08-21T17:20:39.000Z (almost 9 years ago)
• Default Branch: master
• Last Pushed: 2018-04-21T08:08:37.000Z (about 8 years ago)
• Last Synced: 2024-10-24T06:28:35.935Z (over 1 year ago)
• Topics: computer-vision, video-pixel-network, vpn
• Language: Python
• Homepage:
• Size: 82 KB
• Stars: 55
• Watchers: 6
• Forks: 12
• Open Issues: 2
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# VPN: Video Pixel Networks in Tensorflow 

The VPN is a probabilistic video model that estimates the discrete joint distribution of the raw pixel values in a video. It approaches the best possible performance on the Moving MNIST benchmark. 

This repository contains a tensorflow implementation of the VPN architecture proposed in the [paper](https://arxiv.org/abs/1610.00527). However, this code hasn’t been trained and tested on the full Moving MNIST dataset due to lack of the computation power. It has been overfitted on one sequence to insure the correctness of the implementation. 

This repository also contains some additional experiments with the VPN architecture that are not mentioned in the original paper. These experiments are:

* Mini VPN architecture.

* Micro VPN architecture.

## Examples of the network output

##### Here's different steps from predicting the first frame pixel-by-pixel:

![alt text](https://github.com/3ammor/Video-Pixel-Networks/blob/master/images/0.PNG "frame 1")

![alt text](https://github.com/3ammor/Video-Pixel-Networks/blob/master/images/1.PNG "frame 2")

![alt text](https://github.com/3ammor/Video-Pixel-Networks/blob/master/images/2.PNG "frame 3")

![alt text](https://github.com/3ammor/Video-Pixel-Networks/blob/master/images/3.PNG "frame 4")

## The loss curves for the different architectures

##### full:pink

##### mini:red

##### micro:orange

![alt text](https://github.com/3ammor/Video-Pixel-Networks/blob/master/images/graph.PNG "loss")

### The Moving MNIST Dataset

You can download the full moving MNIST dataset from toronto [website](http://www.cs.toronto.edu/~nitish/unsupervised_video/). 

### Overfitting On One Sequence

```

python vpn.py --vpn_arch='mini' --train=True --overfitting=Ture --data_dir='/numpy/file/directory/' --exp_dir='/tmp/vpn/'

```

### Training On The Full Dataset

```

python vpn.py --vpn_arch='mini' --train=True --overfitting=False --data_dir='/numpy/file/directory/' --exp_dir='/tmp/vpn/'

```

### TODO

* Train on data generated on the air.