Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/HasnainRaz/Fast-SRGAN
A Fast Deep Learning Model to Upsample Low Resolution Videos to High Resolution at 30fps
https://github.com/HasnainRaz/Fast-SRGAN
artificial-intelligence cnn fastsrgan gans generative-adversarial-network neural-network realtime-super-resolution residual-blocks resolution-image single-image-super-resolution sisr srgan super-resolution tensorboard tensorflow tf-keras tf2 upsample
Last synced: about 2 months ago
JSON representation
A Fast Deep Learning Model to Upsample Low Resolution Videos to High Resolution at 30fps
- Host: GitHub
- URL: https://github.com/HasnainRaz/Fast-SRGAN
- Owner: HasnainRaz
- License: mit
- Created: 2019-10-20T16:41:22.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2024-06-25T10:01:01.000Z (6 months ago)
- Last Synced: 2024-08-01T03:16:50.840Z (4 months ago)
- Topics: artificial-intelligence, cnn, fastsrgan, gans, generative-adversarial-network, neural-network, realtime-super-resolution, residual-blocks, resolution-image, single-image-super-resolution, sisr, srgan, super-resolution, tensorboard, tensorflow, tf-keras, tf2, upsample
- Language: Python
- Homepage:
- Size: 4.42 MB
- Stars: 657
- Watchers: 18
- Forks: 115
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-tensorflow-2 - Fast-SRGAN (Single Image Super Resolution GAN)
- Awesome-Tensorflow2 - HasnainRaz/Fast-SRGAN
README
# Fast-SRGAN
The goal of this repository is to enable real time super resolution for upsampling low resolution videos. Currently, the design follows the [SR-GAN](https://arxiv.org/pdf/1609.04802.pdf) architecture. For speed, the upsampling is done through pixel shuffle.
The training setup looks like the following diagram:
# Speed Benchmarks
The following runtimes/fps are obtained by averaging runtimes over 800 frames. Measured on MPS (MacBook M1 Pro GPU).
| Input Image Size | Output Size | Time (s) | FPS |
| ------------- |:--------------------:|:---------:|:---:|
| 90x160 | 360x640 (360p) | 0.01 | 82 |
| 180x320 | 720x1080 (720p) | 0.04 | 27 |
We see it's possible to upsample to 720p at around 30fps.
# Requirements
This was tested on Python 3.10. To install the required packages, use the provided Pipfile:
```bash
pip install pipenv --upgrade
pipenv install --system --deploy
```
# Pre-trained Model
A pretrained generator model on the DIV2k dataset is provided in the 'models' directory. It uses 8 residual blocks, with 64 filters in every layer of the generator.
To try out the provided pretrained model on your own images, run the following:
```bash
python inference.py --image_dir 'path/to/your/image/directory' --output_dir 'path/to/save/super/resolution/images'
```
# Training
To train, simply edit the config file in the folder `configs/config.yaml` with your settings, and then launch the training with:
```bash
python train.py
```
You can also change the config parameters from the command line. The following will run training with a `batch_size` of 32, a generator with 12 residual blocks, and a path to the image directory `/path/to/image/dataset`.
```
python train.py data.image_dir="/path/to/image/dataset" training.batch_size=32 generator.n_layers=12
```
This is powered by `hydra`, which means all the parameters in the config are editable via the CLI.
Model checkpoints and training summaries are saved in tensorboard. To monitor training progress, open up tensorboard by pointing it to the `outputs` directory that will be created when you start training.
# Samples
Following are some results from the provided trained model. Left shows the low res image, after 4x bicubic upsampling. Middle is the output of the model. Right is the actual high resolution image.
The following shows images upsampled 4x by bicubic interpolation, the pretrained model from this repository and the original high resolution image as a comparison
# Contributing
If you have ideas on improving model performance, adding metrics, or any other changes, please make a pull request or open an issue. I'd be happy to accept any contributions.