Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/araxeus/png-upscale

AI Super - Resolution
https://github.com/araxeus/png-upscale
java javacv machine-learning png resampling resize resizing-images software super-resolution upscaling
Last synced: 4 days ago
JSON representation
AI Super - Resolution
Host: GitHub
URL: https://github.com/araxeus/png-upscale
Owner: Araxeus
License: mit
Created: 2021-02-17T16:08:34.000Z (almost 4 years ago)
Default Branch: main
Last Pushed: 2023-06-30T12:17:55.000Z (over 1 year ago)
Last Synced: 2024-12-09T11:38:25.081Z (13 days ago)
Topics: java, javacv, machine-learning, png, resampling, resize, resizing-images, software, super-resolution, upscaling
Language: Java
Homepage:
Size: 155 MB
Stars: 221
Watchers: 3
Forks: 25
Open Issues: 23
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project

README

        # :fire: PNG Upscale -  AI Super Resolution :fire:

> Small tool using pretrained models to upscale images

 

 ## Download is available from the [Releases Page](https://github.com/Araxeus/PNG-Upscale/releases/latest) or [Google Drive](https://drive.google.com/drive/folders/1pMvzL5sqTRcJOhByU_Am5vU2oik_KHH2?usp=sharing) or [MediaFire](https://app.mediafire.com/nz88rdbl8u041)

* Hosted Folder include full "[Models](https://github.com/Araxeus/PNG-Upscale/tree/main/Models)" folder 📁 and executable Files 🖼️ to download

* `Windows 64bit` [[Exe]](https://github.com/Araxeus/PNG-Upscale/releases/download/v1.0/png-upscale-1.0_Windows64bit.exe) / [[Jar]](https://github.com/Araxeus/PNG-Upscale/releases/download/v1.0/png-upscale-1.0_Windows64bit.jar) 

* `Linux 64bit` [[Jar]](https://github.com/Araxeus/PNG-Upscale/releases/download/v1.0/png-upscale-1.0_LINUX_64bit.jar) 

* `macOS 64bit` [[Jar]](https://github.com/Araxeus/PNG-Upscale/releases/download/v1.0/png-upscale-1.0_macOS_64bit.jar)

* :electron:	Download the executable corresponding with your operating system, and the Models folder

* It's possible to download only some of the models if you want (It just wont let you use them inside the program)

* The Models folder needs to be in the same directory as the Jar/Exe to use them

 ---

* ⚠️ Be careful when trying to resize very large pictures, it can take considerable time and resources ⚠️

* To upscale an image you just need to choose a mode, load a picture and press start

* Save button *can* be used to choose an output folder and filename *before* you start the process (either just name or .png)

* You can double click the text box to change [Dark <-> Light] theme (disabled when upScaling)

* Use PNG images for best results

* if faced with a JNI Error see this issue for a possible fix https://github.com/Araxeus/PNG-Upscale/issues/33

 ---

## Models

>`all of the model download links below are already included in the MediaFire folder.`

There are four trained models integrated into the program :

### EDSR 

##### `[Best Quality]+[Slowest]`

- Size of the model: ~38.5MB x3. This is a quantized version, so that it can be uploaded to GitHub. (Original was 150MB each.)

- This model was trained for 3 days with a batch size of 16

- Link to implementation code: https://github.com/Saafke/EDSR_Tensorflow

- x2, x3, x4 trained models available

- Advantage: *Highly accurate*

- Disadvantage: *Slow* and large filesize

- Speed: < 3 sec for every scaling factor on 256x256 images on an Intel i7-9700K CPU.

- Original paper: [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://arxiv.org/pdf/1707.02921.pdf) [1]

> Trained models can be downloaded from [here](https://github.com/Saafke/EDSR_Tensorflow/tree/master/models).

### ESPCN

##### `[Fast]`

- Size of the model: ~100kb x3

- This model was trained for ~100 iterations with a batch size of 32

- Link to implementation code: https://github.com/fannymonori/TF-ESPCN

- x2, x3, x4 trained models available

- Advantage: It is tiny and fast, and still performs well.

- Disadvantage: Perform worse visually than newer, more robust models.

- Speed: < 0.01 sec for every scaling factor on 256x256 images on an Intel i7-9700K CPU.

- Original paper: [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://arxiv.org/pdf/1707.02921.pdf) [2]

> Trained models can be downloaded from [here](https://github.com/fannymonori/TF-ESPCN/tree/master/export).

### FSRCNN

##### `[Fast]`

- Size of the model: ~40KB x3

- This model was trained for ~30 iterations with a batch size of 1

- Link to implementation code: https://github.com/Saafke/FSRCNN_Tensorflow

- Advantage: Fast, small and accurate

- Disadvantage: Not state-of-the-art accuracy

- Speed: < 0.01 sec for every scaling factor on 256x256 images on an Intel i7-9700K CPU.

- Notes: FSRCNN-small has fewer parameters, thus less accurate but faster.

- Original paper: [Accelerating the Super-Resolution Convolutional Neural Network](http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html) [3]

> Trained models can be downloaded from [here](https://github.com/Saafke/FSRCNN_Tensorflow/tree/master/models).

### LapSRN

##### `[Has x8]`

- Size of the model: between 1-5Mb x3

- This model was trained for ~50 iterations with a batch size of 32

- Link to implementation code: https://github.com/fannymonori/TF-LAPSRN

- x2, x4, x8 trained models available

- Advantage: The model can do multi-scale super-resolution with one forward pass. It can now support 2x, 4x, 8x, and [2x, 4x] and [2x, 4x, 8x] super-resolution.

- Disadvantage: It is slower than ESPCN and FSRCNN, and the accuracy is worse than EDSR.

- Speed: < 0.1 sec for every scaling factor on 256x256 images on an Intel i7-9700K CPU.

- Original paper: [Deep laplacian pyramid networks for fast and accurate super-resolution](https://arxiv.org/pdf/1707.02921.pdf) [4]

> Trained models can be downloaded from [here](https://github.com/fannymonori/TF-LapSRN/tree/master/export).

 ---

### Benchmarks

Comparing different algorithms. Scale x4 on monarch.png

|               | Inference time in seconds (CPU)| PSNR | SSIM |

| ------------- |:-------------------:| ---------:|--------:|

| ESPCN            |0.01159   | 26.5471 | 0.88116 |

| EDSR             |3.26758     |**29.2404**  |**0.92112**  |

| FSRCNN           | 0.01298   | 26.5646 | 0.88064 |

| LapSRN           |0.28257    |26.7330   |0.88622  |

| Bicubic          |0.00031 |26.0635  |0.87537  |

| Nearest neighbor |**0.00014** |23.5628  |0.81741  |

| Lanczos          |0.00101  |25.9115  |0.87057  |

 ---

 

### As a Demo this image was resized from 256x256 to 85x85, and then upscaled using this program

![Original](https://github.com/Araxeus/PNG-Upscale/blob/main/test/original.png)

## x2 Demo (85x85 -> 170x170)

|      Original             |  Bicubic Interpolation    |        EDSR               |

| ------------------------- |------------------------- |------------------------- |

![Original](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x2/original.png)   |  ![Bicubic](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x2/input(BicubicX2).png) |  ![EDSR](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x2/input(EDSRx2).png) |

|         ESPCN             |       FSRCNN              |        LapSRN             |

| ------------------------- | ------------------------- | ------------------------- |

![ESPCN](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x2/input(ESPCNx2).png) | ![FSRCNN](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x2/input(FSRCNNx2).png) |  ![LapSRN](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x2/input(LapSRNx2).png) |

> Bicubic Interpolation is the standart resizing technique used by most editing tools like photoship etc..

## x4 Demo (85x85 -> 340x340)

|      Original             |  Bicubic Interpolation    |        EDSR               |

| ------------------------- | ------------------------- | ------------------------- |

![Original](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x4/original.png)   |  ![Bicubic](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x4/input(BicubicX4).png)|  ![EDSR](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x4/input(EDSRx4).png)|

 

|        ESPCN             |       FSRCNN              |        LapSRN             |

| ------------------------- | ------------------------- | ------------------------- |

![ESPCN](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x4/input(ESPCNx4).png)   |  ![FSRCNN](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x4/input(FSRCNNx4).png)|  ![LapSRN](https://github.com/Araxeus/PNG-Upscale/blob/main/test/x4/input(LapSRNx4).png)|

### References

[1] Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee, **"Enhanced Deep Residual Networks for Single Image Super-Resolution"**,  2nd NTIRE: New Trends in Image Restoration and Enhancement workshop and challenge on image super-resolution in conjunction with **CVPR 2017**.  [[PDF](http://openaccess.thecvf.com/content_cvpr_2017_workshops/w12/papers/Lim_Enhanced_Deep_Residual_CVPR_2017_paper.pdf)] [[arXiv](https://arxiv.org/abs/1707.02921)] [[Slide](https://cv.snu.ac.kr/research/EDSR/Presentation_v3(release).pptx)]

[2] Shi, W., Caballero, J., Huszár, F., Totz, J., Aitken, A., Bishop, R., Rueckert, D. and Wang, Z., **"Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network"**, Proceedings of the IEEE conference on computer vision and pattern recognition **CVPR 2016**. [[PDF](http://openaccess.thecvf.com/content_cvpr_2016/papers/Shi_Real-Time_Single_Image_CVPR_2016_paper.pdf)] [[arXiv](https://arxiv.org/abs/1609.05158)]

[3] Chao Dong, Chen Change Loy, Xiaoou Tang. **"Accelerating the Super-Resolution Convolutional Neural Network"**,  in Proceedings of European Conference on Computer Vision **ECCV 2016**. [[PDF](http://personal.ie.cuhk.edu.hk/~ccloy/files/eccv_2016_accelerating.pdf)]

[[arXiv](https://arxiv.org/abs/1608.00367)] [[Project Page](http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html)]

[4] Lai, W. S., Huang, J. B., Ahuja, N., and Yang, M. H., **"Deep laplacian pyramid networks for fast and accurate super-resolution"**,  In Proceedings of the IEEE conference on computer vision and pattern recognition **CVPR 2017**. [[PDF](http://openaccess.thecvf.com/content_cvpr_2017/papers/Lai_Deep_Laplacian_Pyramid_CVPR_2017_paper.pdf)] [[arXiv](https://arxiv.org/abs/1710.01992)] [[Project Page](http://vllab.ucmerced.edu/wlai24/LapSRN/)]