Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/baldassarrefe/deep-koalarization

Keras/Tensorflow implementation of our paper Grayscale Image Colorization using deep CNN and Inception-ResNet-v2 (https://arxiv.org/abs/1712.03400)
https://github.com/baldassarrefe/deep-koalarization

colorization convolutional-neural-networks deep-learning inception-resnet keras tensorflow

Last synced: 1 day ago
JSON representation

Keras/Tensorflow implementation of our paper Grayscale Image Colorization using deep CNN and Inception-ResNet-v2 (https://arxiv.org/abs/1712.03400)

Host: GitHub
URL: https://github.com/baldassarrefe/deep-koalarization
Owner: baldassarreFe
License: mit
Created: 2017-03-28T10:00:58.000Z (almost 8 years ago)
Default Branch: master
Last Pushed: 2022-11-23T19:38:31.000Z (about 2 years ago)
Last Synced: 2025-01-13T05:04:34.275Z (8 days ago)
Topics: colorization, convolutional-neural-networks, deep-learning, inception-resnet, keras, tensorflow
Language: Python
Homepage: https://lcsrg.me/deep-koalarization
Size: 32.4 MB
Stars: 413
Watchers: 21
Forks: 79
Open Issues: 3
Metadata Files:
- Readme: README.md
- License: LICENSE
- Citation: CITATION.cff

Awesome Lists containing this project

README

        
 🐨 deep koalarization



Impementation of our paper Deep Koalarization: Image Colorization using CNNs and Inception-ResNet-v2 (2017)





  

    

  





  

  

  

  

    

  

  





  Federico Baldassarre^*,

  Diego Gonzalez Morín^* and Lucas Rodés-Guirao^* 





  ^{* Authors contributed equally}

 

**deep-koalarization** was developed as part of the [DD2424 Deep Learning in Data Science course](https://www.kth.se/student/kurser/kurs/DD2424?l=en) at [KTH Royal Institute of Technology](https://www.kth.se/en), spring 2017.

The code is built using [Keras](https://keras.io) and [Tensorflow](https://www.tensorflow.org/).

Consider starring this project if you found it useful :star:!

### Table of contents

- [Citation](#citation)

- [Abstract](#abstract)

- [Project Overview](#project-overview)

- [Results](#results)

- [Use the code](#use-the-code)

- [Community](#community)

## Citation

If you find Deep Koalarization useful in your research, please consider citing our paper as

```

@article{deepkoal2017,

  author          = {Federico Baldassarre, Diego Gonzalez-Morin, Lucas Rodes-Guirao},

  title           = {Deep-Koalarization: Image Colorization using CNNs and Inception-ResNet-v2},

  journal         = {ArXiv:1712.03400},

  url             = {https://arxiv.org/abs/1712.03400},

  year            = 2017,

  month           = dec

}

```

[arXiv e-print](https://arxiv.org/abs/1712.03400)

## Abstract

We review some of the most recent approaches to colorize gray-scale images using deep learning methods. Inspired by these, we propose a model which combines a deep Convolutional Neural Network trained from scratch with high-level features extracted from the Inception-ResNet-v2 pre-trained model. Thanks to its fully convolutional architecture, our encoder-decoder model can process images of any size and aspect ratio. Other than presenting the training results, we assess the "public acceptance" of the generated images by means of a user study. Finally, we present a carousel of applications on different types of images, such as historical photographs.

## Project overview

Inspired by [Iizuka and Simo-Serra *et al.* (2016)](http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/data/colorization_sig2016.pdf), we combine a deep CNN architecture with [Inception-ResNet-v2](https://arxiv.org/abs/1602.07261) pre-trained on ImageNet dataset, which assists the overall colorization process by extracting high-level features. In particular, Inception-ResNet-v2

![](assets/our_net.png)

The _fusion_ between the fixed-size embedding and the intermediary result of the convolutions is performed by means of replication and stacking as described in [Iizuka and Simo-Serra *et al.* (2016)](http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/data/colorization_sig2016.pdf).

![Fusion](assets/fusion_layer.png)

We have used the MSE loss as the objective function.

The Training data for this experiment could come from any source. We decuded to use [ImageNet](http://www.image-net.org), which nowadays is considered the de-facto reference for image tasks. This way, it makes easier for others to replicate our experiments.

## Results

#### ImageNet

![ImageNet 1](assets/comparison.png)

#### Historical pictures

![Historical 1](assets/historical.png)

---

## Use the code

Refer to [INSTRUCTIONS](INSTRUCTIONS.md) to install and use the code in this repo.

## Community

### Thanks to the people who noticed our work!

We are proud if our work gets noticed and helps/inspires other people on their path to knowledge. Here's a list of references we are aware of, some of the authors contacted us, some others we just happened to find online:

- François Chollet [tweeted](https://twitter.com/fchollet/status/917846097430638592) about this project (thank you for Keras)

- Emil Wallnér on [FloydHub Blog](https://blog.floydhub.com/colorizing-b&w-photos-with-neural-networks/) and [freecodecamp](https://medium.freecodecamp.org/colorize-b-w-photos-with-a-100-line-neural-network-53d9b4449f8d)

- Amir Kalron on [Logz.io Blog](https://logz.io/blog/open-source-machine-learning/)

- sparkexpert on [CSDN](http://blog.csdn.net/sparkexpert/article/details/74452523)

- Eryk Lewinson on [Medium](https://towardsdatascience.com/image-colorization-using-convolutional-autoencoders-fdabc1cb1dbe)

### Projects originated from here

- _[Coloring Black and White Images with Neural Networks](https://github.com/emilwallner/Coloring-greyscale-images)_, by [emilwallner](https://github.com/emilwallner).

- _[Ensemble Image Colorization using Convolutional Neural Networks with Refinement Network](https://github.com/Kriztoper/deep-koalarization)_, by [Kriztoper](https://github.com/Kriztoper).