{"id":17819217,"url":"https://github.com/baldassarrefe/deep-koalarization","last_synced_at":"2025-04-05T12:07:31.792Z","repository":{"id":71580938,"uuid":"86444372","full_name":"baldassarreFe/deep-koalarization","owner":"baldassarreFe","description":"Keras/Tensorflow implementation of our paper Grayscale Image Colorization using deep CNN and Inception-ResNet-v2 (https://arxiv.org/abs/1712.03400)","archived":false,"fork":false,"pushed_at":"2022-11-23T19:38:31.000Z","size":33964,"stargazers_count":413,"open_issues_count":3,"forks_count":79,"subscribers_count":20,"default_branch":"master","last_synced_at":"2025-03-29T11:09:30.765Z","etag":null,"topics":["colorization","convolutional-neural-networks","deep-learning","inception-resnet","keras","tensorflow"],"latest_commit_sha":null,"homepage":"https://lcsrg.me/deep-koalarization","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/baldassarreFe.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-03-28T10:00:58.000Z","updated_at":"2025-03-21T07:16:55.000Z","dependencies_parsed_at":"2023-04-04T05:16:05.670Z","dependency_job_id":null,"html_url":"https://github.com/baldassarreFe/deep-koalarization","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baldassarreFe%2Fdeep-koalarization","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baldassarreFe%2Fdeep-koalarization/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baldassarreFe%2Fdeep-koalarization/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/baldassarreFe%2Fdeep-koalarization/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/baldassarreFe","download_url":"https://codeload.github.com/baldassarreFe/deep-koalarization/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247332609,"owners_count":20921853,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["colorization","convolutional-neural-networks","deep-learning","inception-resnet","keras","tensorflow"],"created_at":"2024-10-27T16:59:52.637Z","updated_at":"2025-04-05T12:07:31.748Z","avatar_url":"https://github.com/baldassarreFe.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\" style=\"border-bottom: none;\"\u003e \u003ca href=\"https://lcsrg.me/deep-koalarization\"\u003e🐨 deep koalarization\u003c/a\u003e\n\u003c/h1\u003e\n\u003ch3 align=\"center\"\u003eImpementation of our paper \u003ca href=\"https://arxiv.org/abs/1712.03400\"\u003eDeep Koalarization: Image Colorization using CNNs and Inception-ResNet-v2 (2017)\u003c/a\u003e\u003c/h3\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/baldassarreFe/deep-koalarization\"\u003e\n    \u003cimg alt=\"Package version\" src=\"https://img.shields.io/badge/koalarization-v0.2.0-blue.svg?style=for-the-badge\u0026logo=python\u0026logoColor=yellow\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://www.python.org/downloads/release/python-360/\"\u003e\u003cimg alt=\"Python 3.6\" src=\"https://img.shields.io/badge/python-3.6-blue.svg\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/baldassarreFe/deep-koalarization/blob/master/LICENSE\"\u003e\u003cimg alt=\"GitHub License\" src=\"https://img.shields.io/github/license/baldassarreFe/deep-koalarization.svg\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/baldassarreFe/deep-koalarization/stargazers\"\u003e\u003cimg alt=\"GitHub stars\" src=\"https://img.shields.io/github/stars/baldassarreFe/deep-koalarization.svg\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/baldassarreFe/deep-koalarization/network\"\u003e\u003cimg alt=\"GitHub forks\" src=\"https://img.shields.io/github/forks/baldassarreFe/deep-koalarization.svg\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/baldassarreFe/deep-koalarization\"\u003e\u003cimg alt=\"HitCount\" src=\"https://views.whatilearened.today/views/github/baldassarreFe/deep-koalarization.svg\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://arxiv.org/abs/1712.03400\"\u003e\u003cimg alt=\"arXiv\" src=\"https://img.shields.io/badge/paper-arXiv-_.svg?\u0026color=B31B1B\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://twitter.com/intent/tweet?text=Wow:\u0026url=https%3A%2F%2Fgithub.com%2FbaldassarreFe%2Fdeep-koalarization\"\u003e\u003cimg alt=\"Twitter\" src=\"https://img.shields.io/twitter/url/https/github.com/baldassarreFe/deep-koalarization.svg?style=social\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/baldassarreFe\"\u003eFederico Baldassarre\u003c/a\u003e\u003csup\u003e*\u003c/sup\u003e,\n  \u003ca href=\"https://github.com/diegomorin8\"\u003eDiego Gonzalez Morín\u003c/a\u003e\u003csup\u003e*\u003c/sup\u003e and \u003ca href=\"https://github.com/lucasrodes\"\u003eLucas Rodés-Guirao\u003c/a\u003e\u003csup\u003e*\u003c/sup\u003e \n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003csup\u003e* Authors contributed equally\u003c/sup\u003e\n\u003c/p\u003e \n\n\n**deep-koalarization** was developed as part of the [DD2424 Deep Learning in Data Science course](https://www.kth.se/student/kurser/kurs/DD2424?l=en) at [KTH Royal Institute of Technology](https://www.kth.se/en), spring 2017.\n\nThe code is built using [Keras](https://keras.io) and [Tensorflow](https://www.tensorflow.org/).\n\nConsider starring this project if you found it useful :star:!\n\n### Table of contents\n\n- [Citation](#citation)\n- [Abstract](#abstract)\n- [Project Overview](#project-overview)\n- [Results](#results)\n- [Use the code](#use-the-code)\n- [Community](#community)\n\n## Citation\n\nIf you find Deep Koalarization useful in your research, please consider citing our paper as\n\n```\n@article{deepkoal2017,\n  author          = {Federico Baldassarre, Diego Gonzalez-Morin, Lucas Rodes-Guirao},\n  title           = {Deep-Koalarization: Image Colorization using CNNs and Inception-ResNet-v2},\n  journal         = {ArXiv:1712.03400},\n  url             = {https://arxiv.org/abs/1712.03400},\n  year            = 2017,\n  month           = dec\n}\n```\n[arXiv e-print](https://arxiv.org/abs/1712.03400)\n\n\n## Abstract\n\nWe review some of the most recent approaches to colorize gray-scale images using deep learning methods. Inspired by these, we propose a model which combines a deep Convolutional Neural Network trained from scratch with high-level features extracted from the Inception-ResNet-v2 pre-trained model. Thanks to its fully convolutional architecture, our encoder-decoder model can process images of any size and aspect ratio. Other than presenting the training results, we assess the \"public acceptance\" of the generated images by means of a user study. Finally, we present a carousel of applications on different types of images, such as historical photographs.\n\n\u003c!---\n## Intro\nWe got the inspiration from the work of Richard Zhang, Phillip Isola and Alexei A. Efros, who realized a network able to colorize black and white images ([blog post](https://richzhang.github.io/colorization/) and [paper](https://arxiv.org/abs/1603.08511)). They trained a network on ImageNet pictures preprocessed to make them gray-scale, with the colored image as the output target.\n\nThen we also saw the experiments of Satoshi Iizuka, Edgar Simo-Serra and Hiroshi Ishikawa, who added image classification features to raw pixels fed to the network, improving the overall results ([YouTube review](https://www.youtube.com/watch?v=MfaTOXxA8dM), [blog post](http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/en/) and [paper](http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/data/colorization_sig2016.pdf)).\n--\u003e\n\n## Project overview\nInspired by [Iizuka and Simo-Serra *et al.* (2016)](http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/data/colorization_sig2016.pdf), we combine a deep CNN architecture with [Inception-ResNet-v2](https://arxiv.org/abs/1602.07261) pre-trained on ImageNet dataset, which assists the overall colorization process by extracting high-level features. In particular, Inception-ResNet-v2\n\n![](assets/our_net.png)\n\n\u003c!---\nThe hidden layers of these models are learned to create a semantic representation of the image that is then used by the final layer (fully connected + softmax) to label the objects in the image. By “cutting” the model at one of its final layers we will get a high dimensional representation of image features, that will be used by our network to perform the colorization task (TensorFlow [tutorial](https://www.tensorflow.org/tutorials/image_retraining) on transfer learning, another [tutorial](https://kwotsin.github.io/tech/2017/02/11/transfer-learning.html) and arXiv [paper](https://arxiv.org/abs/1403.6382)).\n--\u003e\n\nThe _fusion_ between the fixed-size embedding and the intermediary result of the convolutions is performed by means of replication and stacking as described in [Iizuka and Simo-Serra *et al.* (2016)](http://hi.cs.waseda.ac.jp/~iizuka/projects/colorization/data/colorization_sig2016.pdf).\n\n![Fusion](assets/fusion_layer.png)\n\nWe have used the MSE loss as the objective function.\n\nThe Training data for this experiment could come from any source. We decuded to use [ImageNet](http://www.image-net.org), which nowadays is considered the de-facto reference for image tasks. This way, it makes easier for others to replicate our experiments.\n\n## Results\n\n#### ImageNet\n\n![ImageNet 1](assets/comparison.png)\n\n#### Historical pictures\n\n![Historical 1](assets/historical.png)\n\n---\n\n## Use the code\n\nRefer to [INSTRUCTIONS](INSTRUCTIONS.md) to install and use the code in this repo.\n\n## Community\n\n### Thanks to the people who noticed our work!\n\nWe are proud if our work gets noticed and helps/inspires other people on their path to knowledge. Here's a list of references we are aware of, some of the authors contacted us, some others we just happened to find online:\n\n- François Chollet [tweeted](https://twitter.com/fchollet/status/917846097430638592) about this project (thank you for Keras)\n- Emil Wallnér on [FloydHub Blog](https://blog.floydhub.com/colorizing-b\u0026w-photos-with-neural-networks/) and [freecodecamp](https://medium.freecodecamp.org/colorize-b-w-photos-with-a-100-line-neural-network-53d9b4449f8d)\n- Amir Kalron on [Logz.io Blog](https://logz.io/blog/open-source-machine-learning/)\n- sparkexpert on [CSDN](http://blog.csdn.net/sparkexpert/article/details/74452523)\n- Eryk Lewinson on [Medium](https://towardsdatascience.com/image-colorization-using-convolutional-autoencoders-fdabc1cb1dbe)\n\n### Projects originated from here\n- _[Coloring Black and White Images with Neural Networks](https://github.com/emilwallner/Coloring-greyscale-images)_, by [emilwallner](https://github.com/emilwallner).\n- _[Ensemble Image Colorization using Convolutional Neural Networks with Refinement Network](https://github.com/Kriztoper/deep-koalarization)_, by [Kriztoper](https://github.com/Kriztoper).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbaldassarrefe%2Fdeep-koalarization","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbaldassarrefe%2Fdeep-koalarization","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbaldassarrefe%2Fdeep-koalarization/lists"}