{"id":14399443,"url":"https://github.com/piddnad/DDColor","last_synced_at":"2025-08-24T10:31:22.225Z","repository":{"id":104747569,"uuid":"581456375","full_name":"piddnad/DDColor","owner":"piddnad","description":"[ICCV 2023] DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders","archived":false,"fork":false,"pushed_at":"2025-04-14T05:19:45.000Z","size":14765,"stargazers_count":1297,"open_issues_count":14,"forks_count":135,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-05-28T22:32:02.636Z","etag":null,"topics":["computer-vision","image-colorization","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/piddnad.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-12-23T08:36:07.000Z","updated_at":"2025-05-27T13:47:32.000Z","dependencies_parsed_at":"2024-03-15T04:35:11.484Z","dependency_job_id":"f901c4ae-2ce4-4df9-8bab-cf56ecfe2172","html_url":"https://github.com/piddnad/DDColor","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/piddnad/DDColor","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piddnad%2FDDColor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piddnad%2FDDColor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piddnad%2FDDColor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piddnad%2FDDColor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/piddnad","download_url":"https://codeload.github.com/piddnad/DDColor/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/piddnad%2FDDColor/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":271846003,"owners_count":24833046,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-24T02:00:11.135Z","response_time":111,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","image-colorization","pytorch"],"created_at":"2024-08-29T06:01:17.309Z","updated_at":"2025-08-24T10:31:22.214Z","avatar_url":"https://github.com/piddnad.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# 🎨 DDColor\n[![arXiv](https://img.shields.io/badge/arXiv-2212.11613-b31b1b.svg)](https://arxiv.org/abs/2212.11613)\n[![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Models-FF8000)](https://huggingface.co/piddnad/DDColor-models)\n[![ModelScope demo](https://img.shields.io/badge/%F0%9F%91%BE%20ModelScope-Demo-8A2BE2)](https://www.modelscope.cn/models/damo/cv_ddcolor_image-colorization/summary)\n[![Replicate](https://replicate.com/piddnad/ddcolor/badge)](https://replicate.com/piddnad/ddcolor)\n![visitors](https://visitor-badge.laobi.icu/badge?page_id=piddnad/DDColor)\n\nOfficial PyTorch implementation of ICCV 2023 Paper \"DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders\".\n\n\u003e Xiaoyang Kang, Tao Yang, Wenqi Ouyang, Peiran Ren, Lingzhi Li, Xuansong Xie  \n\u003e *DAMO Academy, Alibaba Group*\n\n🪄 DDColor can provide vivid and natural colorization for historical black and white old photos.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/teaser.webp\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n🎲 It can even colorize/recolor landscapes from anime games, transforming your animated scenery into a realistic real-life style! (Image source: Genshin Impact)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/anime_landscapes.webp\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n\n## News\n- [2024-01-28] Support inference via 🤗 Hugging Face! Thanks @[Niels](https://github.com/NielsRogge) for the suggestion and example code and @[Skwara](https://github.com/Skwarson96) for fixing bug.\n- [2024-01-18] Add Replicate demo and API! Thanks @[Chenxi](https://github.com/chenxwh).\n- [2023-12-13] Release the DDColor-tiny pre-trained model!\n- [2023-09-07] Add the Model Zoo and release three pretrained models!\n- [2023-05-15] Code release for training and inference!\n- [2023-05-05] The online demo is available!\n\n\n## Online Demo\nTry our online demos at [ModelScope](https://www.modelscope.cn/models/damo/cv_ddcolor_image-colorization/summary) and [Replicate](https://replicate.com/piddnad/ddcolor).\n\n\n## Methods\n*In short:* DDColor uses multi-scale visual features to optimize **learnable color tokens** (i.e. color queries) and achieves state-of-the-art performance on automatic image colorization.\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/network_arch.jpg\" width=\"100%\"\u003e\n\u003c/p\u003e\n\n\n## Installation\n### Requirements\n- Python \u003e= 3.7\n- PyTorch \u003e= 1.7\n\n### Installation with conda (recommended)\n\n```sh\nconda create -n ddcolor python=3.9\nconda activate ddcolor\npip install torch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 --index-url https://download.pytorch.org/whl/cu118\n\npip install -r requirements.txt\n\n# Install basicsr, only required for training\npython3 setup.py develop  \n```\n\n## Quick Start\n### Inference Using Local Script (No `basicsr` Required)\n1. Download the pretrained model:\n\n```python\nfrom modelscope.hub.snapshot_download import snapshot_download\n\nmodel_dir = snapshot_download('damo/cv_ddcolor_image-colorization', cache_dir='./modelscope')\nprint('model assets saved to %s' % model_dir)\n```\n\n2.\tRun inference with\n\n```sh\npython infer.py --model_path ./modelscope/damo/cv_ddcolor_image-colorization/pytorch_model.pt --input ./assets/test_images\n```\nor\n```sh\nsh scripts/inference.sh\n```\n\n### Inference Using Hugging Face \nLoad the model via Hugging Face Hub:\n\n```python\nfrom infer_hf import DDColorHF\n\nddcolor_paper_tiny = DDColorHF.from_pretrained(\"piddnad/ddcolor_paper_tiny\")\nddcolor_paper      = DDColorHF.from_pretrained(\"piddnad/ddcolor_paper\")\nddcolor_modelscope = DDColorHF.from_pretrained(\"piddnad/ddcolor_modelscope\")\nddcolor_artistic   = DDColorHF.from_pretrained(\"piddnad/ddcolor_artistic\")\n```\n\nCheck `infer_hf.py` for the details of the inference, or directly perform model inference by running:\n\n```sh\npython infer_hf.py --model_name ddcolor_modelscope --input ./assets/test_images\n# model_name: [ddcolor_paper | ddcolor_modelscope | ddcolor_artistic | ddcolor_paper_tiny]\n```\n\n### Inference Using ModelScope\n1. Install modelscope:\n\n```sh\npip install modelscope\n```\n\n2. Run inference:\n\n```python\nimport cv2\nfrom modelscope.outputs import OutputKeys\nfrom modelscope.pipelines import pipeline\nfrom modelscope.utils.constant import Tasks\n\nimg_colorization = pipeline(Tasks.image_colorization, model='damo/cv_ddcolor_image-colorization')\nresult = img_colorization('https://modelscope.oss-cn-beijing.aliyuncs.com/test/images/audrey_hepburn.jpg')\ncv2.imwrite('result.png', result[OutputKeys.OUTPUT_IMG])\n```\n\nThis code will automatically download the `ddcolor_modelscope` model (see [ModelZoo](#model-zoo)) and performs inference. The model file `pytorch_model.pt` can be found in the local path `~/.cache/modelscope/hub/damo`.\n\n### Gradio Demo\nInstall the gradio and other required libraries:\n\n```sh\npip install gradio gradio_imageslider timm\n```\n\nThen, you can run the demo with the following command:\n\n```sh\npython gradio_app.py\n```\n\n## Model Zoo\nWe provide several different versions of pretrained models, please check out [Model Zoo](MODEL_ZOO.md).\n\n\n## Train\n1. Dataset Preparation: Download the [ImageNet](https://www.image-net.org/) dataset or create a custom dataset. Use this script to obtain the dataset list file:\n\n```sh\npython data_list/get_meta_file.py\n```\n\n2. Download the pretrained weights for [ConvNeXt](https://dl.fbaipublicfiles.com/convnext/convnext_large_22k_224.pth) and [InceptionV3](https://download.pytorch.org/models/inception_v3_google-1a9a5a14.pth) and place them in the `pretrain` folder.\n\n3. Specify 'meta_info_file' and other options in `options/train/train_ddcolor.yml`.\n\n4. Start training:\n\n```sh\nsh scripts/train.sh\n```\n\n## ONNX export\nSupport for ONNX model exports is available.\n\n1.\tInstall dependencies:\n\n```sh\npip install onnx==1.16.1 onnxruntime==1.19.2 onnxsim==0.4.36\n```\n\n2. Usage example:\n\n```sh\npython export.py\nusage: export.py [-h] [--input_size INPUT_SIZE] [--batch_size BATCH_SIZE] --model_path MODEL_PATH [--model_size MODEL_SIZE] \n[--decoder_type DECODER_TYPE] [--export_path EXPORT_PATH] [--opset OPSET]\n```\n\nDemo of ONNX export using a `ddcolor_paper_tiny` model is available [here](notebooks/colorization_pipeline_onnxruntime.ipynb).\n\n\n## Citation\n\nIf our work is helpful for your research, please consider citing:\n\n```\n@inproceedings{kang2023ddcolor,\n  title={DDColor: Towards Photo-Realistic Image Colorization via Dual Decoders},\n  author={Kang, Xiaoyang and Yang, Tao and Ouyang, Wenqi and Ren, Peiran and Li, Lingzhi and Xie, Xuansong},\n  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},\n  pages={328--338},\n  year={2023}\n}\n```\n\n## Acknowledgments\nWe thank the authors of BasicSR for the awesome training pipeline.\n\n\u003e Xintao Wang, Ke Yu, Kelvin C.K. Chan, Chao Dong and Chen Change Loy. BasicSR: Open Source Image and Video Restoration Toolbox. https://github.com/xinntao/BasicSR, 2020.\n\nSome codes are adapted from [ColorFormer](https://github.com/jixiaozhong/ColorFormer), [BigColor](https://github.com/KIMGEONUNG/BigColor), [ConvNeXt](https://github.com/facebookresearch/ConvNeXt), [Mask2Former](https://github.com/facebookresearch/Mask2Former), and [DETR](https://github.com/facebookresearch/detr). Thanks for their excellent work!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiddnad%2FDDColor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpiddnad%2FDDColor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpiddnad%2FDDColor/lists"}