{"id":13717339,"url":"https://github.com/richzhang/colorization-pytorch","last_synced_at":"2025-04-04T15:08:48.375Z","repository":{"id":39338791,"uuid":"144341337","full_name":"richzhang/colorization-pytorch","owner":"richzhang","description":"PyTorch reimplementation of Interactive Deep Colorization","archived":false,"fork":false,"pushed_at":"2020-06-04T21:39:31.000Z","size":11309,"stargazers_count":600,"open_issues_count":15,"forks_count":114,"subscribers_count":18,"default_branch":"master","last_synced_at":"2025-03-28T14:07:20.950Z","etag":null,"topics":["computer-graphics","computer-vision","convolutional-neural-networks","deep-learning","image-colorization","pytorch","siggraph"],"latest_commit_sha":null,"homepage":"https://richzhang.github.io/ideepcolor/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/richzhang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-08-10T23:42:30.000Z","updated_at":"2025-03-02T05:26:57.000Z","dependencies_parsed_at":"2022-07-14T03:40:30.370Z","dependency_job_id":null,"html_url":"https://github.com/richzhang/colorization-pytorch","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richzhang%2Fcolorization-pytorch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richzhang%2Fcolorization-pytorch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richzhang%2Fcolorization-pytorch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/richzhang%2Fcolorization-pytorch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/richzhang","download_url":"https://codeload.github.com/richzhang/colorization-pytorch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247198461,"owners_count":20900080,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-graphics","computer-vision","convolutional-neural-networks","deep-learning","image-colorization","pytorch","siggraph"],"created_at":"2024-08-03T00:01:20.937Z","updated_at":"2025-04-04T15:08:48.340Z","avatar_url":"https://github.com/richzhang.png","language":"Python","funding_links":[],"categories":["Pytorch \u0026 related libraries｜Pytorch \u0026 相关库","Pytorch \u0026 related libraries","Python"],"sub_categories":["Other libraries｜其他库:","Other libraries:"],"readme":"# Interactive Deep Colorization in PyTorch\n[Project Page](https://richzhang.github.io/ideepcolor/) |  [Paper](https://arxiv.org/abs/1705.02999) | [Video](https://youtu.be/eL5ilZgM89Q) | [Talk](https://www.youtube.com/watch?v=rp5LUSbdsys) | [UI code](https://github.com/junyanz/interactive-deep-colorization/)\n\n\u003cimg src='imgs/demo.gif' width=600\u003e  \n\nReal-Time User-Guided Image Colorization with Learned Deep Priors.  \n[Richard Zhang](https://richzhang.github.io/)\\*, [Jun-Yan Zhu](http://people.csail.mit.edu/junyanz/)\\*, [Phillip Isola](http://people.eecs.berkeley.edu/~isola/), [Xinyang Geng](http://young-geng.xyz/), Angela S. Lin, Tianhe Yu, and [Alexei A. Efros](https://people.eecs.berkeley.edu/~efros/).  \nIn ACM Transactions on Graphics (SIGGRAPH 2017).  \n\nThis is our PyTorch reimplementation for interactive image colorization, written by [Richard Zhang](https://github.com/richzhang) and [Jun-Yan Zhu](https://github.com/junyanz).\n\nThis repository contains training usage. The original, official GitHub repo (with an interactive GUI, and originally Caffe backend) is [here](https://richzhang.github.io/ideepcolor/). The official repo has been updated to support PyTorch models on the backend, which can be trained in this repository.\n\n## Prerequisites\n- Linux or macOS\n- Python 2 or 3\n- CPU or NVIDIA GPU + CUDA CuDNN\n\n## Getting Started\n### Installation\n- Install PyTorch 0.4+ and torchvision from http://pytorch.org and other dependencies (e.g., [visdom](https://github.com/facebookresearch/visdom) and [dominate](https://github.com/Knio/dominate)). You can install all the dependencies by\n```bash\npip install -r requirements.txt\n```\n- Clone this repo:\n```bash\ngit clone https://github.com/richzhang/colorization-pytorch\ncd colorization-pytorch\n```\n\n### Dataset preparation\n- Download the ILSVRC 2012 dataset and run the following script to prepare data\n`python make_ilsvrc_dataset.py --in_path /PATH/TO/ILSVRC12`. This will make symlinks into the training set, and divide the ILSVRC validation set into validation and test splits for colorization.\n\n### Training interactive colorization\n- Train a model: ```bash ./scripts/train_siggraph.sh```. This is a 2 stage training process. First, the network is trained for automatic colorization using classification loss. Results are in `./checkpoints/siggraph_class`. Then, the network is fine-tuned for interactive colorization using regression loss. Final results are in `./checkpoints/siggraph_reg2`.\n\n- To view training results and loss plots, run `python -m visdom.server` and click the URL http://localhost:8097. The following values are monitored:\n    * `G_CE` is a cross-entropy loss between predicted color distribution and ground truth color.\n    * `G_entr` is the entropy of the predicted distribution.\n    * `G_entr_hint` is the entropy of the predicted distribution at points where a color hint is given.\n    * `G_L1_max` is the L1 distance between the ground truth color and argmax of the predicted color distribution.\n    * `G_L1_mean` is the L1 distance between the ground truth color and mean of the predicted color distribution.\n    * `G_L1_reg` is the L1 distance between the ground truth color and the predicted color.\n    * `G_fake_real` is the L1 distance between the predicted color and the ground truth color (in locations where a hint is given).\n    * `G_fake_hint` is the L1 distance between the predicted color and the input hint color (in locations where a hint is given). It's a measure of how much the network \"trusts\" the input hint.\n    * `G_real_hint` is the L1 distance between the ground truth color and the input hint color (in locations where a hint is given).\n\n\n### Testing interactive colorization\n- Get a model. Either:\n    * (1) download the pretrained model by running ```bash pretrained_models/download_siggraph_model.sh```, which will give you a few models.\n        * Original caffe weights [Recommended] `./checkpoints/siggraph_caffemodel/latest_net_G.pth` is the original caffemodel weights, converted to PyTorch. It is recommended. Be sure to set `--mask_cent 0` when running it.\n        * Retrained model: `./checkpoints/siggraph_retrained/latest_net_G.pth`. The model achieves better PSNR but performs qualitatively differently. Note that this repository is an approximate reimplementation of the siggraph paper.\n    * (2) train your own model (as described in the section above), which will leave a model in `./checkpoints/siggraph_reg2/latest_net_G.pth`\n\n- Test the model on validation data:\n    * ```python test.py --name siggraph_caffemodel --mask_cent 0``` for original caffemodel weights\n    * ```python test.py --name siggraph_retrained ``` for retrained weights.\n    * ```python test.py --name siggraph_reg2 ``` if you retrained your own model\n    The test results will be saved to an HTML file in `./results/[[NAME]]/latest_val/index.html`. For each image in the validation set, it will test (1) automatic colorization, (2) interactive colorization with a few random hints, and (3) interactive colorization with lots of random hints.\n\n- Test the model by making PSNR vs. the number of hints plot: ```python test_sweep.py --name [[NAME]] ```. This plot was used in Figure 6 of the [paper](https://arxiv.org/abs/1705.02999). This test randomly reveals 6x6 color hint patches to the network and sees how accurate the colorization is with respect to the ground truth.\n\n   \u003cimg src=\"./checkpoints/siggraph_pretrained/sweep_reference.png\" height=\"300\"/\u003e\n\n- Test the model interactively with the original official [repository](https://github.com/junyanz/interactive-deep-colorization). Follow installation instructions in that repo and run `python ideepcolor.py --backend pytorch --color_model [[PTH/TO/MODEL]] --dist_model [[PTH/TO/MODEL]]`.\n\n\n### Citation\nIf you use this code for your research, please cite our paper:\n```\n@article{zhang2017real,\n  title={Real-Time User-Guided Image Colorization with Learned Deep Priors},\n  author={Zhang, Richard and Zhu, Jun-Yan and Isola, Phillip and Geng, Xinyang and Lin, Angela S and Yu, Tianhe and Efros, Alexei A},\n  journal={ACM Transactions on Graphics (TOG)},\n  volume={9},\n  number={4},\n  year={2017},\n  publisher={ACM}\n}\n```\n\n## Acknowledgments\nThis code borrows heavily from the [pytorch-CycleGAN](https://github.com/junyanz/pytorch-CycleGAN-and-pix2pix) repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frichzhang%2Fcolorization-pytorch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frichzhang%2Fcolorization-pytorch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frichzhang%2Fcolorization-pytorch/lists"}