{"id":13586944,"url":"https://github.com/UE2020/colorize","last_synced_at":"2025-04-07T18:35:19.067Z","repository":{"id":208904013,"uuid":"702293353","full_name":"UE2020/colorize","owner":"UE2020","description":"A deep learning image \u0026 video colorizer","archived":false,"fork":false,"pushed_at":"2023-11-25T18:35:05.000Z","size":44245,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-02-13T21:43:50.207Z","etag":null,"topics":["colorization","image-colorization","libtorch","pix2pix","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/UE2020.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-10-09T03:07:38.000Z","updated_at":"2024-01-17T22:40:11.000Z","dependencies_parsed_at":"2023-11-23T23:22:42.437Z","dependency_job_id":"49029fbe-0991-48db-a8b2-af50d83e15de","html_url":"https://github.com/UE2020/colorize","commit_stats":null,"previous_names":["ue2020/colorize"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UE2020%2Fcolorize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UE2020%2Fcolorize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UE2020%2Fcolorize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/UE2020%2Fcolorize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/UE2020","download_url":"https://codeload.github.com/UE2020/colorize/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223290311,"owners_count":17120886,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["colorization","image-colorization","libtorch","pix2pix","rust"],"created_at":"2024-08-01T15:05:55.538Z","updated_at":"2024-11-06T05:30:57.894Z","avatar_url":"https://github.com/UE2020.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# Colorize!\nA deep learning image \u0026amp; video colorizer using Rust and libtorch. The model is a slightly modified pix2pix (Isola et al.). A video demos are available here:\n\n[The Three Stooges Episode 117 (Malice In The Palace) colorized](https://www.youtube.com/watch?v=F3TNbHVFwqw)\n\n[MLK Interview colorized](https://www.youtube.com/watch?v=OosQa905nYQ)\n\n## Training\nTo initialize the model, you'll need to run `src/transform.py` and `src/model.py` to initialize the LAB\u003c-\u003eRGB and pre-trained generator torchscripts, respectively.\nThis requires PyTorch and the fastai library.\n\nTraining is as simple as running with the following arguments, where use_gan is a boolean argument:\n```\n./target/release/autoencoder train starting_model.pt /data/path duration_in_hours use_gan\n```\n\nSee below for pre-trained model.\n\n### Obtaining a dataset\nThe ImageNet Object Localization Challenge dataset (a subset of the full ImageNet dataset) is available on Kaggle,\nand was used to train the baseline model. A diverse sampling of images is recommended to avoid overfitting.\n\nAny dataset that consists of images in a folder is usable, as long as there are no corrupted images or non-image files. Subdirectories will be crawled automatically.\n\n### 3-Step Training Procedure\nModels are trained in three steps to reduce the undesirable visual artifacts caused by GAN training:\n1. Train for a long time without the discriminator network (use_gan = false).\n2. Continue training the network produced by the previous step for a shorter time with the discriminator network enabled (use_gan = true).\n3. **Merge** the two resulting networks using the pre-defined weighted average formula: `./target/release/autoencoder merge gan.pt no_gan.pt` (order matters). The merged model will be saved to `./merged.pt`, beware of overwriting any model that may have already been there.\n\n## Running\nRunning the model is as simple as:\n```\n./target/release/autoencoder test model.pt image.jpg image_size\n```\nImages will be written to `./fixed.jpg`.\nOnly powers of 2 may be used for the image_size parameter, although 256 is recommended, 512 and 1024 are useful for colorizing fine details.\n\nA pre-trained model is available here: \nhttps://drive.google.com/file/d/1S6wAA-YkJsOVdh5-oHC6DkyPvfWiACA7/view?usp=sharing\n\n## Demo\n\nColorizing legacy photos:\n\n\u003cimg src=\"https://i.imgur.com/LDGGVt0.png\" width=\"400\"\u003e\n\n\u003cimg src=\"https://i.imgur.com/O0Lhm75.jpeg\" width=\"400\"\u003e\n\n\u003cimg src=\"https://i.imgur.com/MefnRvW.jpeg\" width=\"400\"\u003e\n\n\u003cimg src=\"https://i.imgur.com/ly1q00t.jpeg\" width=\"400\"\u003e\n\n\u003cimg src=\"https://i.imgur.com/sulnni9.jpeg\" width=\"400\"\u003e\n\n## Credits\nAlthough it's currently unused, the multi-scale discriminator implementation in `src/model.py` is courtesy of https://github.com/NVIDIA/pix2pixHD.\n\n## Citation\nThe model is based on the following papers:\n\n```\n@article{pix2pix2017,\n  title={Image-to-Image Translation with Conditional Adversarial Networks},\n  author={Isola, Phillip and Zhu, Jun-Yan and Zhou, Tinghui and Efros, Alexei A},\n  journal={CVPR},\n  year={2017}\n}\n```\n\n```\n@inproceedings{wang2018pix2pixHD,\n  title={High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs},\n  author={Ting-Chun Wang and Ming-Yu Liu and Jun-Yan Zhu and Andrew Tao and Jan Kautz and Bryan Catanzaro},  \n  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},\n  year={2018}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUE2020%2Fcolorize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FUE2020%2Fcolorize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FUE2020%2Fcolorize/lists"}