{"id":29009900,"url":"https://github.com/tencentarc/colorflow","last_synced_at":"2025-06-25T15:33:39.889Z","repository":{"id":268486253,"uuid":"904199028","full_name":"TencentARC/ColorFlow","owner":"TencentARC","description":"The official implementation of paper \"ColorFlow: Retrieval-Augmented Image Sequence Colorization\"","archived":false,"fork":false,"pushed_at":"2025-04-16T09:08:29.000Z","size":82076,"stargazers_count":394,"open_issues_count":8,"forks_count":31,"subscribers_count":8,"default_branch":"main","last_synced_at":"2025-04-16T12:15:18.566Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://zhuang2002.github.io/ColorFlow/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TencentARC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-12-16T12:40:28.000Z","updated_at":"2025-04-16T09:08:40.000Z","dependencies_parsed_at":null,"dependency_job_id":"6788a698-403a-4cbb-a90c-7da1b0ee45ec","html_url":"https://github.com/TencentARC/ColorFlow","commit_stats":null,"previous_names":["tencentarc/colorflow"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TencentARC/ColorFlow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FColorFlow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FColorFlow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FColorFlow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FColorFlow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TencentARC","download_url":"https://codeload.github.com/TencentARC/ColorFlow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FColorFlow/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":261901407,"owners_count":23227593,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-25T15:33:38.481Z","updated_at":"2025-06-25T15:33:39.852Z","avatar_url":"https://github.com/TencentARC.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🎨 ColorFlow\n\n*Retrieval-Augmented Image Sequence Colorization*\n\n**Authors:** Junhao Zhuang, Xuan Ju, Zhaoyang Zhang, Yong Liu, Shiyi Zhang, Chun Yuan, Ying Shan\n\n\u003ca href='https://zhuang2002.github.io/ColorFlow/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e \u0026nbsp;\n\u003ca href='https://huggingface.co/spaces/TencentARC/ColorFlow'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Demo-blue'\u003e\u003c/a\u003e \u0026nbsp;\n\u003ca href=\"https://arxiv.org/abs/2412.11815\"\u003e\u003cimg src=\"https://img.shields.io/badge/arXiv-2412.11815-b31b1b.svg\"\u003e\u003c/a\u003e \u0026nbsp;\n\u003ca href=\"https://huggingface.co/TencentARC/ColorFlow\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue\"\u003e\u003c/a\u003e\n\n**Your star means a lot for us to develop this project!** :star:\n\n\u003cimg src='https://zhuang2002.github.io/ColorFlow/fig/teaser.png'/\u003e\n\n### 🌟 Abstract \n\nAutomatic black-and-white image sequence colorization while preserving character and object identity (ID) is a complex task with significant market demand, such as in cartoon or comic series colorization. Despite advancements in visual colorization using large-scale generative models like diffusion models, challenges with controllability and identity consistency persist, making current solutions unsuitable for industrial application.\n\nTo address this, we propose **ColorFlow**, a three-stage diffusion-based framework tailored for image sequence colorization in industrial applications. Unlike existing methods that require per-ID finetuning or explicit ID embedding extraction, we propose a novel robust and generalizable **Retrieval Augmented Colorization** pipeline for colorizing images with relevant color references.\n\nOur pipeline also features a dual-branch design: one branch for color identity extraction and the other for colorization, leveraging the strengths of diffusion models. We utilize the self-attention mechanism in diffusion models for strong in-context learning and color identity matching.\n\nTo evaluate our model, we introduce **ColorFlow-Bench**, a comprehensive benchmark for reference-based colorization. Results show that ColorFlow outperforms existing models across multiple metrics, setting a new standard in sequential image colorization and potentially benefiting the art industry.\n\n### 📰 News\n\n- **Update Date:** December 23, 2024 - We have released the weights for the Sketch_Shading model, along with updates to the related code and demo. You can access the model weights in our [Hugging Face model repository](https://huggingface.co/TencentARC/ColorFlow) and explore the updated demo [here](https://huggingface.co/spaces/TencentARC/ColorFlow). 🎉🔥\n\n- **Release Date:** December 17, 2024 - The inference code and model weights have also been released! 🎉\n\n### 📋 TODO\n\n- ✅ Release inference code and model weights\n- ⬜️ Release training code\n\n### 🚀 Getting Started\n\nFollow these steps to set up and run ColorFlow on your local machine:\n\n- **Clone the Repository**\n  \n  Download the code from our GitHub repository:\n  ```bash\n  git clone https://github.com/TencentARC/ColorFlow\n  cd ColorFlow\n  ```\n\n- **Set Up the Python Environment**\n\n  Ensure you have Anaconda or Miniconda installed, then create and activate a Python environment and install required dependencies:\n  ```bash\n  conda create -n colorflow python=3.8.5\n  conda activate colorflow\n  pip install -r requirements.txt\n  ```\n\n- **Run the Application**\n\n  You can launch the Gradio interface for ColorFlow by running the following command:\n  ```bash\n  python app.py\n  ```\n\n- **Access ColorFlow in Your Browser**\n\n  Open your browser and go to `http://localhost:7860`. If you're running the app on a remote server, replace `localhost` with your server's IP address or domain name. To use a custom port, update the `server_port` parameter in the `demo.launch()` function of app.py.\n\n### 🎉 Demo\n\nYou can [try the demo](https://huggingface.co/spaces/TencentARC/ColorFlow) of ColorFlow on Hugging Face Space.\n\n### 🛠️ Method\n\nThe overview of ColorFlow. This figure presents the three primary components of our framework: the **Retrieval-Augmented Pipeline (RAP)**, the **In-context Colorization Pipeline (ICP)**, and the **Guided Super-Resolution Pipeline (GSRP)**. Each component is essential for maintaining the color identity of instances across black-and-white image sequences while ensuring high-quality colorization.\n\n\u003cimg src=\"https://zhuang2002.github.io/ColorFlow/fig/flowchart.png\" width=\"1000\"\u003e\n\n🤗 We welcome your feedback, questions, or collaboration opportunities. Thank you for trying ColorFlow!\n\n### 📄 Acknowledgments\n\nWe would like to acknowledge the following open-source projects that have inspired and contributed to the development of ColorFlow:\n\n- **ScreenStyle**: https://github.com/msxie92/ScreenStyle\n- **MangaLineExtraction_PyTorch**: https://github.com/ljsabc/MangaLineExtraction_PyTorch\n\nWe are grateful for the valuable resources and insights provided by these projects.\n\n### 📞 Contact\n\n- **Junhao Zhuang**  \n  Email: [zhuangjh23@mails.tsinghua.edu.cn](mailto:zhuangjh23@mails.tsinghua.edu.cn)\n\n### 📜 Citation\n\n```\n@misc{zhuang2024colorflow,\ntitle={ColorFlow: Retrieval-Augmented Image Sequence Colorization},\nauthor={Junhao Zhuang and Xuan Ju and Zhaoyang Zhang and Yong Liu and Shiyi Zhang and Chun Yuan and Ying Shan},\nyear={2024},\neprint={2412.11815},\narchivePrefix={arXiv},\nprimaryClass={cs.CV},\nurl={https://arxiv.org/abs/2412.11815},\n}\n```\n\n### 📄 License\n\nPlease refer to our [license file](LICENSE) for more details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fcolorflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftencentarc%2Fcolorflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftencentarc%2Fcolorflow/lists"}