{"id":27577455,"url":"https://github.com/Zhuang2002/Cobra","last_synced_at":"2025-04-21T18:02:11.628Z","repository":{"id":288289583,"uuid":"967522645","full_name":"zhuang2002/Cobra","owner":"zhuang2002","description":"[SIGGRAPH 2025] Official code of the paper \"Cobra: Efficient Line Art COlorization with BRoAder References\"","archived":false,"fork":false,"pushed_at":"2025-04-16T20:54:53.000Z","size":31829,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-16T22:57:24.195Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zhuang2002.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-04-16T15:27:12.000Z","updated_at":"2025-04-16T20:54:56.000Z","dependencies_parsed_at":"2025-04-16T23:14:06.071Z","dependency_job_id":null,"html_url":"https://github.com/zhuang2002/Cobra","commit_stats":null,"previous_names":["zhuang2002/cobra"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhuang2002%2FCobra","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhuang2002%2FCobra/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhuang2002%2FCobra/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zhuang2002%2FCobra/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zhuang2002","download_url":"https://codeload.github.com/zhuang2002/Cobra/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250107882,"owners_count":21376019,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-04-21T18:01:24.362Z","updated_at":"2025-04-21T18:02:11.605Z","avatar_url":"https://github.com/zhuang2002.png","language":"Python","funding_links":[],"categories":["Colorization"],"sub_categories":[],"readme":"# 🎨 Cobra\n\n**Efficient Line Art COlorization with BRoAder References**\n\n**Authors:** Junhao Zhuang, Lingen Li, Xuan Ju, Zhaoyang Zhang, Chun Yuan† and Ying Shan†\n\n\u003ca href='https://zhuang2002.github.io/Cobra/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e \u0026nbsp;\n\u003ca href='https://huggingface.co/spaces/JunhaoZhuang/Cobra'\u003e\u003cimg src='https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Demo-blue'\u003e\u003c/a\u003e \u0026nbsp;\n\u003ca href=\"https://arxiv.org/abs/2504.12240\"\u003e\u003cimg src=\"https://img.shields.io/badge/arXiv-2504.12240-b31b1b.svg\"\u003e\u003c/a\u003e \u0026nbsp;\n\u003ca href=\"https://huggingface.co/JunhaoZhuang/Cobra\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-Model-blue\"\u003e\u003c/a\u003e\n\n**Your star means a lot for us to develop this project!** :star:\n\n\u003cimg src='./figs/teaser.png'/\u003e\n\n### 🌟 Abstract \n\nThe comic production industry requires reference-based line art colorization with high accuracy, efficiency, contextual consistency, and flexible control.\nA comic page often involves diverse characters, objects, and backgrounds, which complicates the coloring process.\nDespite advancements in diffusion models for image generation, their application in line art colorization remains limited, facing challenges related to handling extensive reference images, time-consuming inference, and flexible control.\nWe investigate the necessity of extensive contextual image guidance on the quality of line art colorization. To address these challenges, we introduce **Cobra**, an efficient and versatile method that supports color hints and utilizes **over 200 reference images** while maintaining low latency. \nCentral to Cobra is a Causal Sparse DiT architecture, which leverages specially designed positional encodings, causal sparse attention, and Key-Value Cache to effectively manage long-context references and ensure color identity consistency.\nResults demonstrate that Cobra achieves accurate line art colorization through extensive contextual reference, significantly enhancing inference speed and interactivity, thereby meeting critical industrial demands.\n\n### 📰 News\n\n- **Release Date:** April 17, 2025 - The inference code and model weights have also been released! 🎉\n\n### 📋 TODO\n\n- ✅ Release inference code and model weights\n- ⬜️ Release training code\n\n### 🚀 Getting Started\n\nFollow these steps to set up and run Cobra on your local machine:\n\n- **Clone the Repository**\n  \n  Download the code from our GitHub repository:\n  ```bash\n  git clone https://github.com/zhuang2002/Cobra\n  cd Cobra\n  ```\n\n- **Set Up the Python Environment**\n\n  Ensure you have Anaconda or Miniconda installed, then create and activate a Python environment and install required dependencies:\n  ```bash\n  conda create -n cobra python=3.11.11\n  conda activate cobra\n  pip install -r requirements.txt\n  ```\n\n- **Run the Application**\n\n  You can launch the Gradio interface for Cobra by running the following command:\n  ```bash\n  python app.py\n  ```\n\n- **Access Cobra in Your Browser**\n\n  Open your browser and go to `http://localhost:7860`. If you're running the app on a remote server, replace `localhost` with your server's IP address or domain name. To use a custom port, update the `server_port` parameter in the `demo.launch()` function of app.py.\n\n### 🎉 Demo\n\nYou can [try the demo](https://huggingface.co/spaces/JunhaoZhuang/Cobra) of Cobra on Hugging Face Space.\n\n### 🛠️ Method\n\nThe overview of Cobra.\nThis figure depicts the framework of Cobra, which utilizes a large collection of retrieved reference images to guide the colorization of comic line art. The framework effectively manages an arbitrary number of contextual image references through localized reusable positional encoding, ensuring appropriate aspect ratios and resolutions. Additionally, the causal sparse DiT architecture processes long contextual references, enhancing identity preservation and color accuracy while reducing computational complexity. The integration of optional color hints further ensures user flexibility, culminating in high-quality coloring that is highly suitable for industrial applications.\n\n\u003cimg src=\"./figs/flowchart.png\" width=\"1000\"\u003e\n\n🤗 We welcome your feedback, questions, or collaboration opportunities. Thank you for trying Cobra!\n\n### 📄 Acknowledgments\n\nWe would like to acknowledge the following open-source projects that have inspired and contributed to the development of Cobra:\n\n- **MangaLineExtraction_PyTorch**: https://github.com/ljsabc/MangaLineExtraction_PyTorch\n\nWe are grateful for the valuable resources and insights provided by these projects.\n\n### 📞 Contact\n\n- **Junhao Zhuang**  \n  Email: [zhuangjh23@mails.tsinghua.edu.cn](mailto:zhuangjh23@mails.tsinghua.edu.cn)\n\n### 📜 Citation\n\n```\n@misc{zhuang2025cobraefficientlineart,\n      title={Cobra: Efficient Line Art COlorization with BRoAder References}, \n      author={Junhao Zhuang and Lingen Li and Xuan Ju and Zhaoyang Zhang and Chun Yuan and Ying Shan},\n      year={2025},\n      eprint={2504.12240},\n      archivePrefix={arXiv},\n      primaryClass={cs.CV},\n      url={https://arxiv.org/abs/2504.12240}, \n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZhuang2002%2FCobra","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FZhuang2002%2FCobra","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FZhuang2002%2FCobra/lists"}