{"id":24323918,"url":"https://github.com/ali-vilab/MangaNinjia","last_synced_at":"2025-09-27T08:31:21.614Z","repository":{"id":272402896,"uuid":"907251846","full_name":"ali-vilab/MangaNinjia","owner":"ali-vilab","description":null,"archived":false,"fork":false,"pushed_at":"2025-01-14T07:05:07.000Z","size":16010,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-14T08:19:36.084Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ali-vilab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-23T07:05:53.000Z","updated_at":"2025-01-14T07:05:10.000Z","dependencies_parsed_at":"2025-01-14T08:29:41.228Z","dependency_job_id":null,"html_url":"https://github.com/ali-vilab/MangaNinjia","commit_stats":null,"previous_names":["ali-vilab/manganinjia"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMangaNinjia","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMangaNinjia/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMangaNinjia/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ali-vilab%2FMangaNinjia/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ali-vilab","download_url":"https://codeload.github.com/ali-vilab/MangaNinjia/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234415643,"owners_count":18829133,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-01-17T19:01:14.335Z","updated_at":"2025-09-27T08:31:16.566Z","avatar_url":"https://github.com/ali-vilab.png","language":"Python","funding_links":[],"categories":["Colorization"],"sub_categories":[],"readme":"# MangaNinja: Line Art Colorization with Precise Reference Following\n\nThis repository represents the official implementation of the paper titled \"MangaNinja: Line Art Colorization with Precise Reference Following\".\n\n[![Website](docs/badge-website.svg)](https://johanan528.github.io/MangaNinjia/)\n[![Paper](https://img.shields.io/badge/arXiv-PDF-b31b1b)](https://arxiv.org/abs/2501.08332)\n[![License](https://img.shields.io/badge/License-CC%20BY--NC%204.0-929292)](https://creativecommons.org/licenses/by-nc/4.0/)\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://johanan528.github.io/\"\u003e\u003cstrong\u003eZhiheng Liu*\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://felixcheng97.github.io/\"\u003e\u003cstrong\u003eKa Leong Cheng*\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://xavierchen34.github.io/\"\u003e\u003cstrong\u003eXi Chen\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://jiexiaou.github.io/\"\u003e\u003cstrong\u003eJie Xiao\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://ken-ouyang.github.io/\"\u003e\u003cstrong\u003eHao Ouyang\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://scholar.google.com/citations?user=Mo_2YsgAAAAJ\u0026hl=zh-CN\"\u003e\u003cstrong\u003eKai Zhu\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://scholar.google.com/citations?user=8zksQb4AAAAJ\u0026hl=zh-CN\"\u003e\u003cstrong\u003eYu Liu\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://shenyujun.github.io/\"\u003e\u003cstrong\u003eYujun Shen\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://cqf.io/\"\u003e\u003cstrong\u003eQifeng Chen\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"http://luoping.me/\"\u003e\u003cstrong\u003ePing Luo\u003c/strong\u003e\u003c/a\u003e\n    \u003cbr\u003e\n  \u003c/p\u003e\n\nWe propose **MangaNinja**, a reference-based line art colorization method. MangaNinja\nautomatically aligns the reference with the line art for colorization, demonstrating remarkable consistency. Additionally, users can achieve\nmore complex tasks using point control. We hope that MangaNinja can accelerate the colorization process in the anime industry.\n\n![teaser](docs/teaser.gif)\n## 📢 News\n* 2025-01-15: Inference code and paper are released.\n* 2025-01-16: MangaNinja is available on windows, 6G VRAM need Auto install and Download Model. Thanks @sdbds ! You can found it [here](https://github.com/sdbds/MangaNinjia-for-windows). 🔥\n* 🏃: We will open an issue area to investigate user needs and adjust the model accordingly. This includes more memory-efficient structures, data formats for line art (such as binary line art), and considering retraining MangaNinjia on a better foundation model (sd3,flux).\n\n## 🛠️ Setup\n\n### 📦 Repository\n\nClone the repository (requires git):\n\n```bash\ngit clone https://github.com/ali-vilab/MangaNinjia.git\ncd MangaNinjia\n```\n\n### 💻 Dependencies\n\nInstall with `conda`: \n```bash\nconda env create -f environment.yaml\nconda activate MangaNinjia\n```\n### ⚙️ Weights\n* You could download them from HuggingFace: [StableDiffusion](https://modelscope.cn/models/AI-ModelScope/stable-diffusion-v1-5), [clip-vit-large-patch14](https://huggingface.co/openai/clip-vit-large-patch14), [control_v11p_sd15_lineart](https://huggingface.co/lllyasviel/control_v11p_sd15_lineart) and [Annotators](https://huggingface.co/lllyasviel/Annotators/blob/main/sk_model.pth)\n* You could download our [MangaNinjia model](https://huggingface.co/Johanan0528/MangaNinjia) from HuggingFace \n* The downloaded checkpoint directory has the following structure:\n```\n-- checkpoints\n    |-- StableDiffusion\n    |-- models\n        |-- clip-vit-large-patch14\n        |-- control_v11p_sd15_lineart\n        |-- Annotators\n            |--sk_model.pth\n    |-- MangaNinjia\n        |-- denoising_unet.pth\n        |-- reference_unet.pth\n        |-- point_net.pth\n        |-- controlnet.pth\n```\n\n\n## 🎮 Inference \n```bash\ncd scripts\nbash infer.sh\n```\n\nYou can find all results in `output/`. Enjoy!\n\n#### 📍 Inference settings\n\nThe default settings are optimized for the best result. However, the behavior of the code can be customized:\n  - `--denoise_steps`: Number of denoising steps of each inference pass. For the original (DDIM) version, it's recommended to use 20-50 steps.\n  - `--is_lineart`: If the user provides an image and the task is to color the line art within that image, this parameter is not needed. However, if the input is already a line art and no additional extraction is necessary, then this parameter should be included.\n  - `--guidance_scale_ref`: Increasing makes the model more inclined to accept the guidance of the reference image.\n  - `--guidance_scale_point`: Increasing makes the model more inclined to input point guidance to achieve more customized colorization.\n  - `--point_ref_paths` and `--point_lineart_paths` (**optional**): Two 512x512 matrices are used to represent the matching points between the corresponding reference and line art with continuously increasing integers. That is, the coordinates of the matching points in both matrices will have the same values: 1, 2, 3, etc., while the values in other positions will be 0 (you can refer to the provided samples). Of course, we recommend using Gradio for point guidance.\n\n## 🌱 Gradio\nFirst, modify `./configs/inference.yaml` to set the path of model weight. Afterwards, run the script:\n```bash\npython run_gradio.py\n```\nThe gradio demo would look like the UI shown below. \n\u003ctable align=\"center\"\u003e\n  \u003ctr\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"docs/gradio1.png\" width=\"300\" height=\"400\"\u003e\n    \u003c/td\u003e\n    \u003ctd\u003e\n      \u003cimg src=\"docs/gradio2.png\" width=\"300\" height=\"400\"\u003e\n    \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\nA biref tutorial:\n\n1. Upload the reference image and target image. \n\n    Note that for the target image, there are two modes: you can upload an RGB image, and the model will automatically extract the line art; or you can directly upload the line art by checking the 'input is lineart' option. \n\n    The line art images are single-channel grayscale images, where the input consists of floating-point values with the background set to 0 and the line art close to 1. Additionally, we would like to further communicate with our users: if the line art you commonly use is binarized, please let us know. We will fine-tune the model and release an updated version to better suit your needs. 😆\n\n2. Click 'Process Images' to resize the images to 512*512 resolution.\n3. (Optional) **Starting from the reference image**, **alternately** click on the reference and target images in sequence to define matching points. Use 'Undo' to revert the last action.\n4. Click 'Generate' to produce the result.\n## 🌺 Acknowledgements\nThis project is developped on the codebase of [MagicAnimate](https://github.com/magic-research/magic-animate). We appreciate this great work! \n\n## 🎓 Citation\n\nPlease cite our paper:\n\n```bibtex\n@article{liu2024manganinja,\n  author  = {Zhiheng Liu and Ka Leong Cheng and Xi Chen and Jie Xiao and Hao Ouyang and Kai Zhu and Yu Liu and Yujun Shen\n             and Qifeng Chen and Ping Luo},\n  title   = {MangaNinja: Line Art Colorization with Precise Reference Following},\n  journal = {CoRR},\n  volume  = {abs/xxxx.xxxxx},\n  year    = {2024}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fali-vilab%2FMangaNinjia","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fali-vilab%2FMangaNinjia","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fali-vilab%2FMangaNinjia/lists"}