https://github.com/ZhexinLiang/Control-Color

Control Color: Multimodal Diffusion-based Interactive Image Colorization
https://github.com/ZhexinLiang/Control-Color

Last synced: about 2 months ago
JSON representation

Control Color: Multimodal Diffusion-based Interactive Image Colorization

• Host: GitHub
• URL: https://github.com/ZhexinLiang/Control-Color
• Owner: ZhexinLiang
• License: other
• Created: 2023-07-07T02:39:47.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2025-02-23T07:08:25.000Z (3 months ago)
• Last Synced: 2025-02-23T08:19:12.929Z (3 months ago)
• Language: Python
• Homepage: https://zhexinliang.github.io/Control_Color/
• Size: 89.2 MB
• Stars: 139
• Watchers: 21
• Forks: 3
• Open Issues: 10
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-diffusion-categorized - [Code

README

        



Control Color: Multimodal Diffusion-based Interactive Image Colorization




    Zhexin Liang 

    Zhaochen Li 

    Shangchen Zhou 

    Chongyi Li 

    Chen Change Loy





    S-Lab, Nanyang Technological University  





    


        

        

        

         

        

         

        

        

        

        

    





Control Color (CtrlColor) achieves highly controllable multimodal image colorization based on stable diffusion model.



    

    

    Region colorization

    Iterative editing

    

:open_book: For more visual results and applications of CtrlColor, go checkout our project page.

---

## :mega: Updates

- **2024.12.16**: The test codes (gradio demo), colorization model checkpoint, and autoencoder checkpoint are now publicly available.

## :desktop_computer: Requirements

- required packages in `CtrlColor_environ.yaml`

```

# git clone this repository

git clone https://github.com/ZhexinLiang/Control-Color.git

cd Control_Color

# create new anaconda env and install python dependencies

conda env create -f CtrlColor_environ.yaml

conda activate CtrlColor

```

## :running_woman: Inference

### Prepare models:

Please download the checkpoints of both colorization model and vae from [[Google Drive](https://drive.google.com/drive/folders/1lgqstNwrMCzymowRsbGM-4hk0-7L-eOT?usp=sharing)] and put both checkpoints in `./pretrained_models` folder.

### Testing:

You can use the following cmd to run gradio demo:

```

python test.py

```

Then you will get our interactive interface as below:



## :love_you_gesture: Citation

If you find our work useful for your research, please consider citing the paper:

```

@article{liang2024control,

  title={Control Color: Multimodal Diffusion-based Interactive Image Colorization},

  author={Liang, Zhexin and Li, Zhaochen and Zhou, Shangchen and Li, Chongyi and Loy, Chen Change},

  journal={arXiv preprint arXiv:2402.10855},

  year={2024}

}

```

### Contact

If you have any questions, please feel free to reach out at `[email protected]`.