https://github.com/huchenlei/ComfyUI-IC-Light-Native
ComfyUI native implementation of IC-Light
https://github.com/huchenlei/ComfyUI-IC-Light-Native
Last synced: 7 days ago
JSON representation
ComfyUI native implementation of IC-Light
- Host: GitHub
- URL: https://github.com/huchenlei/ComfyUI-IC-Light-Native
- Owner: huchenlei
- License: apache-2.0
- Created: 2024-05-08T13:55:49.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-02-25T16:35:36.000Z (about 2 months ago)
- Last Synced: 2025-04-07T22:04:39.002Z (10 days ago)
- Language: Python
- Homepage:
- Size: 51.8 KB
- Stars: 604
- Watchers: 4
- Forks: 33
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-comfyui - **ComfyUI-IC-Light-Native** - Light](https://github.com/lllyasviel/IC-Light). (All Workflows Sorted by GitHub Stars)
README
# ComfyUI-IC-Light-Native
ComfyUI native implementation of [IC-Light](https://github.com/lllyasviel/IC-Light).
## Install
Download the repository and unpack into the custom_nodes folder in the ComfyUI installation directory.
Or clone via GIT, starting from ComfyUI installation directory:
```bash
cd custom_nodes
git clone [email protected]:huchenlei/ComfyUI-IC-Light-Native.git
```
### Download models
IC-Light main repo is based on diffusers. In order to load it with UnetLoader in ComfyUI, state_dict keys need to convert to ldm format. You can download models with ldm keys here: https://huggingface.co/huchenlei/IC-Light-ldm/tree/main
There are 2 models:
- iclight_sd15_fc_unet_ldm: Use this in FG workflows
- iclight_sd15_fbc_unet_ldm: Use this in BG workflows
After you download these models, please put them under `ComfyUI/models/unet` and load them with `UNETLoader` node.
### Recommended nodes
- [ComfyUI-KJNodes](https://github.com/kijai/ComfyUI-KJNodes): Provides various mask nodes to create light map.
- [ComfyUI-Easy-Use](https://github.com/yolain/ComfyUI-Easy-Use): A giant node pack of everything. The remove bg node used in workflow comes from this pack.
- [ComfyUI_essentials](https://github.com/cubiq/ComfyUI_essentials): Many useful tooling nodes. Image resize node used in the workflow comes from this pack.
- [ComfyUI-IC-Light](https://github.com/kijai/ComfyUI-IC-Light): The IC-Light impl from kijai. It includes a very useful `DetailTransfer` node to help preverse high frequency details from input fg image.
## Workflows
Please make sure the fg image's masked/transparent area are grey before you pass it to the VAE. Otherwise, you will get background obscured in FC workflows or
darkened background in FBC workflows. You can use `IC Light Apply Mask Grey` to make sure the masked area's color is correct. See following examples:
### [Given FG, Generate BG and relight](https://github.com/huchenlei/ComfyUI-IC-Light/blob/main/examples/fg.json)
If you want to keep the original color of the fg object, you can put the fg object in the latent space to further guide the generation. [workflow](https://github.com/huchenlei/ComfyUI-IC-Light-Native/blob/main/examples/ic_light_preserve_color.json)
### [Given FG and light map, Genereate BG and relight](https://github.com/huchenlei/ComfyUI-IC-Light/blob/main/examples/fg_lightmap.json)
Light from right
Light from left
### [Given FG and BG, Put FG on BG and relight](https://github.com/huchenlei/ComfyUI-IC-Light/blob/main/examples/fg_bg_combine.json)
### [Recover high frequency detail (Text, etc) from original input image](https://github.com/huchenlei/ComfyUI-IC-Light-Native/blob/main/examples/ic_light_detail_transfer.json)
- Input image:
- Raw generation:
- After detail transfer:
## Common Issues
IC-Light's unet is accepting extra inputs on top of the common noise input. FG model accepts extra 1 input (4 channels). BG model accepts 2 extra input (8 channels).
The original unet's input is 4 channels as well.
If you see following error, it means you are using FG workflow but loaded the BG model.
```
RuntimeError: Given groups=1, weight of size [320, 8, 3, 3], expected input[2, 12, 64, 64] to have 8 channels, but got 12 channels instead
```
If you see following error, it means you are using FG workflow but loaded the BG model.
```
RuntimeError: Given groups=1, weight of size [320, 12, 3, 3], expected input[2, 8, 64, 64] to have 12 channels, but got 8 channels instead
```