https://github.com/AIWarper/ComfyUI-NormalCrafterWrapper
https://github.com/AIWarper/ComfyUI-NormalCrafterWrapper
Last synced: 6 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/AIWarper/ComfyUI-NormalCrafterWrapper
- Owner: AIWarper
- Created: 2025-05-09T18:15:00.000Z (7 months ago)
- Default Branch: main
- Last Pushed: 2025-05-09T18:37:26.000Z (7 months ago)
- Last Synced: 2025-05-09T19:30:32.874Z (7 months ago)
- Language: Python
- Size: 20.5 KB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-comfyui - **NormalCrafterWrapper**
- awesome-comfyui - **NormalCrafterWrapper**
README
# ComfyUI NormalCrafter
This is a ComfyUI custom node implementation for [NormalCrafter: Learning Temporally Consistent Normals from Video Diffusion Priors](https://github.com/Binyr/NormalCrafter) by Yanrui Bin, Wenbo Hu, Haoyuan Wang, Xinya Chen, and Bing Wang.
It allows you to generate temporally consistent normal map sequences from input video frames.
## Installation
1. Clone this repository into your `ComfyUI/custom_nodes/` directory:
```bash
cd ComfyUI/custom_nodes/
git clone https://github.com/AIWarper/ComfyUI-NormalCrafterWrapper.git
```
2. Install the required dependencies. Navigate to the `ComfyUI-NormalCrafter` directory and install using the `requirements.txt`:
```bash
ACTIVATE YOUR VENV FIRST
ComfyUI-NormalCrafterWrapper
pip install -r requirements.txt
```
(See the `requirements.txt` for notes on the `diffusers` dependency, which ComfyUI often manages.)
3. Restart ComfyUI.
## Node: NormalCrafter (Process Video)
This node takes a sequence of images (video frames) and processes them to output a corresponding sequence of normal map images.
### Parameters
* **`images` (Input Socket)**: The input image sequence (video frames).
* **`pipe_override` (Input Socket, Optional)**: Allows providing a pre-loaded NormalCrafter pipeline instance. If unconnected, the node loads its own.
* **`seed`**: (Integer, Default: 42) Controls the randomness for reproducible results.
* **`control_after_generate`**: (Fixed, Increment, Decrement, Randomize) Standard ComfyUI widget for seed behavior on subsequent runs. Note: The underlying pipeline uses the seed for each full video processing.
* **`max_res_dimension`**: (Integer, Default: 1024) The maximum dimension (height or width) to which input frames are resized while maintaining aspect ratio.
* **`window_size`**: (Integer, Default: 14) The number of consecutive frames processed together in a sliding window. Affects temporal consistency.
* **`time_step_size`**: (Integer, Default: 10) How many frames the sliding window moves forward after processing a chunk. If less than `window_size`, frames will overlap, potentially improving smoothness.
* **`decode_chunk_size`**: (Integer, Default: 4) Number of latent frames decoded by the VAE at once. Primarily a VRAM management setting.
* **`fps_for_time_ids`**: (Integer, Default: 7) Conditions the model on an intended Frames Per Second, influencing motion characteristics in the generated normals. *Note: In testing, this parameter showed minimal to no visible effect on the output for this specific model and task. As such I hard coded the value*
* **`motion_bucket_id`**: (Integer, Default: 127) Conditions the model on an expected amount of motion. *Note: In testing, this parameter showed minimal to no visible effect on the output for this specific model and task. As such I hard coded the value*
* **`noise_aug_strength`**: (Float, Default: 0.0) Strength of noise augmentation applied to conditioning information. *Note: In testing, this parameter showed minimal to no visible effect on the output for this specific model and task. As such I hard coded the value*
### Troubleshooting Flicker / Improving Temporal Consistency
If you are experiencing flickering or temporal inconsistencies in your output:
* **Increase `window_size`**: A larger window allows the model to see more temporal context, which can significantly improve consistency between frames.
* **Adjust `time_step_size`**: Using a `time_step_size` smaller than `window_size` creates an overlap between processed windows. This overlap is merged, which can smooth transitions. For example, if `window_size` is 20, try a `time_step_size` of 10 or 15.
You may be able to increase `window_size` and `time_step_size` substantially (e.g., to their maximum values) without encountering Out Of Memory (OOM) issues, depending on your hardware. Experiment to find the best balance for your needs.
### Dependencies
* `mediapy`
* `decord`
* `diffusers` (and its dependencies like `transformers`, `huggingface_hub`) - ComfyUI usually manages its own `diffusers` version. Install manually if you encounter specific import errors related to it.
* `torch`, `numpy`, `Pillow` (standard Python ML/Image libraries)
Refer to `requirements.txt` for more details.