https://github.com/yvann-ba/comfyui_yvann-nodes
Audio Reactivity Nodes for ComfyUI 🔊 Create AI generated audio-driven animations. Compatible with IPAdapter, ControlNets, AnimateDiff...
https://github.com/yvann-ba/comfyui_yvann-nodes
audioreactive comfyui generative-ai
Last synced: 14 days ago
JSON representation
Audio Reactivity Nodes for ComfyUI 🔊 Create AI generated audio-driven animations. Compatible with IPAdapter, ControlNets, AnimateDiff...
- Host: GitHub
- URL: https://github.com/yvann-ba/comfyui_yvann-nodes
- Owner: yvann-ba
- License: gpl-3.0
- Created: 2024-08-26T20:52:14.000Z (9 months ago)
- Default Branch: main
- Last Pushed: 2025-04-30T10:07:45.000Z (29 days ago)
- Last Synced: 2025-04-30T11:35:50.742Z (29 days ago)
- Topics: audioreactive, comfyui, generative-ai
- Language: Python
- Homepage: https://youtube.com/@yvann_ba
- Size: 41.8 MB
- Stars: 409
- Watchers: 4
- Forks: 13
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# 🔊 ComfyUI_Yvann-Nodes [](https://www.youtube.com/channel/yvann_ba)
#### Made with the great help of [Lilien](https://x.com/Lilien_RIG) 😎
### **A pack of custom nodes that enable audio reactivity within [ComfyUI](https://github.com/comfyanonymous/ComfyUI), allowing you to generate AI-driven animations that sync with music**
---
## What Does This Do?
- **Create** Audio Reactive AI videos, enable controls over AI generations styles, content and composition with any audio
- **Simple**: Just Drop one of our [Workflows](/example_workflows) in ComfyUI and specify your audio and visuals input
- **Flexible**: Works with existing ComfyUI AI tech and nodes (eg: IPAdapter, AnimateDiff, ControlNet, etc.)
---
## Video result (generated by our amazing users 😁)
#### Audio Reactive Images to Video ⬇️
#### Audio Reactive Video to Video ⬇️
## Quick Setup
- Install [ComfyUI](https://github.com/comfyanonymous/ComfyUI) and [ComfyUI-Manager](https://github.com/ltdrdata/ComfyUI-Manager)
### Pick a Workflow (Images → Video or Video → Video)
1. **Images → Video**
- Takes a **set of images** plus an **audio** track.
- *Watch Tutorial*:
[](https://www.youtube.com/watch?v=O2s6NseXlMc)
2. **Video → Video**
- Takes a **source video** plus an **audio** track.
- *Watch Tutorial*:
[](https://www.youtube.com/watch?v=BiQHWKP3q0c)
---
### Load Your Chosen Workflow in ComfyUI
1. **Download** the `.json` file for the workflow you picked:
- [AudioReactive_ImagesToVideo_Yvann.json](example_workflows/AudioReactive_ImagesToVideo_Yvann.json)
- [AudioReactive_VideoToVideo_Yvann.json](example_workflows/AudioReactive_VideoToVideo_Yvann.json)
2. **Drop** the `.json` file into the **ComfyUI window**.
3. **Open the "🧩 Manager"** → **"Install Missing Custom Nodes"**
- Install each pack of nodes that appears.
- **Restart** ComfyUI if prompted.
4. **Set Your Inputs & Generate**
- Provide the inputs needed (everything explained [here](https://www.youtube.com/@yvann_ba)
- Click **Queue** button to produce your **audio-reactive** animation!
**That's it!** Have fun playing with the differents settings now !!
(if you have any questions or problems, check my [Youtube Tutorials](https://www.youtube.com/@yvann_ba)
---
## Nodes Details
Click to Expand: Node-by-Node Reference
### Audio Analysis 🔍
Analyzes audio to generate reactive weights for each frame.
Node Parameters
- **audio_sep_model**: Model from "Load Audio Separation Model"
- **audio**: Input audio file
- **batch_size**: Frames to associate with audio weights
- **fps**: Frame rate for the analysis
**Parameters**:
- **analysis_mode**: e.g., Drums Only, Vocals, Full Audio
- **threshold**: Minimum weight pass-through
- **multiply**: Amplification factor
**Outputs**:
- **graph_audio** (image preview),
- **processed_audio**, **original_audio**,
- **audio_weights** (list of values).
---
### Load Audio Separation Model 🎧
Loads or downloads an audio separation model (e.g., HybridDemucs, OpenUnmix).
Node Parameters
- **model**: Choose between HybridDemucs / OpenUnmix.
- **Outputs**: **audio_sep_model** (connect to Audio Analysis or Remixer).
---
### Audio Peaks Detection 📈
Identifies peaks in the audio weights to trigger transitions or events.
Node Parameters
- **peaks_threshold**: Sensitivity.
- **min_peaks_distance**: Minimum gap in frames between peaks.
- **Outputs**: Binary peak list, alternate list, peak indices/count, graph.
---
### Audio IP Adapter Transitions 🔄
Manages transitions between images based on peaks. Great for stable or style transitions.
Node Parameters
- **images**: Batch of images.
- **peaks_weights**: From "Audio Peaks Detection".
- **blend_mode**, **transitions_length**, **min_IPA_weight**, etc.
---
### Audio Prompt Schedule 📝
Links text prompts to peak indices.
Node Parameters
- **peaks_index**: Indices from peaks detection.
- **prompts**: multiline string.
- **Outputs**: mapped schedule string.
---
### Audio Remixer 🎛️
Adjusts volume levels (drums, vocals, bass, others) in a track.
Node Parameters
- **drums_volume**, **vocals_volume**, **bass_volume**, **others_volume**
- **Outputs**: single merged audio track.
---
### Repeat Image To Count 🔁
Repeats a set of images N times.
Node Parameters
- **mask**: Mask input.
- **Outputs**: Repeated images.
---
### Invert Floats 🔄
Flips sign of float values.
Node Parameters
- **floats**: list of floats.
- **Outputs**: inverted list.
---
### Floats Visualizer 📈
Plots float values as a graph.
Node Parameters
- **floats** (and optional second/third).
- **Outputs**: visual graph image.
---
### Mask To Float 🎭
Converts a mask into a single float value.
Node Parameters
- **mask**: input.
- **Outputs**: float.
---
### Floats To Weights Strategy 🏋️
Transforms float lists into an IPAdapter "weight strategy."
Node Parameters
- **floats**: list of floats.
- **Outputs**: dictionary with strategy info.
---
Please give a ⭐ on GitHub it helps us enhance our Tool and it's Free !! (:
