https://github.com/huchenlei/ComfyUI_densediffusion

DenseDiffusion custom node for ComfyUI
https://github.com/huchenlei/ComfyUI_densediffusion

Last synced: 4 months ago
JSON representation

DenseDiffusion custom node for ComfyUI

• Host: GitHub
• URL: https://github.com/huchenlei/ComfyUI_densediffusion
• Owner: huchenlei
• License: apache-2.0
• Created: 2024-06-05T01:39:47.000Z (11 months ago)
• Default Branch: main
• Last Pushed: 2024-07-27T18:33:35.000Z (9 months ago)
• Last Synced: 2024-12-12T17:51:23.295Z (4 months ago)
• Language: Python
• Size: 25.4 KB
• Stars: 118
• Watchers: 1
• Forks: 8
• Open Issues: 8
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-comfyui - **ComfyUI DenseDiffusion** - ai/DenseDiffusion) custom node for ComfyUI. (All Workflows Sorted by GitHub Stars)

README

        
# ComfyUI_densediffusion

DenseDiffusion custom node for ComfyUI. Implements the [DenseDiffusion](https://github.com/naver-ai/DenseDiffusion)-like method for regional prompt used in [Omost](https://github.com/lllyasviel/Omost) project.

## What this repo implements

Normal attention calculation can be written as `y=softmax(q@k)@v`. DenseDiffusion introduces the method of attention manipulation on `q@k`, which makes the expression look like `y=softmax(modify(q@k))@v`.

The original DenseDiffusion's implementation does not perform very well according to my testing so here I only implemented the version used in Omost repo. Refer to https://github.com/lllyasviel/Omost#regional-prompter for other regional prompt methods.

## How to use

![image](https://github.com/huchenlei/ComfyUI_densediffusion/assets/20929282/d75c1354-8f62-4e84-9b9c-67698e2a5f32)

## Limitation [IMPORTANT]

Currently ComfyUI's attention replacements do not compose with each other, so this regional prompt method does not compose with IPAdapter. I am currently working on a universal model patcher to solve this issue.