https://github.com/aifsh/comfyui-diffsynth-studio

make DiffSynth-Studio avialbe in ComfyUI
https://github.com/aifsh/comfyui-diffsynth-studio

Last synced: 2 days ago
JSON representation

make DiffSynth-Studio avialbe in ComfyUI

• Host: GitHub
• URL: https://github.com/aifsh/comfyui-diffsynth-studio
• Owner: AIFSH
• License: apache-2.0
• Created: 2024-06-25T22:21:37.000Z (7 months ago)
• Default Branch: main
• Last Pushed: 2024-08-05T08:48:03.000Z (5 months ago)
• Last Synced: 2025-01-01T15:07:25.459Z (9 days ago)
• Language: Python
• Size: 3.79 MB
• Stars: 74
• Watchers: 2
• Forks: 8
• Open Issues: 12
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# ComfyUI-DiffSynth-Studio

make [DiffSynth-Studio](https://github.com/modelscope/DiffSynth-Studio) avialbe in ComfyUI



  

  

  



## how to use

test on py3.10，2080ti 11gb，torch==2.3.0+cu121

make sure `ffmpeg` is worked in your commandline

for Linux

```

apt update

apt install ffmpeg

```

for Windows,you can install `ffmpeg` by [WingetUI](https://github.com/marticliment/WingetUI) automatically

then!

```

# in ComfyUI/custom_nodes

git clone https://github.com/AIFSH/ComfyUI-DiffSynth-Studio.git

cd ComfyUI-DiffSynth-Studio

pip install -r requirements.txt

```

weights will be downloaded from huggingface or model scope

## Tutorial

- [Demo for Diffutoon](https://b23.tv/z7hEXlX)[DiffSynth-Studio!ComfyUI插件之Diffutoon节点-哔哩哔哩](https://b23.tv/z7hEXlX)

- [Run on 4090](https://www.xiangongyun.com/image/detail/13706bf7-f3e6-4e29-bb97-c79405f5def4)

- wechat：aifsh_98

## Nodes Detail and Workflow

### ExVideo Node

[ExVideo workflow](./workfolws/exvideo_workflow.json)

```

"image":("IMAGE",),

"svd_base_model":("SD_MODEL_PATH",),

"exvideo_model":("SD_MODEL_PATH",),

"num_frames":("INT",{ 

    "default": 128

}),

"fps":("INT",{

    "default": 30

}),

"num_inference_steps":("INT",{

    "default": 50

}),

"if_upscale":("BOOLEAN",{

    "default": True,

}),

"seed": ("INT",{

    "default": 1

})

```

### image synthesis

comming soon

### Diffutoon Node

[Diffutoon workflow](./workfolws/diffutoon_workflow.json)

```

"required":{

    "source_video_path": ("VIDEO",),

    "sd_model_path":("SD_MODEL_PATH",),

    "postive_prompt":("TEXT",),

    "negative_prompt":("TEXT",),

    "start":("INT",{

        "default": 0 ## from which second of your video to be shaded

    }),

    "length":("INT",{

        "default": -1 ## how long you want to shade, -1 name the whole video frames

    }),

    "seed":("INT",{

        "default": 42 

    }),

    "cfg_scale":("INT",{

        "default": 3

    }),

    "num_inference_steps":("INT",{

        "default": 10

    }),

    "animatediff_batch_size":("INT",{

        "default": 32 ## lower it till you can run

    }),

    "animatediff_stride":("INT",{

        "default": 16 ## lower it till you can run

    }),

    "vram_limit_level":("INT",{

        "default": 0 ## meet killed? try to 1

    }),

},

"optional":{

    "controlnet1":("ControlNetConfigUnit",),

    "controlnet2":("ControlNetConfigUnit",),

    "controlnet3":("ControlNetConfigUnit",),

}

```

### Video Stylization

### Chinese Models

## ask for answer as soon as you want

wechat: aifsh_98

need donate if you mand it,

but please feel free to new issue for answering

Windows环境配置太难？可以添加微信：aifsh_98，赞赏获取Windows一键包，当然你也可以提issue等待大佬为你答疑解惑。