https://github.com/thecooltechguy/ComfyUI-Stable-Video-Diffusion

ComfyUI nodes for Stable Video Diffusion
https://github.com/thecooltechguy/ComfyUI-Stable-Video-Diffusion

Last synced: 4 months ago
JSON representation

ComfyUI nodes for Stable Video Diffusion

• Host: GitHub
• URL: https://github.com/thecooltechguy/ComfyUI-Stable-Video-Diffusion
• Owner: thecooltechguy
• Created: 2023-11-23T06:31:03.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2023-11-24T06:14:27.000Z (about 2 years ago)
• Last Synced: 2024-05-14T00:11:02.474Z (over 1 year ago)
• Language: Python
• Size: 20.8 MB
• Stars: 269
• Watchers: 5
• Forks: 18
• Open Issues: 24
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-comfyui - **ComfyUI Stable Video Diffusion**
• awesome-comfyui - **ComfyUI Stable Video Diffusion**

README

          
# ComfyUI Stable Video Diffusion

Easily use Stable Video Diffusion inside ComfyUI!

[![](https://dcbadge.vercel.app/api/server/MfVCahkc2y)](https://discord.gg/MfVCahkc2y)

- [Installation](#1-installation)

- [Node types](#node-types)

- [Example workflows](#example-workflows)

    - [Image to video](#image-to-video)

    - [Image to video generation (high FPS w/ frame interpolation)](#image-to-video-generation-high-fps-w-frame-interpolation)

Need help? Join our Discord!

## 1. Installation

### Option 1: Install via ComfyUI Manager

Open ComfyUI Manager and install the **ComfyUI Stable Video Diffusion** (author: **thecooltechguy**) custom node.

### Option 2: Install manually

```

cd ComfyUI/custom_nodes/

git clone https://github.com/thecooltechguy/ComfyUI-Stable-Video-Diffusion

cd ComfyUI-Stable-Video-Diffusion/

python install.py

```

### 2. Download models into `ComfyUI/models/svd/`

 - svd.safetensors - [Download](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid/resolve/main/svd.safetensors?download=true)

 - svd_image_decoder.safetensors - [Download](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid/resolve/main/svd_image_decoder.safetensors?download=true)

 - svd_xt.safetensors - [Download](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/resolve/main/svd_xt.safetensors?download=true)

 - svd_xt_image_decoder.safetensors - [Download](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt/resolve/main/svd_xt_image_decoder.safetensors?download=true)

## Node types

- **SVDModelLoader**

    - Loads the Stable Video Diffusion model

- **SVDSampler**

    - Runs the sampling process for an input image, using the model, and outputs a latent

- **SVDDecoder**

    - Decodes the sampled latent into a series of image frames

- **SVDSimpleImg2Vid**

    - Combines the above 3 nodes above into a single node

## Example workflows

### Image to video

[https://comfyworkflows.com/workflows/5a4cd9fd-9685-4985-adb8-7be84e8636ad](https://comfyworkflows.com/workflows/5a4cd9fd-9685-4985-adb8-7be84e8636ad)

![workflow graph](./svd_workflow_graph.png)

![sample output](./svd_workflow.gif)

### Image to video generation (high FPS w/ frame interpolation)

[https://comfyworkflows.com/workflows/bf3b455d-ba13-4063-9ab7-ff1de0c9fa75](https://comfyworkflows.com/workflows/bf3b455d-ba13-4063-9ab7-ff1de0c9fa75)

![workflow graph](./svd_rife_workflow_graph.png)

![sample output](./svd_rife.gif)