https://github.com/garystafford/svd-examples

Created with Stability AIʼs Stable Video Diffusion XT 1.1 Image-to-Video latent diffusion model (SVD XT 1.1)
https://github.com/garystafford/svd-examples

amazon-web-services aws gen-ai generative-ai stability-ai stable-diffusion stable-video-diffusion video

Last synced: about 2 months ago
JSON representation

Created with Stability AIʼs Stable Video Diffusion XT 1.1 Image-to-Video latent diffusion model (SVD XT 1.1)

• Host: GitHub
• URL: https://github.com/garystafford/svd-examples
• Owner: garystafford
• Created: 2024-04-04T00:04:28.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-04-12T13:47:18.000Z (about 2 years ago)
• Last Synced: 2024-04-12T21:08:52.190Z (about 2 years ago)
• Topics: amazon-web-services, aws, gen-ai, generative-ai, stability-ai, stable-diffusion, stable-video-diffusion, video
• Homepage:
• Size: 81.9 MB
• Stars: 1
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Stability AI Stable Video Diffusion XT 1.1 Examples

All videos I created here were with [Stability AI](https://stability.ai/stable-video)ʼs Stable Video Diffusion XT 1.1 Image-to-Video latent diffusion model (SVD XT 1.1), available on [Hugging Face](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1). Videos were created on Amazon Web Services (AWS) from single images using the SVD XT 1.1 model and [ComfyUI](https://github.com/comfyanonymous/ComfyUI) on a G4dn instance, powered by NVIDIA T4 GPUs. The videos were rendered as MP4 files with recommended 25 frames at an average of 9 fps and a resolution of 1,024 x 576 pixels. The videos were also rendered as [WebP](https://developers.google.com/speed/webp) format files (or in some cases, the MP4 files were then converted to WebP) for display in GitHub, shown below.

One of the best instructional videos I've seen on the subject of what is possible with SVD, is [ComfyUI: Stable Video Diffusion (Workflow Tutorial)](https://www.youtube.com/watch?v=m-ZoxcYNWFg&list=LL&index=7), by ControlAltAI, on YouTube.

## Gated Model and Output

Note that SVD XT 1.1 is a 'Gated model', but, according to the Stable Video Diffusion 1.1 License Agreement, _"Derivative Work(s)" means (a) any derivative work of the Software Products as recognized by U.S. copyright laws and (b) any modifications to a Model, and any other model created which is based on or derived from the Model or the Model’s output._ __For clarity, Derivative Works do not include the output of any Model.__ <<<

## ComfyUI Workflow

The [ComfyUI](workflows/svd_xt_workflow.json) workflow is included in this repository. It creates both an MP4 and a WebP video file. This [workflow](https://comfyanonymous.github.io/ComfyUI_examples/video/) is based on the workflow referenced in the comments for the YouTube video, [Image2Video. Stable Video Diffusion Tutorial.](https://youtu.be/HOVYu2UbgEE?si=N65J15eWrnRh2jPj), by [Sebastian Kamph](https://www.youtube.com/@sebastiankamph).

![ComfyUI Workflow](comfy_ui_workflow.png)

## Wide-format Videos

   

      

      Source image

      

      Source image

   

   

      

      Source image

      

      Source image

   

   

      

      Source image

      

      Source image

   

## Tall-format Videos

   

      

      Source image

      

      Source image

   

   

      

      Source image

      

      Source image

   

   

      

      Source image

   

---

_The contents of this repository represent my viewpoints and not of my past or current employers, including Amazon Web Services (AWS). All third-party libraries, modules, plugins, and SDKs are the property of their respective owners._