https://github.com/garystafford/svdxt-sagemaker-huggingface

Deploy and invoke Stability AI's Stable Video Diffusion XT (SVT-XT) 1.1 foundation model on Amazon SageMaker.
https://github.com/garystafford/svdxt-sagemaker-huggingface

amazon-sagemaker aws generative-ai stability-ai stable-video-diffusion

Last synced: about 1 month ago
JSON representation

Deploy and invoke Stability AI's Stable Video Diffusion XT (SVT-XT) 1.1 foundation model on Amazon SageMaker.

• Host: GitHub
• URL: https://github.com/garystafford/svdxt-sagemaker-huggingface
• Owner: garystafford
• License: mit
• Created: 2024-04-20T03:33:08.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-05-18T12:07:26.000Z (8 months ago)
• Last Synced: 2024-05-18T13:24:45.109Z (8 months ago)
• Topics: amazon-sagemaker, aws, generative-ai, stability-ai, stable-video-diffusion
• Language: Jupyter Notebook
• Homepage:
• Size: 89.4 MB
• Stars: 4
• Watchers: 1
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Stable Video Diffusion XT 1.1 on Amazon SageMaker

Stability AI's [Stable Video Diffusion XT (SVT-XT) 1.1](https://stability.ai/stable-video) foundation model, available on [Hugging Face](https://huggingface.co/stabilityai/stable-video-diffusion-img2vid-xt-1-1), is a diffusion model that takes in a still image as a conditioning frame and generates a short 4 second video. The notebook walks through configuring, creating, and invoking an [Asynchronous Inference Endpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/async-inference.html) backed by the SVT-XT foundation model on Amazon SageMaker.

For more information on using this repository, read the associated blog post, [Generative AI Videos with Stability AI’s Stable Video Diffusion XT using Asynchronous Inference on Amazon SageMaker](https://garystafford.medium.com/generative-ai-videos-with-stability-ais-stable-video-diffusion-xt-using-asynchronous-inference-on-6810e0a7cab6).

## Notebook Versions

- **Public URL for Conditioning Image**: Version 1 of the Notebook, [svd_xt_demo_v1.ipynb](svd_xt_demo_v1.ipynb), passes a publicly accessible URL of the image in the request payload used to invoke the model. Use the corresponding custom inference script, [inference_v1/inference.py](inference_v1/inference.py).

- **Conditioning Image Passed in Request Payload**: Version 2 of the Notebook, [svd_xt_demo_v2.ipynb](svd_xt_demo_v2.ipynb), passes an encoded image in the request payload used to invoke the model. Use the corresponding custom inference script, [inference_v2/inference.py](inference_v2/inference.py).

## Architecture for Version 1: Public URL for Conditioning Image

![Architecture V1](architecture/async_inference_v1.png)

## Architecture for Version 2: Conditioning Image Passed in Request Payload

![Architecture V2](architecture/async_inference_v2.png)

## Streamlit Application

The project now includes an interactive Streamlit application to generate the videos. Make sure you have first installed the required Python packages, as outlined in the [Optional: Local Development Environment](https://github.com/garystafford/svdxt-sagemaker-huggingface?tab=readme-ov-file#optional-local-development-environment) section, below. The application can be started with the following command:

```sh

streamlit run app.py --server.runOnSave true

```

![Streamlit App](architecture/stremalit_app.png)

## Wide-format Videos

All videos created using Notebook included in this project.

   

      

      Source image

   

   

      

      Source image

   

   

      

      Source image

   

   

      

      Source image

   

   

      

      Source image

   

   

      

      Source image

   

   

      

      Source image generated with Stable Diffusion XL (SDXL) 1.0

   

## Tall-format Videos

All videos created using Notebook included in this project.

   

      

      Source image

      

      Source image

   

   

      

      Source image

      

      Source image

   

## Optional: Local Development Environment

Setup local environment to modify project.

```sh

python3 -m pip install virtualenv -Uq

virtualenv svd-venv

python3 -m venv svd-venv

source svd-venv/bin/activate

```

```sh

python3 -m pip install -r local_requirements.txt -Uq

```

## References

- 

- 

- 

- 

- 

- 

- 

- 

- 

- 

---

_The contents of this repository represent my viewpoints and not of my past or current employers, including Amazon Web Services (AWS). All third-party libraries, modules, plugins, and SDKs are the property of their respective owners._