https://github.com/lucidrains/flexible-diffusion-modeling-videos-pytorch

Implementation of the video diffusion model and training scheme presented in the paper, Flexible Diffusion Modeling of Long Videos, in Pytorch
https://github.com/lucidrains/flexible-diffusion-modeling-videos-pytorch

artificial-intelligence deep-learning denoising-diffusion video-generation

Last synced: about 11 hours ago
JSON representation

Implementation of the video diffusion model and training scheme presented in the paper, Flexible Diffusion Modeling of Long Videos, in Pytorch

• Host: GitHub
• URL: https://github.com/lucidrains/flexible-diffusion-modeling-videos-pytorch
• Owner: lucidrains
• License: mit
• Created: 2022-05-28T00:45:51.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2022-05-28T00:54:19.000Z (over 2 years ago)
• Last Synced: 2024-05-02T01:14:21.341Z (5 months ago)
• Topics: artificial-intelligence, deep-learning, denoising-diffusion, video-generation
• Homepage:
• Size: 294 KB
• Stars: 81
• Watchers: 7
• Forks: 2
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        


## Flexible Diffusion Modeling of Long Videos - Pytorch (wip)

Implementation of the video diffusion model and training scheme presented in the paper, Flexible Diffusion Modeling of Long Videos, in Pytorch. While the Unet architecture does not look that novel (quite similar to Space-time factored unets, where they do attention across time) they achieved up to 25 minutes of coherent video with their specific frame sampling conditioning scheme during training.

I will also attempt to push this approach even further by introducing a super-resoluting module on top identical to what was used in Imagen

## Citations

```bibtex

@inproceedings{Harvey2022FlexibleDM,

    title   = {Flexible Diffusion Modeling of Long Videos},

    author  = {William Harvey and Saeid Naderiparizi and Vaden Masrani and Christian Weilbach and Frank Wood},

    year    = {2022}

}

```