https://github.com/yhZhai/idol

[ECCV 2024] IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation
https://github.com/yhZhai/idol

aigc depth-generation human-generation multi-modal-generation

Last synced: about 2 months ago
JSON representation

[ECCV 2024] IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation

• Host: GitHub
• URL: https://github.com/yhZhai/idol
• Owner: yhZhai
• License: apache-2.0
• Created: 2024-07-02T15:02:02.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2024-07-16T02:43:01.000Z (5 months ago)
• Last Synced: 2024-07-22T09:33:51.234Z (5 months ago)
• Topics: aigc, depth-generation, human-generation, multi-modal-generation
• Homepage: https://yhzhai.github.io/idol/
• Size: 31.8 MB
• Stars: 24
• Watchers: 4
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-diffusion-categorized - [Code

README

        
# [💃 IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation](https://yhzhai.github.io/idol/)

 

[Yuanhao Zhai](https://www.yhzhai.com/)1, [Kevin Lin](https://sites.google.com/site/kevinlin311tw/)2, [Linjie Li](https://scholar.google.com/citations?hl=en&user=WR875gYAAAAJ)2, [Chung-Ching Lin](https://scholar.google.com/citations?hl=en&user=legkbM0AAAAJ)2, [Jianfeng Wang](http://jianfengwang.me)2, [Zhengyuan Yang](https://zyang-ur.github.io)2, [David Doermann](https://cse.buffalo.edu/~doermann/)1, [Junsong Yuan](https://cse.buffalo.edu/~jsyuan/)1, [Zicheng Liu](https://scholar.google.com/citations?hl=en&user=bkALdvsAAAAJ)3, [Lijuan Wang](https://scholar.google.com/citations?hl=en&user=cDcWXuIAAAAJ)2

1State University of New Yort at Buffalo    |    2Microsoft  |    3Advanced Micro Devices

**European Conference on Computer Vision (ECCV) 2024**

 

**TL;DR**: Our IDOL enables human-centric joint video-depth generation, which could be rendered into realistic 2.5 videos.

![](static/images/teaser.png)

**All code and checkpoints will be released soon!**