https://yhzhai.github.io/idol/

[ECCV 2024] IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation
https://yhzhai.github.io/idol/

aigc depth-generation human-generation multi-modal-generation

Last synced: about 2 months ago
JSON representation

[ECCV 2024] IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation

• Host: GitHub
• URL: https://yhzhai.github.io/idol/
• Owner: yhZhai
• License: apache-2.0
• Created: 2024-07-02T15:02:02.000Z (6 months ago)
• Default Branch: main
• Last Pushed: 2024-09-16T18:24:40.000Z (3 months ago)
• Last Synced: 2024-09-17T00:06:24.119Z (3 months ago)
• Topics: aigc, depth-generation, human-generation, multi-modal-generation
• Homepage: https://yhzhai.github.io/idol/
• Size: 31.8 MB
• Stars: 44
• Watchers: 12
• Forks: 1
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-diffusion-categorized - [Project
• Awesome-Human-Video-Generation - IDOL

README

        
# [💃 IDOL: Unified Dual-Modal Latent Diffusion for Human-Centric Joint Video-Depth Generation](https://yhzhai.github.io/idol/)

 

[Yuanhao Zhai](https://www.yhzhai.com/)1, [Kevin Lin](https://sites.google.com/site/kevinlin311tw/)2, [Linjie Li](https://scholar.google.com/citations?hl=en&user=WR875gYAAAAJ)2, [Chung-Ching Lin](https://scholar.google.com/citations?hl=en&user=legkbM0AAAAJ)2, [Jianfeng Wang](http://jianfengwang.me)2, [Zhengyuan Yang](https://zyang-ur.github.io)2, [David Doermann](https://cse.buffalo.edu/~doermann/)1, [Junsong Yuan](https://cse.buffalo.edu/~jsyuan/)1, [Zicheng Liu](https://scholar.google.com/citations?hl=en&user=bkALdvsAAAAJ)3, [Lijuan Wang](https://scholar.google.com/citations?hl=en&user=cDcWXuIAAAAJ)2

1State University of New Yort at Buffalo    |    2Microsoft  |    3Advanced Micro Devices

**European Conference on Computer Vision (ECCV) 2024**

 

**TL;DR**: Our IDOL enables human-centric joint video-depth generation, which could be rendered into realistic 2.5 videos.

![](static/images/teaser.png)

**All code and checkpoints will be released soon!**