https://github.com/lukasuz/MotionDreamer

[3DV 2025] MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation
https://github.com/lukasuz/MotionDreamer

3dv2025 4d-generation diffusion-features diffusion-models mesh-animation video-diffusion video-diffusion-model zero-shot

Last synced: 3 months ago
JSON representation

[3DV 2025] MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation

• Host: GitHub
• URL: https://github.com/lukasuz/MotionDreamer
• Owner: lukasuz
• Created: 2024-04-26T19:22:45.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-11-27T20:07:54.000Z (7 months ago)
• Last Synced: 2024-11-27T20:35:12.819Z (7 months ago)
• Topics: 3dv2025, 4d-generation, diffusion-features, diffusion-models, mesh-animation, video-diffusion, video-diffusion-model, zero-shot
• Homepage: https://lukas.uzolas.com/MotionDreamer/
• Size: 4.92 MB
• Stars: 20
• Watchers: 5
• Forks: 0
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-diffusion-categorized - [Code

README

        
## MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation

![Header animation](./static/imgs/header.gif)

### 3DV 2025

#### [🌐 Project Page](https://lukas.uzolas.com/MotionDreamer/) | [📝 Paper](https://arxiv.org/abs/2405.20155) 

**[Lukas Uzolas](https://lukas.uzolas.com/), 

[Elmar Eisemann](https://graphics.tudelft.nl/~eisemann/),

[Petr Kellnhofer](https://kellnhofer.xyz/)**




[Delft University of Technology](https://graphics.tudelft.nl/)




Animation techniques bring digital 3D worlds and characters to life. However, manual animation is tedious and automated techniques are often specialized to narrow shape classes. In our work, we propose a technique for automatic re-animation of arbitrary 3D shapes based on a motion prior extracted from a video diffusion model. Unlike existing 4D generation methods, we focus solely on the motion, and we leverage an explicit mesh-based representation compatible with existing computer-graphics pipelines. Furthermore, our utilization of diffusion features enhances accuracy of our motion fitting. We analyze efficacy of these features for animation fitting and we experimentally validate our approach for two different diffusion models and four animation models. Finally, we demonstrate that our time-efficient zero-shot method achieves a superior performance re-animating a diverse set of 3D shapes when compared to existing techniques in a user study.

_Code coming soon._

### Citation

```

@misc{uzolas2024motiondreamerexploringsemanticvideo,

      title={MotionDreamer: Exploring Semantic Video Diffusion features for Zero-Shot 3D Mesh Animation}, 

      author={Lukas Uzolas and Elmar Eisemann and Petr Kellnhofer},

      year={2024},

      eprint={2405.20155},

      archivePrefix={arXiv},

      primaryClass={cs.CV},

      url={https://arxiv.org/abs/2405.20155}, 

}

```