Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/cwchenwang/awesome-4d-generation

List of papers on 4D Generation.
https://github.com/cwchenwang/awesome-4d-generation

List: awesome-4d-generation

Last synced: 3 months ago
JSON representation

List of papers on 4D Generation.

Host: GitHub
URL: https://github.com/cwchenwang/awesome-4d-generation
Owner: cwchenwang
License: mit
Created: 2024-04-23T18:17:56.000Z (8 months ago)
Default Branch: main
Last Pushed: 2024-08-25T15:06:23.000Z (4 months ago)
Last Synced: 2024-08-25T16:26:05.291Z (4 months ago)
Homepage:
Size: 17.6 KB
Stars: 136
Watchers: 8
Forks: 1
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

Awesome-Text2X-Resources - Awesome 4D Generation
ultimate-awesome - awesome-4d-generation - List of papers on 4D Generation. (Other Lists / PowerShell Lists)

README

        # Awesome 4D Generation

This repo collects papers for 4D generation.

## Table of Contents

- [Camera Control for Video Diffusion](#camera-control-for-video-diffusion)

- [Multi-view for Video Diffusion](#multi-view-for-video-diffusion)

- [Distillation from Video Diffusion](#distillation-from-video-diffusion)

- [Generation by Reconstruction](#generation-by-reconstruction)

- [4D Editing](#4d-editing)

- [Physics](#physics)

## Camera Control for Video Diffusion

VD3D: Taming Large Video Diffusion Transformers for 3D Camera Control

[📄 Paper](https://arxiv.org/abs/2407.12781) | [🌐 Project Page](https://snap-research.github.io/vd3d/)

Controlling Space and Time with Diffusion Models

[📄 Paper](https://arxiv.org/pdf/2407.07860) | [🌐 Project Page](https://4d-diffusion.github.io/)

CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation

[📄 Paper](https://arxiv.org/abs/2406.02509) | [🌐 Project Page](https://ir1d.github.io/CamCo/)

Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control

[📄 Paper](https://arxiv.org/pdf/2405.17414) | [🌐 Project Page](https://collaborativevideodiffusion.github.io/)

## Multi-view for Video Diffusion

SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency, Xie et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2407.17470) | [🌐 Project Page](https://sv4d.github.io/) | [💻 Code](https://github.com/Stability-AI/generative-models)

L4GM: Large 4D Gaussian Reconstruction Model, Ren et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2406.10324) | [🌐 Project Page](https://research.nvidia.com/labs/toronto-ai/l4gm/)

4Diffusion: Multi-view Video Diffusion Model for 4D Generation, Zhang et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2405.20674) | [🌐 Project Page](https://aejion.github.io/4diffusion) | [💻 Code](https://github.com/aejion/4Diffusion) 

Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models, Liang et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2405.16645) | [🌐 Project Page](https://vita-group.github.io/Diffusion4D/) | [💻 Code](https://github.com/VITA-Group/Diffusion4D) | [🎥 Video](https://www.youtube.com/watch?v=XJT-cMt_xVo)

## Distillation from Video Diffusion

Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion, Deng et al., SIGGRAPH 2024

[📄 Paper](https://arxiv.org/pdf/2407.13759) | [🌐 Project Page](https://boyangdeng.com/streetscapes/) | [🎥 Video](https://www.youtube.com/watch?v=13hLTnrVVKk)

4Dynamic: Text-to-4D Generation with Hybrid Priors, Yuan et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2407.12684)

STAR: Skeleton-aware Text-based 4D Avatar Generation with In-Network Motion Retargeting, Chai et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2406.04629) | [🌐 Project Page](https://star-avatar.github.io/) | [💻 Code](https://github.com/czh-98/STAR)

MotionDreamer: Zero-Shot 3D Mesh Animation from Video Diffusion Models, Uzolas et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2405.20155) | [💻 Code](https://github.com/lukasuz/MotionDreamer)

PLA4D: Pixel-Level Alignments for Text-to-4D Gaussian Splatting, Miao et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2405.19957) | [🌐 Project Page](https://github.com/MiaoQiaowei/PLA4D.github.io)

MagicPose4D: Crafting Articulated Models with Appearance and Motion Control, Zhang et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2405.14017) | [🌐 Project Page](https://boese0601.github.io/magicpose4d/) | [💻 Code](https://github.com/haoz19/MagicPose4D) 

SC4D: Sparse-Controlled Video-to-4D Generation and Motion Transfer, Wu et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2404.03736) | [🌐 Project Page](https://sc4d.github.io/) | [💻 Code](https://github.com/JarrentWu1031/SC4D) | [🎥 Video](https://www.youtube.com/watch?v=SkpTEuX4B5c)

TC4D: Trajectory-Conditioned Text-to-4D Generation, Bahmani et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2403.17920) | [🌐 Project Page](https://sherwinbahmani.github.io/tc4d) | [💻 Code](https://github.com/sherwinbahmani/tc4d)

Comp4D: LLM-Guided Compositional 4D Scene Generation, Xu et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2403.16993) | [🌐 Project Page](https://vita-group.github.io/Comp4D/) | [💻 Code](https://github.com/VITA-Group/Comp4D) | [🎥 Video](https://www.youtube.com/watch?v=9q8SV1Xf_Xw)

STAG4D: Spatial-Temporal Anchored Generative 4D Gaussians, Zetn et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2403.14939.pdf) | [🌐 Project Page](https://nju-3dv.github.io/projects/STAG4D/) | [💻 Code](https://github.com/zeng-yifei/STAG4D) 

GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation, Gao et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2403.12365) | [🌐 Project Page](https://zerg-overmind.github.io/GaussianFlow.github.io/) | [💻 Code](https://github.com/Zerg-Overmind/GaussianFlow)

4DGen: Grounded 4D Content Generation with Spatial-temporal Consistency, Yin et al., Arxiv 2023

[📄 Paper](https://arxiv.org/pdf/2312.17225) | [🌐 Project Page](https://vita-group.github.io/4DGen/) | [💻 Code](https://github.com/VITA-Group/4DGen) | [🎥 Video](https://www.youtube.com/watch?v=-bXyBKdpQ1o)

DreamGaussian4D: Generative 4D Gaussian Splatting, Ren et al., CVPR 2024

[📄 Paper](https://arxiv.org/pdf/2312.13763) | [🌐 Project Page](https://jiawei-ren.github.io/projects/dreamgaussian4d/)

Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models, Ling et al., Arxiv 2023

[📄 Paper](https://arxiv.org/pdf/2312.03795) | [🌐 Project Page](https://animatabledreamer.github.io/) | [💻 Code](https://github.com/AnimatableDreamer/AnimatableDreamer)

AnimatableDreamer: Text-Guided Non-rigid 3D Model Generation and Reconstruction with Canonical Score Distillation, Wang et al., Arxiv 2023

[📄 Paper](https://arxiv.org/pdf/2312.03795) | [🌐 Project Page](https://animatabledreamer.github.io/) | [💻 Code](https://github.com/AnimatableDreamer/AnimatableDreamer)

4D-fy: Text-to-4D Generation Using Hybrid Score Distillation Sampling, Bahmani et al., CVPR 2024

[📄 Paper](https://arxiv.org/pdf/2311.17984) | [🌐 Project Page](https://research.nvidia.com/labs/nxp/dream-in-4d/) | [💻 Code](https://github.com/sherwinbahmani/4dfy)

A Unified Approach for Text- and Image-guided 4D Scene Generation, Zheng et al., CVPR 2024

[📄 Paper](https://arxiv.org/pdf/2311.17984) | [🌐 Project Page](https://sherwinbahmani.github.io/4dfy) | [💻 Code](https://github.com/NVlabs/dream-in-4d)

Animate124: Animating One Image to 4D Dynamic Scene, Zhao et al., Arxiv 2023

[📄 Paper](https://arxiv.org/pdf/2311.14603) | [🌐 Project Page](https://animate124.github.io/) | [💻 Code](https://github.com/HeliosZhao/Animate124)

Consistent4D: Consistent 360° Dynamic Object Generation from Monocular Video, Jiang et al., Arxiv 2024

[📄 Paper](https://arxiv.org/pdf/2311.02848) | [🌐 Project Page](https://consistent4d.github.io/) | [💻 Code](https://github.com/yanqinJiang/Consistent4D)

Text-To-4D Dynamic Scene Generation, Singer et al., Arxiv 2023

[📄 Paper](https://arxiv.org/pdf/2301.11280) | [🌐 Project Page](https://make-a-video3d.github.io)

## Generation by Reconstruction

4Real: Towards Photorealistic 4D Scene Generation via Video Diffusion Models, Yu et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2406.07472) | [🌐 Project Page](https://snap-research.github.io/4Real/)

Vidu4D: Single Generated Video to High-Fidelity 4D Reconstruction with Dynamic Gaussian Surfels, Wang et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2405.16822) | [🌐 Project Page](https://vidu4d-dgs.github.io/)

## 4D Editing

Instruct 4D-to-4D: Editing 4D Scenes as Pseudo-3D Scenes Using 2D Diffusion, Mou et al., CVPR 2024

[📄 Paper](https://arxiv.org/abs/2406.09402) | [🌐 Project Page](https://immortalco.github.io/Instruct-4D-to-4D/)

## Physics

Sync4D: Video Guided Controllable Dynamics for Physics-Based 4D Generation, Fu et al., Arxiv 2024

[📄 Paper](https://arxiv.org/abs/2405.16849) | [🌐 Project Page](https://sync4dphys.github.io/)