World-Simulator
Simulating the Real World: Survey & Resources, which contains our survey "Simulating the Real World: A Unified Survey of Multimodal Generative Models" and Awesome-Text2X-Resources. Watch this repository for the latest updates! 🔥
https://github.com/ALEEEHU/World-Simulator
Last synced: 2 days ago
JSON representation
-
Text to Video
-
🎉 T2V Accepted Papers
-
💡 Video ArXiv Papers
- Link - videoai.github.io/tora_video/) |
- Link - - | [Link](https://motion-prompting.github.io/) |
- Link - AI-Research/StreamingT2V) | [Link](https://streamingt2v.github.io/) |
- Link - animator.html) |
- Link - animator) | [Link](https://laulampaul.github.io/text-animator.html) |
- Link - - | [Link](https://still-moving.github.io/) |
- Link - CS/CustomCrafter) | [Link](https://customcrafter.github.io/) |
- Link - o.github.io/) |
- Link
- Link - - | [Link](https://mvideo-v1.github.io/) |
- Link - shaonian/AnimateAnything) | [Link](https://yu-shaonian.github.io/Animate_Anything/) |
- Link - - | -- |
- Link - - |
- Link - Flow) | [Link](https://pyramid-flow.github.io/) |
- Link - X/GameGen-X) | [Link](https://gamegen-x.github.io/) |
- Link - - | -- |
- Link - - | [Link](https://github.com/hmrishavbandy/FlipSketch) |
- Link - story2video.github.io/) |
- Link - - | -- |
-
🎉 Video Accepted Papers
- Link - - | [Link](https://wangyanhui666.github.io/MicroCinema.github.io/) |
- Link - Page/) |
- Link - videosyn) | -- |
- Link
- Link - zhengcheng/vividzoo) | [Link](https://hi-zhengcheng.github.io/vividzoo/) |
- Link - hanlin/VideoDirectorGPT) | [Link](https://videodirectorgpt.github.io/) |
- Link - Ryan/DEMO) | [Link](https://pr-ryan.github.io/DEMO-project/) |
-
Year 2024
- arXiv - cvc.github.io/VideoGen-Eval/), [GitHub Repo](https://github.com/AILab-CVC/VideoGen-Eval)
-
💡 T2V ArXiv Papers
-
📚 Dataset Works
- Link - FUXI/VidGen) | [Link](https://sais-fuxi.github.io/projects/vidgen-1m/) |
-
-
Text to Human Motion
-
🎉 Motion Accepted Papers
- Link
- Link - - | [Link](https://lingomotions.com/) |
- Link - SEA/OmniMotionGPT) | [Link](https://zshyang.github.io/omgpt-website/) |
- Link - nik/motionfix) | [Link](https://motionfix.is.tue.mpg.de/) |
- Link - RAG/MoRAG) | [Link](https://motion-rag.github.io/) |
- Link - - |
- Link - motion/afford-motion) | [Link](https://afford-motion.github.io/) |
- Link
- Link - latent-diffusion) | [Link](https://chenxin.tech/mld/) |
- Link - page/) |
- Link - page/) |
- Link - motion-transfer/diffusion-motion-transfer) | [Link](https://diffusion-motion-transfer.github.io/) |
- Link
- Link - LYJ-Lab/InstructMotion) | -- |
- Link
- Link - gillman/self-correcting-self-consuming) | [Link](https://cs.brown.edu/people/ngillman//sc-sc.html) |
- Link - editing-release)| [Link](https://purvigoel.github.io/iterative-motion-editing/) |
- Link - Wenxun/MotionLCM) | [Link](https://dai-wenxun.github.io/MotionLCM-page/) |
- Link
- Link - vi/SMooDi) | [Link](https://neu-vi.github.io/SMooDi/) |
- Link - ZY-Dou/EMDM) | [Link](https://frank-zy-dou.github.io/projects/EMDM/index.html) |
- Link - Motion) | [Link](https://moonsliu.github.io/Pro-Motion/) |
- Link - - | [Link](https://research.nvidia.com/labs/toronto-ai/tesmo/) |
- Link - team/Stable-Text-to-motion-Framework) | [Link](https://sato-team.github.io/Stable-Text-to-Motion-Framework/) |
- Link - tlabs/stmc)| [Link](https://mathis.petrovich.fr/stmc/) |
- Link - motion-inbetweening) | [Link](https://setarehc.github.io/CondMDI/) |
- Link
- Link
-
💡 Motion ArXiv Papers
- Link - - | [Link](https://andypinxinliu.github.io/KinMo/) |
- Link - - | [Link](https://story2motion.github.io/) |
- Link
- Link - zhang/LMM) | [Link](https://mingyuan-zhang.github.io/projects/LMM.html) |
- Link - y1heng/StableMoFusion) | [Link](https://h-y1heng.github.io/StableMoFusion-page/) |
- Link
- Link - Motion) | [Link](https://shuochengzhai.github.io/Infinite-motion.github.io/) |
- Link
- Link
- Link - - | -- |
- Link - page/) |
- Link - Research/MotionCLR) | [Link](https://lhchen.top/MotionCLR/) |
- Link - zeyu-zhang/KMM) | [Link](https://steve-zeyu-zhang.github.io/KMM/) |
- Link - - | [Link](https://zkf1997.github.io/DART/) |
- Link
- Link - zeyu-zhang/MotionAnything) | [Link](https://steve-zeyu-zhang.github.io/MotionAnything/) |
- Link
- Link - - | [Link](https://marcb.pro/atu/) |
- Link - - | -- |
-
📚 Dataset Works
- TPAMI 2023
- Link - -|
- Link
- Link - -|
- arXiv 7 Apr 2025 - Human-Interaction-Motion-Generation)]
-
-
Update Logs
-
World Foundation Model Platform
-
-
Text to 4D
-
💡 4D ArXiv Papers
- Link - project.github.io/) |
- Link - - | [Link](https://tau-vailab.github.io/4-LEGS/) |
- Link
- Link - zvg/Efficient4D) | [Link](https://fudan-zvg.github.io/Efficient4D/) |
- Link - Overmind/GaussianFlow) | [Link](https://zerg-overmind.github.io/GaussianFlow.github.io/) |
- Link - Group/Comp4D) |[Link](https://vita-group.github.io/Comp4D/#) |
- Link - - |
- Link - - | [Link](https://github.com/MiaoQiaowei/PLA4D.github.io) |
- Link - 98/STAR) | [Link](https://star-avatar.github.io/) |
- Link - - | [Link](https://4k4dgen.github.io/index.html) |
- Link - of-motion/) | [Link](https://shape-of-motion.github.io/) |
- Link - - | -- |
- Link - - | -- |
- Link - x-d.github.io/) |
- Link
- Link - - | [Link](https://disco-4d.github.io/) |
- Link
- Link - - | [Link](https://snap-research.github.io/4Real-Video/) |
- Link - Verse/WideRange4D) | [Dataset Page](https://huggingface.co/datasets/Gen-Verse/WideRange4D) |
- Link - - | [Link](https://cat-4d.github.io/) |
- Link - - | [Link](https://hanxinzhu-lab.github.io/AR4D/) |
- Link - - | [Link](https://sv4d2.0.github.io/) |
- Link
- Link - 2-4D) | [Link](https://in-2-4d.github.io/) |
- Link
- Link - Embodied-AGI/TesserAct) | [Link](https://tesseractworld.github.io/) |
-
Year 2023
-
Other 4D Additional Info
- Link - AI/generative-models) | [Link](https://sv4d.github.io/) |
-
🎉 4D Accepted Papers
-
Year 2024
-
-
Text to 3D Human
-
💡 Human ArXiv Papers
-
🎉 Human Accepted Papers
- Link
- Link - - | [Link](https://syntec-research.github.io/MagicMirror/) |
- Link
- Link - craft.github.io/) |
- Link
- Link
- Link
- Link
- Link
- Link
- Link - ProjectPage/) |
- Link - - | [Link](https://www.nikoskolot.com/avatarpopup/) |
- Link - SMPL) | [Link](https://shanemankiw.github.io/SO-SMPL/) |
- Link - 1-to-A) | [Link](https://zhenglinzhou.github.io/Zero-1-to-A/) |
- Link - tang/GaussianIP) | [Link](https://silence-tang.github.io/gaussian-ip/) |
- Link
-
Year 2023-2024
-
-
Paradigms
-
4D Generation
- arXiv 2023 - a-video3d.github.io/)]
- CVPR 2024
- ICLR 2023 notable top 25% - diffusion-model)] [[Project Page](https://guytevet.github.io/mdm-page/)]
- NeurIPS 2023 - gpt.github.io/)]
- NeurIPS 2024
- NeurIPS 2024 - us/research/project/compositional-3d-aware-video-generation/)]
- ICML 2024 - Research/HumanTOMATO)] [[Project Page](https://lhchen.top/HumanTOMATO/)]
- NeurIPS 2024
- CVPR 2024 - in-4d)] [[Project Page](https://research.nvidia.com/labs/nxp/dream-in-4d/)]
- CVPR 2024 - ai/AlignYourGaussians/)]
- ECCV 2024
- NeurIPS 2024 - CVGL/DreamMesh4D)] [[Project Page](https://lizhiqi49.github.io/DreamMesh4D/)]
- NeurIPS 2024 - tlabs/L4GM-official)] [[Project Page](https://research.nvidia.com/labs/toronto-ai/l4gm/)]
- arXiv 2024
- CVPR 2024
- ECCV 2024 - yifei/STAG4D)] [[Project Page](https://nju-3dv.github.io/projects/STAG4D/)]
- ECCV 2024
- NeurIPS 2024 - research.github.io/4Real/)]
- NeurIPS 2024 - dgs.github.io/)]
- NeurIPS 2024 - Group/Diffusion4D)] [[Project Page](https://vita-group.github.io/Diffusion4D/)] [[Dataset](https://huggingface.co/datasets/hw-liang/Diffusion4D)]
- CVPR 2024 - M/Instruct-4D-to-4D/)] [[Project Page](https://immortalco.github.io/Instruct-4D-to-4D/)]
- SIGGRAPH 2020
- CVPR 2022 - to-motion)] [[Project Page](https://ericguo5513.github.io/text-to-motion/)]
- SCA 2023
- CVPR 2023 - zys/T2M-GPT)] [[Project Page](https://mael-zys.github.io/T2M-GPT/)]
- CVPR 2024 - codes)] [[Project Page](https://ericguo5513.github.io/momask/)]
- CVPR 2024 - ronghui/LODGE)] [[Project Page](https://li-ronghui.github.io/lodge)]
-
Video Generation
- ICLR 2025 Poster
- AIRC 2023
- ICLR 2018 Poster
- CVPR 2018
- CVPR 2022 - v)] [[Project Page](https://skor.sh/stylegan-v)]
- ICLR 2022 - yu/digan)] [[Project Page](https://sihyun.me/digan/)]
- ICCV 2023 - ntu.com/project/styleinv/index.html)]
- NeurIPS 2022 - diffusion.github.io/)]
- arXiv 2022
- arXiv 2022
- ICLR 2023 Poster - a-video.github.io/)]
- ICCV 2023 - 1)]
- ICCV 2023
- CVPR 2023 - ai/VideoLDM/)]
- IJCV 2024 - 1)] [[Project Page](https://showlab.github.io/Show-1/)]
- NeurIPS 2024 - vilab/videocomposer)] [[Project Page](https://videocomposer.github.io/)]
- ICLR 2024 Spotlight
- CVPR 2024
- ECCV 2024 - video.metademolab.com/)]
- SIGGRAPH Asia 2024 - video.github.io/)]
- ICLR 2024 Poster - 2023.github.io/)]
- ECCV 2024 - video-diffusion.github.io/)]
- CVPR 2024 - research.github.io/snapvideo/)]
- CVPR 2024
- ICLR 2025 Spotlight - VLLM/Lumina-T2X)]
- CVPR 2021 Oral - transformers)]
- CVPR 2023 Highlight - research/magvit)] [[Project Page](https://magvit.cs.cmu.edu/)]
- ICLR 2023 Poster
- ICML 2024
- ICLR 2024 Poster
- arXiv 2024 - Voken)]
- arXiv 2024
- ICLR 2025 Poster - T2I-AR-with-SJD/)]
- ICLR 2024 Poster - A-Video/Ground-A-Video)] [[Project Page](https://ground-a-video.github.io/)]
- ICCV 2023 - A-Video)] [[Project Page](https://tuneavideo.github.io/)]
- ICCV 2023
- CVPR 2024 - diffusion.github.io/)]
- CVPR 2024 - research/Video-P2P)] [[Project Page](https://video-p2p.github.io/)]
- CVPR 2024 Highlight - research/CoDeF)] [[Project Page](https://qiuyu96.github.io/CoDeF/)]
- NeurIPS 2024
- arXiv 2024
- TMLR 2024 - AI-Lab/AnyV2V)] [[Project Page](https://tiger-ai-lab.github.io/AnyV2V/)]
- arXiv 2024
- ICLR 2025 Poster - CameraCtrl/)]
- ICLR 2025 Poster - Zhiyu/NVS_Solver)]
- ICCV 2019
- ICCV 2019 - lab/impersonator)] [[Project Page](https://svip-lab.github.io/project/impersonator.html)] [[Dataset](https://svip-lab.github.io/dataset/iPER_dataset.html)]
- NeurIPS 2019 - order-model)] [[Project Page](https://aliaksandrsiarohin.github.io/first-order-model-website/)]
- ICCV 2023
- ICCV 2023 - Research/HumanSD)] [[Project Page](https://idea-research.github.io/HumanSD/)]
- CVPR 2023 - humans/editable-humans)] [[Project Page](https://custom-humans.github.io/)] [[Dataset](https://custom-humans.ait.ethz.ch/)]
- CVPR 2023 - anyone/)]
- CVPRW 2024 - eldesokey/latentman)] [[Project Page](https://abdo-eldesokey.github.io/latentman/)]
- IJCAI 2024
- arXiv 2024 - vilab/UniAnimate)] [[Project Page](https://unianimate.github.io/)]
- arXiv 2024
- arXiv 2021 - research/fitvid)] [[Project Page](https://sites.google.com/view/fitvidpaper)]
-
3D Generation
- NeurIPS 2023 - Research/DreamWaltz)] [[Project Page](https://idea-research.github.io/DreamWaltz/)]
- NeurIPS 2023 Spotlight - human.github.io/)]
- ECCV 2024 - UCLA/DreamScene360)] [[Project Page](https://dreamscene360.github.io/)]
- ICLR 2025
- arXiv 2022
- arXiv 2022 - e)]
- arXiv 2023 - e)]
- NeurIPS 2023
- ICCV 2023 - ai/ATT3D/)]
- ICLR 2023 Spotlight
- arXiv 2023
- CVPR 2023 - SDF)] [[Project Page](https://ttlmh.github.io/DiffusionSDF/)]
- ICML 2024
- ECCV 2024 - ai/LATTE3D/)]
- arXiv 2024 - research/AToM)] [[Project Page](https://snap-research.github.io/AToM/)]
- ICLR 2023 notable top 5%
- CVPR 2023 Highlight
- CVPR 2023
- ICCV 2023 - Lab-SCUT/Fantasia3D)] [[Project Page](https://fantasia3d.github.io/)]
- NeurIPS 2023 Spotlight - ml/prolificdreamer)] [[Project Page](https://ml.cs.tsinghua.edu.cn/prolificdreamer/)]
- ICLR 2024 Poster - dream.github.io/)]
- ICLR 2024 Oral
- CVPR 2024
- CVPR 2024 - cvpr24.github.io/)]
- CVPR 2024
- CVPR 2024
- CVPR 2024
- ICLR 2024 Poster
- CVPR 2024 - direct2.5)] [[Project Page](https://nju-3dv.github.io/projects/direct25/)]
- CVPR 2024
- arXiv 2023
- NeurIPS 2024
- SIGGRAPH 2024 Best Paper Honorable Mention - 3D/OpenCLAY)] [[Project Page](https://sites.google.com/view/clay-3dlm)]
- arXiv 2024 - 98/CraftsMan3D)] [[Project Page](https://craftsman3d.github.io/)]
- arXiv 2024
- arXiv 2023 - 123.github.io/)]
- arXiv 2023 - dream.github.io/)]
- CVPR 2023
- ICCV 2023 - columbia/zero123)] [[Project Page](https://zero123.cs.columbia.edu/)]
- ICLR 2024 Poster
- ICLR 2024 Poster - Research/TOSS)] [[Project Page](https://toss3d.github.io/)]
- ICLR 2024 Spotlight - pal/SyncDreamer)] [[Project Page](https://liuyuan-pal.github.io/SyncDreamer/)]
- CVPR 2024
- NeurIPS 2023 - 2-3-45/One-2-3-45)] [[Project Page](https://one-2-3-45.github.io/)]
- ECCV 2024 - ml/CRM)] [[Project Page](https://ml.cs.tsinghua.edu.cn/~zhengyi/CRM/)]
- arXiv 2024
- ICLR 2024 Oral
- NeurIPS 2024
- SIGGRAPH 2023
- NeurIPS 2023
- ACM MM 2023
- TVCG 2024
- ECCV 2024 - Project/DreamScene)] [[Project Page](https://dreamscene-project.github.io/)]
- arXiv 2024
- arXiv 2024
- ECCV 2022 - Group/INS)] [[Project Page](https://zhiwenfan.github.io/INS/)]
- ECCV 2022 - 46/ARF-svox2)] [[Project Page](https://www.cs.cornell.edu/projects/arf/)]
- CVPR 2024 Highlight - vision/vivid123)] [[Project Page](https://jgkwak95.github.io/ViVid-1-to-3/)]
- ICML 2024 - 3D/)]
- arXiv 2024
- ECCV 2024 Oral
- NeurIPS 2024 Oral
- CVPR 2023
- SIGGRAPH Asia 2022
- CVPR 2022
- SIGGRAPH 2023
- ICCV 2023
- ICCV 2023 Oral - nerf2nerf)] [[Project Page](https://instruct-nerf2nerf.github.io/)]
-
2D Generation
- NeurIPS 2022
- ICML 2021 - E)]
- arXiv 2022
- [Platform Link
- [GitHub
- CVPR 2022 - diffusion)]
- ICLR 2024 spotlight - AI/generative-models)]
- [Platform Link
-
-
Text to Scene
-
🎉 Scene Accepted Papers
- Link - page/) |
- Link
- Link - to-room/) |
- Link
- Link
- Link - - |--|
- Link
- Link - scene/) |
- Link - and-Fantasy) |[Link](https://leo81005.github.io/Reality-and-Fantasy/) |
- Link
- Link - - |[Link](https://replaceanything3d.github.io/) |
- Link
- Link
- Link - - | [Link](https://jonasschult.github.io/ControlRoom3D/) |
- Link - - |[Link](https://dave.ml/layoutlearning/) |
-
💡 Scene ArXiv Papers
- Link - Room) | [Link](https://fangchuan.github.io/ctrl-room.github.io/) |
- Link - - | [Link](https://ken-ouyang.github.io/text2immersion/index.html) |
- Link
- Link - - |-- |
- Link - - | -- |
- Link
- Link - IMTech/LayerPano3D) |[Link](https://ys-imtech.github.io/projects/LayerPano3D/) |
- Link - language) | [Link](https://ai.stanford.edu/~yzzhang/projects/scene-language/)|
- Link - 3D) | -- |
- Link
- Link
- Link - prometheus/)|
- Link
-
💡 3D Scene ArXiv Papers
-
🎉 3D Scene Accepted Papers
- Link - - | [Link](https://shengjun-zhang.github.io/SceneSplatter/) |
-
Year 2023-2024
-
-
Related Resources
-
Text to 'other tasks'
- ICLR Workshop on Neural Network Weights as a New Data Modality 2025
- arXiv 1 Sep 2024
- arXiv 7 Nov 2024 - MLLM/CAD-MLLM)] [[Project Page](https://cad-mllm.github.io/)]
- NeurIPS 2024 Spotlight - project/)] [[Dataset](https://huggingface.co/datasets/SadilKhan/Text2CAD)]
-
Survey and Awesome Repos
- A Survey on 3D Gaussian Splatting
- Awesome 3D Gaussian Splatting Resources
- 3D Gaussian Splatting Papers
- 3DGS and Beyond Docs
- A Survey On Text-to-3D Contents Generation In The Wild
- Awesome LLM 3D
- arXiv 6 June 2024
- Awesome-Avatars
- Awesome 4D Generation
- Awesome-Text/Image-to-3D
- arXiv 27 Mar 2025 - lin/Awesome-Physics-Cognition-based-Video-Generation)]
- Awesome-Physics-Cognition-based-Video-Generation
- Awesome Digital Human
- Cube
- arXiv 18 Mar 2025 - 4D)]
- arXiv 6 May 2024
- arXiv 17 Mar 2024
- IEEE TVCG 2024
- arXiv 8 Jan 2024 - Benchmarks)]
- arXiv 2 Feb 2024 - AIGC-3D)]
- arXiv 31 Jan 2024
- CVPR 2024
- arXiv 19 Mar 2025
- arXiv 5 Jan 2024
- Awesome 3D AIGC Resources
- Awesome Text-to-3D
- arXiv 19 Jan 2025 - Physics-aware-Generation)]
- CVPR 2024 - 3D-Gaussians)] [[Project Page](https://ingra14m.github.io/Deformable-Gaussians/)]
- CVPR 2024
- CVPR 2024 - GS)] [[Project Page](https://yihua7.github.io/SC-GS-web/)]
- CVPR 2024 Highlight
- SIGGRAPH 2024
- ICLR 2024 - zvg/4d-gaussian-splatting)] [[Project Page](https://fudan-zvg.github.io/4d-gaussian-splatting/)]
- CVPR 2024 Highlight - 3DV/Gaussian-Flow)] [[Project Page](https://nju-3dv.github.io/projects/Gaussian-Flow/)]
- 3DV 2024
- arXiv 15 May 2024
-
-
Other Related Resources
-
World Foundation Model Platform
- Cosmos-Predict1 - purpose world foundation models for Physical AI that can be fine-tuned into customized world models for downstream applications.
- Cosmos-Reason1 - of-thought reasoning processes.
- NVIDIA Cosmos - cosmos)] [[Paper](https://arxiv.org/abs/2501.03575)]): NVIDIA Cosmos is a world foundation model platform for accelerating the development of physical AI systems.
- Cosmos-Transfer1 - to-world transfer model designed to bridge the perceptual divide between simulated and real-world environments.
-
-
📜 Our Survey Paper Collection
Programming Languages
Categories
Sub Categories
3D Generation
68
Video Generation
57
Survey and Awesome Repos
36
🎉 Motion Accepted Papers
28
4D Generation
27
💡 4D ArXiv Papers
26
💡 Motion ArXiv Papers
19
💡 Video ArXiv Papers
19
🎉 Human Accepted Papers
16
🎉 Scene Accepted Papers
15
💡 Scene ArXiv Papers
13
💡 Human ArXiv Papers
11
2D Generation
8
World Foundation Model Platform
8
🎉 Video Accepted Papers
7
📚 Dataset Works
6
Year 2023-2024
6
Text to 'other tasks'
4
🎉 T2V Accepted Papers
4
Year 2023
4
💡 3D Scene ArXiv Papers
4
🎉 4D Accepted Papers
3
💡 T2V ArXiv Papers
3
Year 2024
2
Other 4D Additional Info
1
🎉 3D Scene Accepted Papers
1
Keywords
nerf
4
neural-rendering
4
text-to-3d
4
3dgs
3
3d-gaussian-splatting
3
3d-generation
3
t23d
2
tt3d
2
aigc
2
3d-aigc
2
smpl
2
3d
2
avatar
2
3d-reconstruction
2
image-to-3d
2
gaussian-splatting
2
3dreconstruction
1
diffusion
1
humannerf
1
motion-generation
1
novel-view-synthesis
1
vae
1
prior
1
pose-estimation
1
pose
1
motion
1
human
1
smpl-model
1
3d-keypoints
1
diffusion-to-3d
1
generation
1
computer-vision
1
computer-graphics
1
scene-generation
1
generative-ai
1
360-video
1
shape-generation
1
virtual-try-on
1
digital-human
1
clothed-people-digitalization
1
world-simulator
1
world-models
1
video-generation
1
v2v
1
t2v
1
survey
1
physics
1
embodied-ai
1
4d
1
3d-learning-from-2d
1