{"id":47584990,"url":"https://github.com/OpenDCAI/OpenWorldLib","last_synced_at":"2026-04-16T05:01:09.411Z","repository":{"id":345168668,"uuid":"1101358021","full_name":"OpenDCAI/OpenWorldLib","owner":"OpenDCAI","description":"Unified Codebase for Advanced World Models.","archived":false,"fork":false,"pushed_at":"2026-04-15T07:02:39.000Z","size":13495,"stargazers_count":652,"open_issues_count":4,"forks_count":34,"subscribers_count":11,"default_branch":"main","last_synced_at":"2026-04-15T09:09:57.304Z","etag":null,"topics":["3d-scene-generation","awesome-world-model","diffusion-models","gaussian-splatting","hunyuan-world","lingbot-world","simulator","vggt","video-editing","vla","vlm","world-model","world-models"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/OpenDCAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-21T15:03:35.000Z","updated_at":"2026-04-15T08:34:34.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/OpenDCAI/OpenWorldLib","commit_stats":null,"previous_names":["opendcai/openworldlib"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/OpenDCAI/OpenWorldLib","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDCAI%2FOpenWorldLib","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDCAI%2FOpenWorldLib/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDCAI%2FOpenWorldLib/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDCAI%2FOpenWorldLib/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/OpenDCAI","download_url":"https://codeload.github.com/OpenDCAI/OpenWorldLib/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/OpenDCAI%2FOpenWorldLib/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31872036,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-15T15:24:51.572Z","status":"online","status_checked_at":"2026-04-16T02:00:06.042Z","response_time":69,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-scene-generation","awesome-world-model","diffusion-models","gaussian-splatting","hunyuan-world","lingbot-world","simulator","vggt","video-editing","vla","vlm","world-model","world-models"],"created_at":"2026-04-01T07:00:19.656Z","updated_at":"2026-04-16T05:01:09.404Z","avatar_url":"https://github.com/OpenDCAI.png","language":"Python","funding_links":[],"categories":["Related Works"],"sub_categories":["2022"],"readme":"\u003cdiv align=\"center\" markdown=\"1\"\u003e\n\n\u003cimg src=\"https://github.com/user-attachments/assets/1935c89a-76cb-4edc-a6ac-0c3658d347f6\" alt=\"openworldlib_logo\" width=\"90%\" /\u003e\n\n#### Welcome to join us open-source world model project ! \u003c!-- omit in toc --\u003e\n---\n\n\u003ca href=\"https://github.com/OpenDCAI/OpenWorldLib\"\u003e\u003cimg alt=\"Build\" src=\"https://img.shields.io/github/stars/OpenDCAI/OpenWorldLib\"\u003e\u003c/a\u003e \u003c!-- License --\u003e \u003ca href=\"https://github.com/OpenDCAI/OpenWorldLib/blob/main/LICENSE\"\u003e\u003cimg alt=\"License\" src=\"https://img.shields.io/github/license/OpenDCAI/OpenWorldLib\"\u003e\u003c/a\u003e \u003c!-- GitHub Issues --\u003e \u003ca href=\"https://github.com/OpenDCAI/OpenWorldLib/issues\"\u003e\u003cimg alt=\"Issues\" src=\"https://img.shields.io/github/issues/OpenDCAI/OpenWorldLib\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/user-attachments/assets/35d48c4f-adb3-4f10-b30f-e7f4a245ab48\"\u003e\u003cimg alt=\"Add me on WeChat\" src=\"https://img.shields.io/badge/Connect_on-WeChat-07C160?style=flat-square\u0026logo=wechat\u0026logoColor=white\"\u003e\u003c/a\u003e \u003ca href=\"https://arxiv.org/abs/2604.04707\"\u003e\u003cimg alt=\"Paper\" src=\"https://img.shields.io/badge/arXiv-2604.04707-b31b1b?logo=arxiv\u0026logoColor=white\"\u003e\u003c/a\u003e\n\u003c!-- \u003cimg alt=\"Report\" src=\"https://img.shields.io/badge/πŸ“„ Technical Report-Coming Soon-lightgrey\"\u003e --\u003e\n\n[English](README.md) | [δΈ­ζ–‡](README_zh.md)\n\nExtension repo：[[3D generation]](https://github.com/zengbohan0217/OpenWorldLib-extension-3D) | [[VLA]](https://github.com/yfanDai/OpenWorldLib-extension-VLA) | [[simulator]](https://github.com/YF0224/OpenWorldLib-extension-Simulator)\n\n\u003c/div\u003e\n\n\u003c!-- add demo --\u003e\n\u003cdiv align=\"center\"\u003e\n\u003ctable\u003e\n\u003ctr\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/1c328d17-ef94-4d53-8dc2-f77ec3964a74\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eMatrix-Game-2\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/2fac45f5-d365-4794-bf38-45aee58f3d45\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eHunyuan-GameCraft\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/c594ac11-1fc7-45da-ab2e-426e66e927f6\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eHunyuan-Worldplay\u003c/b\u003e\n \u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/e07456a3-869a-408c-9755-abe896f51c0e\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eLingbot-World\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/ae38060b-f82f-48b1-89b5-6344e85e8354\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eYUME-1.5\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/c6b176fd-1ebe-4760-ab2f-6f2ba20445ca\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eFlashWorld\u003c/b\u003e\n \u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/c4fa5245-e837-4e47-a4e4-585a61358f91\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eWan-2.2-IT2V\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/a47b2e0e-b62c-45fb-bb99-6377f53e3f5e\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eWoW\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/c8ec425d-3815-4652-b2af-7e3c06e76b72\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eCosmos-Predict-2.5\u003c/b\u003e\n \u003c/td\u003e\n\u003c/tr\u003e\n\u003ctr\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/68c59c52-35c5-400c-a5a0-94c8eb802a79\" width=\"218\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003ePi3\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/dab9c489-fcb8-412d-99af-9fdad4e76e0e\" width=\"120\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eLibero\u003c/b\u003e\n \u003c/td\u003e\n \u003ctd align=\"center\"\u003e\n \u003cimg src=\"https://github.com/user-attachments/assets/f8b4253b-9172-4afa-bd0e-ae0bc7ea8b25\" width=\"120\" height=\"120\"\u003e\u003cbr/\u003e\n \u003cb\u003eAi2-THOR\u003c/b\u003e\n \u003c/td\u003e\n\u003c/tr\u003e\n\u003c/table\u003e\n\u003c/div\u003e\n\n\nWe define a world model as: **A model or framework centered on perception, equipped with interaction and long-term memory capabilities, for understanding and predicting the complex world.** Accordingly, πŸŽ“ *Multimodal Understanding*, πŸ€– *Visual Action Prediction*, and πŸ–ΌοΈ *Visual Generation* are all sub-tasks that a world model needs to accomplish.\n\nWe warmly welcome researchers to share their views on this framework or thoughts on world models in the Issues section. We also hope that you can submit valuable world-model-related methods to our framework via Pull Requests, or document and submit them to [[awesome_world_models]](docs/awesome_world_model.md). Feel free to give our repo a star 🌟 to follow the latest progress of OpenWorldLib!\n\n\n### Important Docs \u003c!-- omit in toc --\u003e\nThe following four documents are essential to this project (click to navigate):\n\n- [docs/planning.md](docs/planning.md): This document tracks the short-term optimization goals and future development plans for OpenWorldLib.\n- [docs/awesome_world_models.md](docs/awesome_world_model.md): This document records cutting-edge research, related surveys, and open-source projects on world models.\n- [docs/installation.md](docs/installation.md): This document provides installation instructions for different methods in OpenWorldLib.\n- [Development Guide](https://wcny4qa9krto.feishu.cn/wiki/XtPJwf5XQipP7RkeVv0ckyWlnNd?from=from_copylink): This document provides a framework template for OpenWorldLib, intended to serve as a reference for code integration for developers in the world model community.\n\n\n### Table of Contents \u003c!-- omit in toc --\u003e\n- [Features](#features)\n - [Project Goals](#project-goals)\n - [Supported Tasks](#supported-tasks)\n- [Getting Started](#getting-started)\n - [Installation](#installation)\n - [Quickstart](#quickstart)\n- [Structure](#structure)\n- [Planning](#planning)\n- [For Developers](#for-developers)\n- [Acknowledgment](#acknowledgment)\n- [Citation](#citation)\n\n\n### Features\n#### Project Goals\nThe main goals of OpenWorldLib include:\n- Establishing a unified and standardized **world model framework** to make the invocation of existing world-model-related code more consistent and well-structured;\n- Integrating open-source world model research outcomes and systematically curating related papers for researchers' reference and use.\n\n#### Supported Tasks\nOpenWorldLib covers the following research directions related to **World Models**, **We sincerely thank all the excellent methods included in this framework for their significant contributions to world model**:\n\n| Task Category | Sub-task | Representative Methods/Models |\n| :--- | :--- | :--- |\n| **Video Generation** | Navigation Generation | lingbot, matrix-game, hunyuan-worldplay, genie3, etc. |\n| | Long Video Generation | sora-2, veo-3, wan, etc. |\n| | Egocentric Interaction Generation | Hand2World, etc. |\n| **3D Scene Generation** | 3D Scene Generation | flash-world, vggt, etc. |\n| **Reasoning** | VQA (Visual Question Answering) | spatialVLM, omnivinci and other VLMs with world understanding |\n| | VLA (Vision-Language-Action) | pi-0, pi-0.5, giga-brain, etc. |\n\u003e Commonly used inference frameworks include: [diffusers](https://github.com/huggingface/diffusers), [DiffSynth](https://github.com/modelscope/DiffSynth-Studio), [LightX2V](https://github.com/ModelTC/LightX2V)\n\n### Getting Started\n#### Installation\nFirst, create a conda environment:\n```bash\nconda create -n \"openworldlib\" python=3.10 -y\nconda activate \"openworldlib\"\n```\nThen install using the provided script:\n```bash\ncd OpenWorldLib\nbash scripts/setup/default_install.sh\n```\nSome methods have special installation requirements. All installation scripts are located in `./scripts/setup`.\n\u003e πŸ“– For the full installation guide, please refer to [docs/installation.md](docs/installation.md)\n\n#### Quickstart\nAfter installing the base environment, you can test matrix-game-2 generation and multi-turn interaction with the following commands:\n```bash\ncd OpenWorldLib\nbash scripts/test_inference/test_nav_video_gen.sh matrix-game-2\nbash scripts/test_stream/test_nav_video_gen.sh matrix-game-2\n```\nScripts for other methods can be found under `scripts/test_inference` and `scripts/test_stream`. Currently, we are primarily using GPUs with **80GB** and **141GB** of VRAM for testing. In the future, we will test more models and provide updates in the `./docs/installation.md` file.\n\n### Structure\nTo help developers and users better understand OpenWorldLib, we provide details about our codebase. The framework structure is as follows:\n```txt\nOpenWorldLib\nβ”œβ”€ assets\nβ”œβ”€ data # Test data\nβ”‚ β”œβ”€ benchmarks\nβ”‚ β”œβ”€ test_case\nβ”‚ └─ ...\nβ”œβ”€ docs # Documentation\nβ”œβ”€ examples # Benchmark evaluation code\nβ”œβ”€ scripts # All key test scripts\nβ”œβ”€ src\nβ”‚ └─ openworldlib # Main source path\nβ”‚ β”œβ”€ base_models # Base models, provide basic modules for other parts\nβ”‚ β”‚ β”œβ”€ diffusion_model\nβ”‚ β”‚ β”œβ”€ llm_mllm_core\nβ”‚ β”‚ β”œβ”€ perception_core\nβ”‚ β”‚ └─ three_dimensions\nβ”‚ β”œβ”€ memories # Memory module\nβ”‚ β”‚ β”œβ”€ reasoning\nβ”‚ β”‚ └─ visual_synthesis\nβ”‚ β”œβ”€ operators # Input \u0026 interaction signal processing\nβ”‚ β”œβ”€ pipelines # All runtime pipelines\nβ”‚ β”œβ”€ reasoning # Reasoning module\nβ”‚ β”‚ β”œβ”€ audio_reasoning\nβ”‚ β”‚ β”œβ”€ general_reasoning\nβ”‚ β”‚ └─ spatial_reasoning\nβ”‚ β”œβ”€ representations # Representation module\nβ”‚ β”‚ β”œβ”€ point_clouds_generation\nβ”‚ β”‚ └─ simulation_environment\nβ”‚ └─ synthesis # Generation module\nβ”‚ β”œβ”€ audio_generation\nβ”‚ β”œβ”€ visual_generation\nβ”‚ └─ vla_generation\nβ”œβ”€ submodules # Auxiliary installs (e.g., diff-gaussian-raster)\nβ”œβ”€ test # All test code\nβ”œβ”€ test_stream # All interactive test code\n└─ tools # Utilities\n └─ vibe_code\n```\nWhen using OpenWorldLib, users typically call the **pipeline** class directly, which handles weight loading, environment initialization, and other tasks. Users interact with the **operator** class, and leverage the **synthesis**, **reasoning**, and **representation** classes for generation. In multi-turn interactions, the **memory** class is used to maintain the running context.\n```txt\nUser Input\n β”‚\n β–Ό\nβ”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”\nβ”‚ Pipeline │──── from_pretrained(): load models \u0026 initialize modules\nβ”‚ β”‚\nβ”‚ __call__() │──┬── process() ──► Operator (validate \u0026 preprocess)\nβ”‚ β”‚ β”‚ β”‚\nβ”‚ β”‚ β”‚ β”œβ”€β”€ β–Ί Synthesis.predict() β†’ multimodel outputs\nβ”‚ β”‚ β”‚ β”œβ”€β”€ β–Ί Reasoning.inference() β†’ text outputs\nβ”‚ β”‚ β”‚ └── β–Ί Representation.get_repr..() β†’ 3D outputs\nβ”‚ β”‚ β”‚\nβ”‚ stream() │──┬── memory.select() β†’ retrieve context\nβ”‚ β”‚ β”‚ β”œβ”€β”€ __call__() β†’ generate current turn\nβ”‚ β”‚ β”‚ └── memory.record() β†’ store results\nβ””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜\n```\n\n\n### Planning\n\n- We document the latest cutting-edge world model research in docs/awesome_world_models.md, and welcome contributions of valuable research.\n- We document our upcoming training and optimization plans in docs/planning.md.\n\n\n### For Developers\nWe welcome all developers to contribute and help improve **OpenWorldLib** as a unified world model repository. We recommend using **Vibe Coding** for quick code contributions β€” related prompts can be found under `tools/vibe_code/prompts`. You are also encouraged to add high-quality world model works to [docs/planning.md](docs/planning.md) and [docs/awesome_world_models.md](docs/awesome_world_model.md). We look forward to your contributions!\n\nRelated documents: [[Development Guide]](https://wcny4qa9krto.feishu.cn/wiki/XtPJwf5XQipP7RkeVv0ckyWlnNd?from=from_copylink)\n\n\n### Acknowledgment\nThis project is an extension of [DataFlow](https://github.com/OpenDCAI/DataFlow) and [DataFlow-MM](https://github.com/OpenDCAI/DataFlow-MM) for world model tasks. We are also actively collaborating with [RayOrch](https://github.com/OpenDCAI/RayOrch), [Paper2Any](https://github.com/OpenDCAI/Paper2Any), and other projects.\n\n\n### Citation\nIf OpenWorldLib has been helpful to you, please consider giving our repo a star 🌟 and citing the related papers:\n\u003c!-- @misc{dataflow-team-openworldlib,\n author = {{OpenDCAI}},\n title = {OpenWorldLib: A Unified Codebase for Advanced World Models},\n year = {2026},\n publisher = {GitHub},\n journal = {GitHub repository},\n howpublished = {\\url{https://github.com/OpenDCAI/OpenWorldLib}}\n} --\u003e\n```bibtex\n@article{dataflow2026openworldlib,\n title={OpenWorldLib: A Unified Codebase and Definition of Advanced World Models},\n author={DataFlow Team and others},\n journal={arXiv preprint arXiv:2604.04707},\n year={2026}\n}\n\n@article{zeng2026research,\n title={Research on World Models Is Not Merely Injecting World Knowledge into Specific Tasks},\n author={Zeng, Bohan and Zhu, Kaixin and Hua, Daili and Li, Bozhou and Tong, Chengzhuo and Wang, Yuran and Huang, Xinyi and Dai, Yifan and Zhang, Zixiang and Yang, Yifan and others},\n journal={arXiv preprint arXiv:2602.01630},\n year={2026}\n}\n```\nTo further elaborate on our framework's design philosophy and our understanding of world models, we will release a technical report for OpenWorldLib. We hope our work is helpful to you!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenDCAI%2FOpenWorldLib","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FOpenDCAI%2FOpenWorldLib","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FOpenDCAI%2FOpenWorldLib/lists"}