https://github.com/OpenDCAI/OpenWorldLib

Unified Codebase for Advanced World Models.
https://github.com/OpenDCAI/OpenWorldLib

3d-scene-generation awesome-world-model diffusion-models gaussian-splatting hunyuan-world lingbot-world simulator vggt video-editing vla vlm world-model world-models

Last synced: about 2 months ago
JSON representation

Unified Codebase for Advanced World Models.

Host: GitHub
URL: https://github.com/OpenDCAI/OpenWorldLib
Owner: OpenDCAI
License: apache-2.0
Created: 2025-11-21T15:03:35.000Z (7 months ago)
Default Branch: main
Last Pushed: 2026-04-15T07:02:39.000Z (about 2 months ago)
Last Synced: 2026-04-15T09:09:57.304Z (about 2 months ago)
Topics: 3d-scene-generation, awesome-world-model, diffusion-models, gaussian-splatting, hunyuan-world, lingbot-world, simulator, vggt, video-editing, vla, vlm, world-model, world-models
Language: Python
Homepage:
Size: 12.9 MB
Stars: 652
Watchers: 11
Forks: 34
Open Issues: 4
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

awesome-embodied-vla-va-vln - [repo

README

openworldlib_logo

#### Welcome to join us open-source world model project !
---

[English](README.md) | [中文](README_zh.md)

Extension repo：[[3D generation]](https://github.com/zengbohan0217/OpenWorldLib-extension-3D) | [[VLA]](https://github.com/yfanDai/OpenWorldLib-extension-VLA) | [[simulator]](https://github.com/YF0224/OpenWorldLib-extension-Simulator)

Matrix-Game-2

Hunyuan-GameCraft

Hunyuan-Worldplay

Lingbot-World

YUME-1.5

FlashWorld

Wan-2.2-IT2V

WoW

Cosmos-Predict-2.5

Pi3

Libero

Ai2-THOR

We define a world model as: **A model or framework centered on perception, equipped with interaction and long-term memory capabilities, for understanding and predicting the complex world.** Accordingly, 🎓 *Multimodal Understanding*, 🤖 *Visual Action Prediction*, and 🖼️ *Visual Generation* are all sub-tasks that a world model needs to accomplish.

We warmly welcome researchers to share their views on this framework or thoughts on world models in the Issues section. We also hope that you can submit valuable world-model-related methods to our framework via Pull Requests, or document and submit them to [[awesome_world_models]](docs/awesome_world_model.md). Feel free to give our repo a star 🌟 to follow the latest progress of OpenWorldLib!

### Important Docs
The following four documents are essential to this project (click to navigate):

- [docs/planning.md](docs/planning.md): This document tracks the short-term optimization goals and future development plans for OpenWorldLib.
- [docs/awesome_world_models.md](docs/awesome_world_model.md): This document records cutting-edge research, related surveys, and open-source projects on world models.
- [docs/installation.md](docs/installation.md): This document provides installation instructions for different methods in OpenWorldLib.
- [Development Guide](https://wcny4qa9krto.feishu.cn/wiki/XtPJwf5XQipP7RkeVv0ckyWlnNd?from=from_copylink): This document provides a framework template for OpenWorldLib, intended to serve as a reference for code integration for developers in the world model community.

### Table of Contents
- [Features](#features)
- [Project Goals](#project-goals)
- [Supported Tasks](#supported-tasks)
- [Getting Started](#getting-started)
- [Installation](#installation)
- [Quickstart](#quickstart)
- [Structure](#structure)
- [Planning](#planning)
- [For Developers](#for-developers)
- [Acknowledgment](#acknowledgment)
- [Citation](#citation)

### Features
#### Project Goals
The main goals of OpenWorldLib include:
- Establishing a unified and standardized **world model framework** to make the invocation of existing world-model-related code more consistent and well-structured;
- Integrating open-source world model research outcomes and systematically curating related papers for researchers' reference and use.

#### Supported Tasks
OpenWorldLib covers the following research directions related to **World Models**, **We sincerely thank all the excellent methods included in this framework for their significant contributions to world model**:

### Getting Started
#### Installation
First, create a conda environment:
```bash
conda create -n "openworldlib" python=3.10 -y
conda activate "openworldlib"
```
Then install using the provided script:
```bash
cd OpenWorldLib
bash scripts/setup/default_install.sh
```
Some methods have special installation requirements. All installation scripts are located in `./scripts/setup`.
> 📖 For the full installation guide, please refer to [docs/installation.md](docs/installation.md)

#### Quickstart
After installing the base environment, you can test matrix-game-2 generation and multi-turn interaction with the following commands:
```bash
cd OpenWorldLib
bash scripts/test_inference/test_nav_video_gen.sh matrix-game-2
bash scripts/test_stream/test_nav_video_gen.sh matrix-game-2
```
Scripts for other methods can be found under `scripts/test_inference` and `scripts/test_stream`. Currently, we are primarily using GPUs with **80GB** and **141GB** of VRAM for testing. In the future, we will test more models and provide updates in the `./docs/installation.md` file.

### Structure
To help developers and users better understand OpenWorldLib, we provide details about our codebase. The framework structure is as follows:
```txt
OpenWorldLib
├─ assets
├─ data # Test data
│ ├─ benchmarks
│ ├─ test_case
│ └─ ...
├─ docs # Documentation
├─ examples # Benchmark evaluation code
├─ scripts # All key test scripts
├─ src
│ └─ openworldlib # Main source path
│ ├─ base_models # Base models, provide basic modules for other parts
│ │ ├─ diffusion_model
│ │ ├─ llm_mllm_core
│ │ ├─ perception_core
│ │ └─ three_dimensions
│ ├─ memories # Memory module
│ │ ├─ reasoning
│ │ └─ visual_synthesis
│ ├─ operators # Input & interaction signal processing
│ ├─ pipelines # All runtime pipelines
│ ├─ reasoning # Reasoning module
│ │ ├─ audio_reasoning
│ │ ├─ general_reasoning
│ │ └─ spatial_reasoning
│ ├─ representations # Representation module
│ │ ├─ point_clouds_generation
│ │ └─ simulation_environment
│ └─ synthesis # Generation module
│ ├─ audio_generation
│ ├─ visual_generation
│ └─ vla_generation
├─ submodules # Auxiliary installs (e.g., diff-gaussian-raster)
├─ test # All test code
├─ test_stream # All interactive test code
└─ tools # Utilities
└─ vibe_code
```
When using OpenWorldLib, users typically call the **pipeline** class directly, which handles weight loading, environment initialization, and other tasks. Users interact with the **operator** class, and leverage the **synthesis**, **reasoning**, and **representation** classes for generation. In multi-turn interactions, the **memory** class is used to maintain the running context.
```txt
User Input
│
▼
┌─────────────┐
│ Pipeline │──── from_pretrained(): load models & initialize modules
│ │
│ __call__() │──┬── process() ──► Operator (validate & preprocess)
│ │ │ │
│ │ │ ├── ► Synthesis.predict() → multimodel outputs
│ │ │ ├── ► Reasoning.inference() → text outputs
│ │ │ └── ► Representation.get_repr..() → 3D outputs
│ │ │
│ stream() │──┬── memory.select() → retrieve context
│ │ │ ├── __call__() → generate current turn
│ │ │ └── memory.record() → store results
└─────────────┘
```

### Planning

- We document the latest cutting-edge world model research in docs/awesome_world_models.md, and welcome contributions of valuable research.
- We document our upcoming training and optimization plans in docs/planning.md.

### For Developers
We welcome all developers to contribute and help improve **OpenWorldLib** as a unified world model repository. We recommend using **Vibe Coding** for quick code contributions — related prompts can be found under `tools/vibe_code/prompts`. You are also encouraged to add high-quality world model works to [docs/planning.md](docs/planning.md) and [docs/awesome_world_models.md](docs/awesome_world_model.md). We look forward to your contributions!

Related documents: [[Development Guide]](https://wcny4qa9krto.feishu.cn/wiki/XtPJwf5XQipP7RkeVv0ckyWlnNd?from=from_copylink)

### Acknowledgment
This project is an extension of [DataFlow](https://github.com/OpenDCAI/DataFlow) and [DataFlow-MM](https://github.com/OpenDCAI/DataFlow-MM) for world model tasks. We are also actively collaborating with [RayOrch](https://github.com/OpenDCAI/RayOrch), [Paper2Any](https://github.com/OpenDCAI/Paper2Any), and other projects.

### Citation
If OpenWorldLib has been helpful to you, please consider giving our repo a star 🌟 and citing the related papers:

```bibtex
@article{dataflow2026openworldlib,
title={OpenWorldLib: A Unified Codebase and Definition of Advanced World Models},
author={DataFlow Team and others},
journal={arXiv preprint arXiv:2604.04707},
year={2026}
}

@article{zeng2026research,
title={Research on World Models Is Not Merely Injecting World Knowledge into Specific Tasks},
author={Zeng, Bohan and Zhu, Kaixin and Hua, Daili and Li, Bozhou and Tong, Chengzhuo and Wang, Yuran and Huang, Xinyi and Dai, Yifan and Zhang, Zixiang and Yang, Yifan and others},
journal={arXiv preprint arXiv:2602.01630},
year={2026}
}
```
To further elaborate on our framework's design philosophy and our understanding of world models, we will release a technical report for OpenWorldLib. We hope our work is helpful to you!

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/OpenDCAI/OpenWorldLib

Awesome Lists containing this project

README