{"id":40708863,"url":"https://github.com/leggedrobotics/robotic_world_model","last_synced_at":"2026-01-21T12:38:17.721Z","repository":{"id":325991232,"uuid":"1103233444","full_name":"leggedrobotics/robotic_world_model","owner":"leggedrobotics","description":"Repository for our papers: Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics and Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots","archived":false,"fork":false,"pushed_at":"2026-01-17T17:20:23.000Z","size":2823,"stargazers_count":434,"open_issues_count":0,"forks_count":24,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-01-18T03:37:15.005Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://sites.google.com/view/roboticworldmodel","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leggedrobotics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-11-24T15:49:49.000Z","updated_at":"2026-01-18T02:04:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/leggedrobotics/robotic_world_model","commit_stats":null,"previous_names":["leggedrobotics/robotic_world_model"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/leggedrobotics/robotic_world_model","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leggedrobotics%2Frobotic_world_model","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leggedrobotics%2Frobotic_world_model/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leggedrobotics%2Frobotic_world_model/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leggedrobotics%2Frobotic_world_model/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leggedrobotics","download_url":"https://codeload.github.com/leggedrobotics/robotic_world_model/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leggedrobotics%2Frobotic_world_model/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28632940,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-21T04:47:28.174Z","status":"ssl_error","status_checked_at":"2026-01-21T04:47:22.943Z","response_time":86,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-21T12:38:17.048Z","updated_at":"2026-01-21T12:38:17.714Z","avatar_url":"https://github.com/leggedrobotics.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Robotic World Model Extension for Isaac Lab\n\n[![IsaacSim](https://img.shields.io/badge/IsaacSim-4.5.0-silver.svg)](https://docs.omniverse.nvidia.com/isaacsim/latest/overview.html)\n[![Isaac Lab](https://img.shields.io/badge/IsaacLab-2.1.0-silver)](https://isaac-sim.github.io/IsaacLab)\n[![Python](https://img.shields.io/badge/python-3.10-blue.svg)](https://docs.python.org/3/whatsnew/3.10.html)\n[![Linux platform](https://img.shields.io/badge/platform-linux--64-orange.svg)](https://releases.ubuntu.com/20.04/)\n[![Windows platform](https://img.shields.io/badge/platform-windows--64-orange.svg)](https://www.microsoft.com/en-us/)\n[![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit\u0026logoColor=white)](https://pre-commit.com/)\n[![License](https://img.shields.io/badge/license-MIT-yellow.svg)](https://opensource.org/license/mit)\n\n## Overview\n\nThis repository extends [**Isaac Lab**](https://github.com/isaac-sim/IsaacLab) with environments and training pipelines for\n- [**Robotic World Model (RWM)**](https://sites.google.com/view/roboticworldmodel/home),\n- [**Uncertainty-Aware Robotic World Model (RWM-U)**](https://sites.google.com/view/uncertainty-aware-rwm),\n\nand related model-based reinforcement learning methods.\n\nIt enables:\n- joint training of policies and neural dynamics models in Isaac Lab (online),\n- training of policies with learned neural network dynamics without any simulator (offline),\n- evaluation of model-based vs. model-free policies,\n- visualization of autoregressive imagination rollouts from learned dynamics,\n- visualization of trained policies in Isaac Lab.\n\n\n\u003ctable\u003e\n  \u003ctr\u003e\n  \u003ctd valign=\"top\" width=\"50%\"\u003e\n\n![Robotic World Model](rwm.png)\n\n**Paper**: [Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics](https://arxiv.org/abs/2501.10100)  \n**Project Page**: [https://sites.google.com/view/roboticworldmodel](https://sites.google.com/view/roboticworldmodel)\n\n  \u003c/td\u003e\n  \u003ctd valign=\"top\" width=\"50%\"\u003e\n\n![Uncertainty-Aware Robotic World Model](rwm-u.png)\n\n**Paper**: [Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots](https://arxiv.org/abs/2504.16680)  \n**Project Page**: [https://sites.google.com/view/uncertainty-aware-rwm](https://sites.google.com/view/uncertainty-aware-rwm)\n\n  \u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\n**Authors**: [Chenhao Li](https://breadli428.github.io/), [Andreas Krause](https://las.inf.ethz.ch/krausea), [Marco Hutter](https://rsl.ethz.ch/the-lab/people/person-detail.MTIxOTEx.TGlzdC8yNDQxLC0xNDI1MTk1NzM1.html)  \n**Affiliation**: [ETH AI Center](https://ai.ethz.ch/), [Learning \u0026 Adaptive Systems Group](https://las.inf.ethz.ch/) and [Robotic Systems Lab](https://rsl.ethz.ch/), [ETH Zurich](https://ethz.ch/en.html)\n\n\n---\n\n\n## Installation\n\n1. **Install Isaac Lab** (not needed for offline policy training)\n\nFollow the official [installation guide](https://isaac-sim.github.io/IsaacLab/main/source/setup/installation/index.html). We recommend using the Conda installation as it simplifies calling Python scripts from the terminal.\n\n2. **Install model-based RSL RL**\n\nFollow the official installation guide of model-based [RSL RL](https://github.com/leggedrobotics/rsl_rl_rwm) for model-based reinforcement learning to replace the `rsl_rl_lib` that comes with Isaac Lab.\n\n3. **Clone this repository** (outside your Isaac Lab directory)\n\n```bash\ngit clone git@github.com:leggedrobotics/robotic_world_model.git\n```\n\n4. **Install the extension** using the Python environment where Isaac Lab is installed\n\n```bash\npython -m pip install -e source/mbrl\n```\n\n5. **Verify the installation** (not needed for offline policy training)\n\n```bash\npython scripts/reinforcement_learning/rsl_rl/train.py --task Template-Isaac-Velocity-Flat-Anymal-D-Init-v0 --headless\n```\n\n---\n\n## World Model Pretraining \u0026 Evaluation\n\nRobotic World Model is a model-based reinforcement learning algorithm that learns a dynamics model and a policy concurrently.\n\n### Configure model inputs/outputs\n\nYou can configure the model inputs and outputs under `ObservationsCfg_PRETRAIN` in [`AnymalDFlatEnvCfg_PRETRAIN`](source/mbrl/mbrl/tasks/manager_based/locomotion/velocity/config/anymal_d/flat_env_cfg.py).\n\nAvailable components:\n- `SystemStateCfg`: state input and output head\n- `SystemActionCfg`: action input\n- `SystemExtensionCfg`: continuous privileged output head (e.g. rewards etc.)\n- `SystemContactCfg`: binary privileged output head (e.g. contacts)\n- `SystemTerminationCfg`: binary privileged output head (e.g. terminations)\n\nAnd you can configure the model architecture and training hyperparameters under `RslRlSystemDynamicsCfg` and `RslRlMbrlPpoAlgorithmCfg` in [`AnymalDFlatPPOPretrainRunnerCfg`](source/mbrl/mbrl/tasks/manager_based/locomotion/velocity/config/anymal_d/agents/rsl_rl_ppo_cfg.py) .\n\nAvailable options:\n- `ensemble_size`: ensemble size for uncertainty estimation\n- `history_horizon`: stacked history horizon\n- `architecture_config`: architecture configuration\n- `system_dynamics_forecast_horizon`: autoregressive prediction steps\n\n### Run dynamics model pretraining:\n\n```bash\npython scripts/reinforcement_learning/rsl_rl/train.py \\\n  --task Template-Isaac-Velocity-Flat-Anymal-D-Pretrain-v0 \\\n  --headless\n```\n\nIt trains a PPO policy from scratch, while the induced experience during training is used to train the dynamics model.\n\n### Visualize autoregressive predictions\n\n```bash\npython scripts/reinforcement_learning/rsl_rl/visualize.py \\\n  --task Template-Isaac-Velocity-Flat-Anymal-D-Visualize-v0 \\\n  --checkpoint \u003ccheckpoint_path\u003e \\\n  --system_dynamics_load_path \u003cdynamics_model_path\u003e\n```\n\nIt visualizes the learned dynamics model by rolling out the model autoregressively in imagination, conditioned on the actions from the learned policy.\nThe `dynamics_model_path` should point to the pretrained dynamics model checkpoint (e.g. `model_\u003citeration\u003e.pt`) inside the saved run directory.\n\n---\n\n## Model-Based Policy Training \u0026 Evaluation\n\nOnce a dynamics model is pretrained, you can train a model-based policy purely from **imagined rollouts** generated by the learned dynamics.\n\nThere are two options:\n- **Option 1: Train policy in imagination *online***, where additional environment interactions are continually collected using the latest policy to update the dynamics model (as implemented with RWM and MBPO-PPO in [Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics](https://arxiv.org/abs/2501.10100)).\n- **Option 2: Train policy in imagination *offline*** where no additional environment interactions are collected and the policy has to rely on the static dynamics model (as implemented with RWM-U and MOPO-PPO in [Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robots](https://arxiv.org/abs/2504.16680)).\n\n### Option 1: Train policy in imagination *online*\n\nThe online data collection relies on interactions with the environment and thus brings up the simulator.\n\n```bash\npython scripts/reinforcement_learning/rsl_rl/train.py --task Template-Isaac-Velocity-Flat-Anymal-D-Finetune-v0 --headless --checkpoint \u003ccheckpoint_path\u003e --system_dynamics_load_path \u003cdynamics_model_path\u003e\n```\n\nYou can either start the policy from pretrained checkpoints or from scratch by simply omitting the `--checkpoint` argument.\n\n### Option 2: Train policy in imagination *offline*\n\nThe offline policy training does not request any new data and thus relies solely on the static dynamics model.\nAlign the model architecture and specify the model load path under `ModelArchitectureConfig` in [`AnymalDFlatConfig`](scripts/reinforcement_learning/model_based/configs/anymal_d_flat_cfg.py).\n\nAdditionally, the offline imagination needs to branch off from some initial states. Specify the data path under `DataConfig` in [`AnymalDFlatConfig`](scripts/reinforcement_learning/model_based/configs/anymal_d_flat_cfg.py).\n\n```bash\npython scripts/reinforcement_learning/model_based/train.py --task anymal_d_flat\n```\n\n### Play the learned model-based policy\n\nYou can play the learned policies with the original Isaac Lab task registry.\n\n```bash\npython scripts/reinforcement_learning/rsl_rl/play.py --task Isaac-Velocity-Flat-Anymal-D-Play-v0 --checkpoint \u003ccheckpoint_path\u003e\n```\n\n---\n\n## Code Structure\n\nWe provide a reference pipeline that enables RWM and RWM-U on ANYmal D.\n\nKey files:\n\n**Online**\n\n- Environment configurations + dynamics model setup\n  [`flat_env_cfg.py`](source/mbrl/mbrl/tasks/manager_based/locomotion/velocity/config/anymal_d/flat_env_cfg.py).\n- Algorithm configuration + training parameters\n  [`rsl_rl_ppo_cfg.py`](source/mbrl/mbrl/tasks/manager_based/locomotion/velocity/config/anymal_d/agents/rsl_rl_ppo_cfg.py).\n- Imagination rollout logic (constructs policy observations \u0026 rewards from model outputs)\n  [`anymal_d_manager_based_mbrl_env`](source/mbrl/mbrl/tasks/manager_based/locomotion/velocity/config/anymal_d/envs/anymal_d_manager_based_mbrl_env.py).\n- Visualization environment + rollout reset\n  [`anymal_d_manager_based_visualize_env.py`](source/mbrl/mbrl/tasks/manager_based/locomotion/velocity/config/anymal_d/envs/anymal_d_manager_based_visualize_env.py).\n\n**Offline**\n\n- Environment configurations + Imagination rollout logic (constructs policy observations \u0026 rewards from model outputs)\n  [`anymal_d_flat.py`](scripts/reinforcement_learning/model_based/envs/anymal_d_flat.py).\n- Algorithm configuration + training parameters\n  [`anymal_d_flat_cfg.py`](scripts/reinforcement_learning/model_based/configs/anymal_d_flat_cfg.py).\n- Pretrained RWM-U checkpoint\n  [`pretrain_rnn_ens.pt`](assets/models/pretrain_rnn_ens.pt).\n- Initial states for imagination rollout\n  [`state_action_data_0.csv`](assets/data/state_action_data_0.csv).\n\n\n---\n\n## Citation\nIf you find this repository useful for your research, please consider citing:\n\n```text\n@article{li2025robotic,\n  title={Robotic world model: A neural network simulator for robust policy optimization in robotics},\n  author={Li, Chenhao and Krause, Andreas and Hutter, Marco},\n  journal={arXiv preprint arXiv:2501.10100},\n  year={2025}\n}\n@article{li2025offline,\n  title={Uncertainty-aware robotic world model makes offline model-based reinforcement learning work on real robots},\n  author={Li, Chenhao and Krause, Andreas and Hutter, Marco},\n  journal={arXiv preprint arXiv:2504.16680},\n  year={2025}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleggedrobotics%2Frobotic_world_model","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleggedrobotics%2Frobotic_world_model","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleggedrobotics%2Frobotic_world_model/lists"}