{"id":16778693,"url":"https://github.com/binary-husky/unreal-map","last_synced_at":"2025-04-13T00:48:31.889Z","repository":{"id":65016891,"uuid":"573737753","full_name":"binary-husky/unreal-map","owner":"binary-husky","description":"Multiagent research environment toolbox based on Unreal Engine","archived":false,"fork":false,"pushed_at":"2025-03-25T03:09:29.000Z","size":154970,"stargazers_count":221,"open_issues_count":8,"forks_count":35,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-13T00:48:02.173Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/binary-husky.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-03T09:14:20.000Z","updated_at":"2025-04-10T14:24:05.000Z","dependencies_parsed_at":"2023-01-12T07:15:52.470Z","dependency_job_id":"67ac698e-6d86-419f-8cb9-40459b821097","html_url":"https://github.com/binary-husky/unreal-map","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binary-husky%2Funreal-map","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binary-husky%2Funreal-map/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binary-husky%2Funreal-map/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binary-husky%2Funreal-map/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/binary-husky","download_url":"https://codeload.github.com/binary-husky/unreal-map/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248650437,"owners_count":21139672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T07:28:27.894Z","updated_at":"2025-04-13T00:48:31.860Z","avatar_url":"https://github.com/binary-husky.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Unreal-MAP \n\n[English](README.md) | [中文](README_CN.md)\n\n[![Version](https://img.shields.io/badge/version-3.14-blue)](https://github.com/binary-husky/unreal-map)\n[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.7+-blue)](https://www.python.org/)\n[![Unreal Engine](https://img.shields.io/badge/Unreal%20Engine-4.27-blue)](https://www.unrealengine.com/)\n[![stars](https://img.shields.io/github/stars/binary-husky/unreal-map)](https://github.com/binary-husky/unreal-map)\n[![Documentation](https://img.shields.io/badge/docs-English-blue)](README.md)\n\n\nThis is **Unreal Multi-Agent Playground** (Unreal-Map), a multi-agent general platform based on [Unreal Engine](https://www.unrealengine.com/).\nHere you can use all the capabilities of Unreal Engine (Blueprints, Behavior tree, Physics engine, AI navigation, 3D models/animations and Plugin resources, etc) to build elegant (but also computational efficient) and magnificent (but also experimentally reproducible) multi-agent environments.\n\nUnreal-MAP can not only be used to develop conventional multi-agent simulation environments, but has also been optimized for Multi-Agent Reinforcement Learning (MARL) simulation. You can use it to develop various realistic and complex MARL scenarios. You can also use Unreal-MAP together with our developed [HMAP](https://github.com/binary-husky/hmp2g) (a powerful MARL-specific experimental framework) to easily develop MARL scenarios and quickly deploy cutting-edge algorithms.\n\n\n\u003e The present study aims to identify potential collaboration partners. If interested in this research project, please feel free to contact our office at CASIA: tenghai.qiu@ia.ac.cn, hutianyi2021@ia.ac.cn.\n\u003e \n\n\n**Please ```star``` the Github project. Your encouragement is extremely important to us as researchers: ```https://github.com/binary-husky/unreal-map```** !\n\n\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"Docs/Imgs/Overall.png\"/ width=\"550\"\u003e \n\u003c/div\u003e\n\n# 1. Introduction\n### 1.1 Basic Introduction\nUnreal-based Multi-Agent Playground (Unreal-MAP) is a new generation of multi-agent general platform based on the Unreal Engine.\nThis platform supports adversial training between swarms \u0026 algorithms, and it is the first (and currently the only) extensible RL/MARL environment based on the Unreal Engine to support multi-team training.\n\n### 1.2 Architecture\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"Docs/Imgs/Architecture.png\"/ width=\"800\"\u003e \n\u003c/div\u003e\n\nUnreal-MAP employs a hierarchical five-layer architecture,\nwhere each layer builds upon the previous one. From bottom\nto top,the five layers are: *native layer*, *specification layer*, *base class layer*, ***advanced module layer***, and ***interface layer***.\nlayer. **You only need to focus on the *advanced module layer* (Blueprint) and the *interface layer* (Python).** \nFrom the perspective of creating a standard MARL scenario, using these two layers is sufficient to modify all elements in the task (e.g., POMDP) such as states, actions, observations, transitions, etc.\n\n### 1.3 Features\n\nUnreal-MAP can be used to develop various multi-agent simulation scenarios. Our case studies have already included scenarios with large-scale, heterogeneous, and multi-team characteristics.\n**Compared to other RL general platforms** such as [Unity ML-Agents](https://unity-technologies.github.io/ml-agents/), Unreal-MAP has the following advantages in terms of scientific research and experiment:\n\n**(1) Fully Open-Source and Easily Modifiable**: Unreal-MAP utilizes a layered design, and all components from the bottom-level engine to the top-level interfaces are open-sourced.\n\n**(2) Optimized Specifically for MARL**: The underlying engine of Unreal-MAP has been optimized to enhance efficiency in large-scale agent simulations and data transmission. \n\n**(3) Parallel Multi-Process Execution and Controllable Single-Process Time Flow**: Unreal-MAP supports the parallel execution of multiple simulation processes as well as the adjustment of the simulation time flow speed in a single process. You can accelerate simulations to speed up training or decelerate simulations for detailed slow-motion analysis.\n\n\n**Compared to all current MARL simulation environments**, Unreal-MAP has advantages in terms of scientific research and experiment:\n\n- **Freely build realistic tasks** using the massive resources available in the [Unreal Engine Marketplace](https://www.fab.com/).\n- Simultaneously supports **large-scale, heterogeneous, multi-team** simulations.\n- **Highly efficient training** with TPS (Timesteps per second) up to 10k+ and FPS (Frames per second) up to 10M+.\n- **Controllable simulation time**: you can accelerate simulation to speed up training (until CPU is fully utilized, acceleration doesn't consume extra memory or VRAM), or decelerate for slow-motion analysis.\n- **Strong reproducibility**: eliminated various butterfly effect factors in Unreal Engine that could cause experimental irreproducibility.\n- **Multi-platform support**: compile both Headless mode and rendering mode clients on Windows, Linux, and MacOS. \n- **Rich rendering mechanisms**: supports a) rendering in the UE editor, b) on a compiled pure rendering client, c) cross-platform real-time rendering. You can train on a Linux server and render on Windows host at the same time!\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"Docs/unreal-island.jpg\" height=\"200\" width=\"320\"/\u003e \u003cimg src=\"https://github.com/binary-husky/unreal-map/assets/96192199/985c2c27-bc0a-4c90-a036-ec676d7aec1d\" height=\"200\" width=\"320\"/\u003e \n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"Docs/Demo/uhmap-bbad.jpg\" height=\"200\" width=\"320\"/\u003e \u003cimg src=\"Docs/Demo/uhmap-hete.jpg\" height=\"200\" width=\"320\"/\u003e \n\u003c/div\u003e\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"Docs/Demo/2023-02-12 155956.jpg\" height=\"200\" width=\"320\"/\u003e \u003cimg src=\"Docs/Demo/2023-02-12 151938.jpg\" height=\"200\" width=\"320\"/\u003e \n\u003c/div\u003e\n\n\n### 1.4 Some Future Works\n\nUnreal-MAP introduces modern game engines into the MARL field with tremendous potential. This potential is mainly reflected in two dimensions: **Scalability** and **Realism**. In terms of scalability, users can not only ***freely*** construct environments using the extremely rich resources from the Unreal Engine community, but can also ***quickly*** build environments according to their ideas using Unreal Engine's future generative AI plugins (such as [ACE](https://developer.nvidia.com/ace.)).\n\nIn terms of realism, users can leverage Unreal-MAP to build ***highly realistic*** MARL environments and even develop ***digital twins*** of real-world scenarios. We attempted a sim2real demo using Unreal-MAP. In this demo, we first deployed a multi-UAV-UGV gaming scenario in the experimental field, then recreated the scenario using Unreal-MAP (including model proportions, agent kinematics and dynamics, etc.). We conducted training in the sim environment and then validated it in the real-world scenario, achieving preliminary positive results. In the current solution, Unreal-MAP not only serves as a simulation environment creator, but also acts as a data transmission intermediary, connecting data from the real-world scenario with the algorithmic side.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"Docs/Imgs/Sim2RealEXP.png\" width=\"800\"/\u003e \n\u003c/div\u003e\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"Docs/Imgs/Sim2RealFra.png\" width=\"700\"/\u003e \n\u003c/div\u003e\n\n\n\n# 2. How to Install \n\n## 2.1 Professional version\n\n- Step 1, you must install the Unreal Engine from the source code. For details, see the official document of the Unreal Engine: ```https://docs.unrealengine.com/4.27/zh-CN/ProductionPipelines/DevelopmentSetup/BuildingUnrealEngine/```\n- Step 2: Clone the git resp ```git clone https://github.com/binary-husky/unreal-hmp.git```\n- Step 3: Download large files that github cannot manage. Run ```python Please_ Run_ This_ First_ To_ Fetch_ Big_ Files.py```\n- Step 4: Right click the ```UHMP.upproject``` downloaded in step 3, select ```switch unreal engine version```, and then select ```source build at xxxxx``` to confirm. Then open the generated ```UHMP. sln``` and compile it\n- Finally, double-click ```UHMP. upproject``` to enter the Unreal Engine Editor.\n\nNote that steps 1 and 4 are difficult. It is recommended to refer to the following video (the 0:00-\u003e1:46 in the video is the steps 1, and 1:46-\u003eend is steps 4): ```https://ageasga-my.sharepoint.com/:v:/g/personal/fuqingxu_yiteam_tech/EawfqsV2jF5Nsv3KF7X1-woBH-VTvELL6FSRX4cIgUboLg?e=Vmp67E```\n\n\n\n## 2.2 Only compiled binary version \n\n```https://github.com/binary-husky/hmp2g/blob/master/ZDOCS/use_unreal_hmap.md```\n\n# 3. Tutorial \nThe document is being improved. For the video tutorial of simple demo, see ```EnvDesignTutorial.pptx``` (you need to complete step 3 of installation to download this pptx file)\n\nDirectory:\n- Chapter I. Unreal Engine\n- - Build a map (Level) ```https://www.bilibili.com/video/BV1U24y1D7i4/?spm_id_from=333.999.0.0\u0026vd_source=e3bc3eddd1d2414cb64ae72b6a64df55```\n- - Establish Agent Actor\n- - Design agent blueprint program logic\n- - Episode key event notification mechanism\n- - Define Custom actions (Unreal Engine side)\n- - The Python side controls the custom parameters of the agent\n- Chapter II. Python Interface\n- - Create a task file (SubTask)\n- - Modify agent initialization code\n- - Modify the agent reward code\n- - Select the control algorithm of each team\n- - Full closed loop debugging method\n- Chapter III. Appendix\n- - Headless acceleration and cross-compiling Linux package\n- - Define Custom actions (Need to be familiar with the full closed-loop debugging method first)\n- - - Draft a list of actions\n- - - Python side action generation\n- - - UE-side action parse and execution\n- - - Action discretization\n- - Installation guide for cross compilation tool chain\n\n\n\n# 4. How to Build Binary Client\nRun following scripts.\n```\npython BuildlinuxRender.py\npython BuildLinuxServer.py\npython BuildWinRender.py\npython BuildWinServer.py\n```\n- Among them, ```Render/Server``` represents ```including graphic rendering / only computing```, the later is generally used for RL training.\n- Among them, ```Windows/linux``` represents the target operating system. Note that you need to install ```Unreal Engine Cross Compilation Tool``` to compile Linux programs on Windows.\n\n\n\n\n- After adding new ActionSets in ```Content/Assets/DefAction/ParseAction.uasset```, you may encounter ```Ensure condition failed: !FindPin(FFunctionEntryHelper::GetWorldContextPinName())``` error during packaging. If so, find and remove an extra blueprint function parameter named ```__WorldContext``` that you created by accident in ```ParseAction.uasset```. For more details: ```https://forums.unrealengine.com/t/ensure-condition-failed-on-project-start/469587```\n\n- If you encounter BuildCMakeLib.Automation.cs(45,54): error CS1002 after project migration, please **Rebuild** (not Build!) the AutomationTool in Visual Studio. For more details: ```https://forums.unrealengine.com/t/unreal-engine-version-4-27-2-i-get-an-error-when-trying-to-package-any-project/270627```\n\n# Cite this project !\n\n```\n@article{unrealmap,\n  title={Unreal-MAP: Unreal-Engine-Based General Platform for Multi-Agent Reinforcement Learning},\n  author={Hu, Tianyi and Fu, Qingxu and Pu, Zhiqiang and Wang, Yuan and Qiu, Tenghai},\n  journal={arXiv preprint arXiv:2503.15947},\n  year={2025}\n}\n```\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinary-husky%2Funreal-map","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbinary-husky%2Funreal-map","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinary-husky%2Funreal-map/lists"}