{"id":13832479,"url":"https://github.com/chauncygu/Safe-Multi-Agent-Mujoco","last_synced_at":"2025-07-09T19:30:29.628Z","repository":{"id":37014028,"uuid":"414196572","full_name":"chauncygu/Safe-Multi-Agent-Mujoco","owner":"chauncygu","description":"Safe Multi-Agent MuJoCo benchmark for safe multi-agent reinforcement learning research.","archived":false,"fork":false,"pushed_at":"2024-06-13T16:56:02.000Z","size":2278,"stargazers_count":43,"open_issues_count":3,"forks_count":7,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-08-04T11:01:14.558Z","etag":null,"topics":["benchmark","reinforcement-learning","robotics","safe"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chauncygu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-06T12:09:06.000Z","updated_at":"2024-07-24T06:40:27.000Z","dependencies_parsed_at":"2024-06-13T19:43:24.727Z","dependency_job_id":"1966c45b-92e2-4223-a934-49f8426779be","html_url":"https://github.com/chauncygu/Safe-Multi-Agent-Mujoco","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chauncygu%2FSafe-Multi-Agent-Mujoco","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chauncygu%2FSafe-Multi-Agent-Mujoco/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chauncygu%2FSafe-Multi-Agent-Mujoco/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chauncygu%2FSafe-Multi-Agent-Mujoco/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chauncygu","download_url":"https://codeload.github.com/chauncygu/Safe-Multi-Agent-Mujoco/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":225581763,"owners_count":17491786,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","reinforcement-learning","robotics","safe"],"created_at":"2024-08-04T11:00:19.121Z","updated_at":"2024-11-20T15:30:43.808Z","avatar_url":"https://github.com/chauncygu.png","language":"Python","funding_links":[],"categories":["Benchmark"],"sub_categories":[],"readme":"# Safe Multi-Agent Mujoco\r\n\r\nWe introduce a safe Multi-Agent Reinforcement Learning Benchmark, **Safe Multi-Agent MuJoCo (Safe MAMujoco)**, a safety-aware modification of [MAMuJoCo](https://github.com/schroederdewitt/multiagent_mujoco).  Safe MAMuJoCo agents learn to not only skilfully manipulate a robot, but also to avoid dangerous obstacles and positions, Figure 1 shows example views of the environment. (This repository is under actively development. We appreciate any constructive comments and suggestions)\r\n\r\n\r\n\r\nIn particular, the background environment, agents, physics simulator, and the reward function are preserved. However, as oppose to its predecessor, Safe MAMuJoCo environments come with obstacles, like walls or bombs. Furthermore, with the increasing risk of an agent stumbling upon an obstacle, the environment emits cost.\r\n\r\n\u003c!--\r\n\u003cdiv align=center\u003e\r\n \u003cimg src=\"https://github.com/chauncygu/Safe-Multi-Agent-Mujoco/blob/main/figures/Ant2x4_new_green_yellow.jpg\" width=\"165\"/\u003e \u003cimg src=\"https://github.com/chauncygu/Safe-Multi-Agent-Mujoco/blob/main/figures/New_Ant4x2.jpg\" width=\"175\"/\u003e \u003cimg src=\"https://github.com/chauncygu/Safe-Multi-Agent-Mujoco/blob/main/figures/couple_halfcheetah_p1p_color.jpg\" width=\"213\"/\u003e\r\n    \u003cimg src=\"https://github.com/chauncygu/Safe-Multi-Agent-Mujoco/blob/main/figures/New_HalfCheetah2x3.jpg\" width=\"202\"/\u003e  \r\n\u003c/div\u003e\r\n\u003cdiv align=center\u003e\r\n\u003ccenter style=\"color:#000000;text-decoration:underline\"\u003eFigure.1 Example views of robots in Safe MAMuJoCo. Body parts of different colours are controlled by differentagents. Agents jointly learn to manipulate the robot, while avoiding crashing into unsafe areas. \u003c/center\u003e\r\n \u003c/div\u003e\r\n--\u003e\r\n \r\n \r\n \u003cdiv align=center\u003e\r\n \u003cimg src=\"https://github.com/chauncygu/Safe-Multi-Agent-Mujoco/blob/main/figures/all-mujoco.png\" width=\"850\"/\u003e \r\n \u003c/div\u003e\r\n\u003cdiv align=center\u003e\r\n\u003ccenter style=\"color:#000000;text-decoration:underline\"\u003eFigure.1 Example views of robots in Safe MAMuJoCo. Body parts of different colours are controlled by differentagents. Agents jointly learn to manipulate the robot, while avoiding crashing into unsafe areas. \u003c/center\u003e\r\n \u003c/div\u003e\r\n \r\n\r\n## Demos\r\n\r\n\u003c!--\u003cdiv align=center\u003e\r\n\u003cimg src=\"https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/202111071600-unsafe-end%2000_00_00-00_00_30.gif\" width=\"700\"/\u003e    \r\n\u003c/div\u003e\r\n    \r\n\u003cdiv align=center\u003e\r\n\u003ccenter style=\"color:#000000;text-decoration:underline\"\u003e\r\n    A demo denotes \u003cb\u003eunsafe\u003c/b\u003e performance using \u003ca href=\"https://arxiv.org/abs/2109.11251\"\u003eHAPPO\u003c/a\u003e on Ant2x4 task.\u003c/center\u003e\r\n\u003c/div\u003e\r\n\r\n\u0026nbsp;\r\n\u003cdiv align=center\u003e\r\n\u003cimg src=\"https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/202111140948-safe-end1%2000_00_00-00_00_30.gif\" width=\"700\"/\u003e    \r\n\u003c/div\u003e\r\n    \r\n\u003cdiv align=center\u003e\r\n\u003ccenter style=\"color:#000000;text-decoration:underline\"\u003e\r\n    A demo denotes \u003cb\u003esafe\u003c/b\u003e performance using \u003ca href=\"http://arxiv.org/abs/2110.02793\"\u003eMAPPO-Lagrangian\u003c/a\u003e on Ant2x4 task.\u003c/center\u003e\r\n\u003c/div\u003e--\u003e\r\n\r\n**Ant Task**: the width of the corridor set by two walls is 10 m. The environment emits the cost of 1 for an agent, if the distance between the robot and the wall is less than 1.8 m, or when the robot topples over.\r\n\r\n|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/202111071600-unsafe-end%2000_00_00-00_00_30.gif)|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/202111140948-safe-end1%2000_00_00-00_00_30.gif)|\r\n| :---: | :---: | \r\n|  A demo denotes \u003cb\u003eunsafe\u003c/b\u003e performance using \u003ca href=\"https://arxiv.org/abs/2109.11251\"\u003eHAPPO\u003c/a\u003e on Ant-2x4 task.\u003c/center\u003e | A demo denotes \u003cb\u003esafe\u003c/b\u003e performance using \u003ca href=\"http://arxiv.org/abs/2110.02793\"\u003eMAPPO-Lagrangian\u003c/a\u003e on Ant-2x4 task.\u003c/center\u003e | \r\n\r\n\r\n**HalfCheetah Task**: In the task, the agents move inside a corridor (which constraints their movement, but does not induce costs). Together with them, there are bombs moving inside the corridor. If an agent finds itself too close to the bomb, the distance between an agent and the bomb is less than 9m, a cost of 1 will be emitted, at the same time, the bomb will turn blood red.\r\n\r\n|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/202111140948-halfcheetah-unsafe-end%2000_00_00-00_00_30.gif)|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/202111140948-halfcheetah-safe-end%2000_00_00-00_00_30.gif)|\r\n| :---: | :---: | \r\n|  A demo denotes \u003cb\u003eunsafe\u003c/b\u003e performance using \u003ca href=\"https://arxiv.org/abs/2109.11251\"\u003eHAPPO\u003c/a\u003e on HalfCheetah-2x3 task.\u003c/center\u003e | A demo denotes \u003cb\u003esafe\u003c/b\u003e performance using \u003ca href=\"http://arxiv.org/abs/2110.02793\"\u003eMAPPO-Lagrangian\u003c/a\u003e on HalfCheetah-2x3 task.\u003c/center\u003e | \r\n\r\n\r\n**ManyAgent Ant Task One**: In the ManyAgent Ant task, the width of the corridor set by two walls is 9m. The environment emits the cost of 1 for an agent, if the distance between the robot and the wall is less than 1.8 m, or when the robot topples over. \r\n\r\n|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/sadppo-manyagent-ant--unsafe-end-have-word%2000_00_00-00_00_30.gif)|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/safe-mappo-manyagent-ant--safe-end-have-word-01%2000_00_00-00_00_30.gif)|\r\n| :---: | :---: | \r\n|  A demo denotes \u003cb\u003eunsafe\u003c/b\u003e performance using \u003ca href=\"https://arxiv.org/abs/2109.11251\"\u003eHAPPO\u003c/a\u003e on ManyAgent Ant-2x3 task.\u003c/center\u003e | A demo denotes \u003cb\u003esafe\u003c/b\u003e performance using \u003ca href=\"http://arxiv.org/abs/2110.02793\"\u003eMAPPO-Lagrangian\u003c/a\u003e on ManyAgent Ant-2x3 task.\u003c/center\u003e | \r\n\r\n\r\n**ManyAgent Ant Task Two**: In the ManyAgent Ant task, the width of the corridor is 12 m; its walls fold at the angle of 30 degrees. The environment emits the cost of 1 for an agent, if the distance between the robot and the wall is less than 1.8 m, or when the robot topples over.\r\n\r\n|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/two-wall-sadppo-unsafe-end-have-word-manyagent-ant%2000_00_00-00_00_30.gif)|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/two-wall-safe-mappo--manyagent-ant-safe-end-have-word-manyagent-ant%2000_00_00-00_00_30.gif)|![](https://github.com/chauncygu/Multi-Agent-Constrained-Policy-Optimisation/blob/main/figures/two-wall-macpo-manyagent-ant--safe-end-have-word%2000_00_00-00_00_30.gif)|\r\n| :---: | :---: | :---: | \r\n|  A demo denotes \u003cb\u003eunsafe\u003c/b\u003e performance using \u003ca href=\"https://arxiv.org/abs/2109.11251\"\u003eHAPPO\u003c/a\u003e on ManyAgent Ant-2x3 task.\u003c/center\u003e | A demo denotes \u003cb\u003eunsafe\u003c/b\u003e performance using \u003ca href=\"http://arxiv.org/abs/2110.02793\"\u003eMAPPO-Lagrangian\u003c/a\u003e on ManyAgent Ant-2x3 task.\u003c/center\u003e | A demo denotes \u003cb\u003esafe\u003c/b\u003e performance using \u003ca href=\"http://arxiv.org/abs/2110.02793\"\u003eMACPO\u003c/a\u003e on ManyAgent Ant-2x3 task.\u003c/center\u003e | \r\n\r\n\r\n## Installation\r\n\r\n- Install mujoco accoring to [mujoco-py](https://github.com/openai/mujoco-py) and [MuJoCo website](https://www.roboti.us/license.html).\r\n- clone safety multi-agent mujoco to the env path.\r\n\u0026nbsp;\r\n\r\n``` Bash\r\nLD_LIBRARY_PATH=${HOME}/.mujoco/mujoco200/bin;\r\nLD_PRELOAD=/usr/lib/x86_64-linux-gnu/libGLEW.so\r\npip install -r requirements.txt\r\n```\r\n\r\n## Test\r\nRun test.py\r\n\r\n``` Bash\r\npython test.py\r\n```\r\n\r\n## Tasks\r\n``` Bash\r\n    ManyAgent Ant\r\n    env_args = {\"scenario\": \"manyagent_ant\",\r\n                  \"agent_conf\": \"3x2\",\r\n                  \"agent_obsk\": 1,\r\n                  \"episode_limit\": 1000}\r\n\r\n    env_args = {\"scenario\": \"manyagent_ant\",\r\n                  \"agent_conf\": \"2x3\",\r\n                  \"agent_obsk\": 1,\r\n                  \"episode_limit\": 1000}\r\n                  \r\n    env_args = {\"scenario\": \"manyagent_ant\",\r\n                  \"agent_conf\": \"6x1\",\r\n                  \"agent_obsk\": 1,\r\n                  \"episode_limit\": 1000}\r\n\r\n    env_args = {\"scenario\": \"manyagent_ant\",\r\n                \"agent_conf\": \"4x2\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n                \r\n    env_args = {\"scenario\": \"manyagent_ant\",\r\n                \"agent_conf\": \"2x4\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n                \r\n                \r\n    env_args = {\"scenario\": \"manyagent_ant\",\r\n                \"agent_conf\": \"8x1\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n                \r\n    HalfCheetah\r\n    env_args = {\"scenario\": \"HalfCheetah-v2\",\r\n                \"agent_conf\": \"2x3\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n                \r\n    env_args = {\"scenario\": \"HalfCheetah-v2\",\r\n                \"agent_conf\": \"3x2\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n                \r\n    env_args = {\"scenario\": \"HalfCheetah-v2\",\r\n                \"agent_conf\": \"6x1\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n                  \r\n    Ant \r\n    env_args = {\"scenario\": \"Ant-v2\",\r\n                \"agent_conf\": \"2x4\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n\r\n    env_args = {\"scenario\": \"Ant-v2\",\r\n                \"agent_conf\": \"8x1\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n\r\n    env_args = {\"scenario\": \"Ant-v2\",\r\n                \"agent_conf\": \"2x4d\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n\r\n    env_args = {\"scenario\": \"Ant-v2\",\r\n                \"agent_conf\": \"4x2\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n                \r\n   Coupled_half_cheetah\r\n    env_args = {\"scenario\": \"coupled_half_cheetah\",\r\n                \"agent_conf\": \"1p1\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n    Hopper\r\n    env_args = {\"scenario\": \"Hopper-v2\",\r\n                \"agent_conf\": \"3x1\",\r\n                \"agent_obsk\": 1,\r\n                \"episode_limit\": 1000}\r\n```\r\n\r\n\r\n## Publication\r\nIf you find the repository useful, please cite the [paper](https://arxiv.org/abs/2110.02793):\r\n```\r\n@article{gu2023safe,\r\n  title={Safe Multi-Agent Reinforcement Learning for Multi-Robot Control},\r\n  author={Gu, Shangding and Kuba, Jakub Grudzien and Chen, Yuanpei and Du, Yali and Yang, Long and Knoll, Alois and Yang, Yaodong},\r\n  journal={Artificial Intelligence},\r\n  pages={103905},\r\n  year={2023},\r\n  publisher={Elsevier}\r\n}\r\n\r\n```\r\n\r\n# Acknowledgments\r\n\r\nWe thank the list of contributors from the following open source repositories: [MAMujoco](https://github.com/schroederdewitt/multiagent_mujoco), [safety-gym](https://github.com/openai/safety-gym), [CMBPO](https://github.com/anyboby/Constrained-Model-Based-Policy-Optimization).\r\n\r\n\r\n\r\n\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchauncygu%2FSafe-Multi-Agent-Mujoco","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchauncygu%2FSafe-Multi-Agent-Mujoco","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchauncygu%2FSafe-Multi-Agent-Mujoco/lists"}