{"id":16271683,"url":"https://github.com/jianzhnie/deep-marl-toolkit","last_synced_at":"2025-10-14T06:09:17.615Z","repository":{"id":103457654,"uuid":"522404548","full_name":"jianzhnie/deep-marl-toolkit","owner":"jianzhnie","description":"MARLToolkit: The Multi-Agent Rainforcement Learning Toolkit. Include implementation of MAPPO, MADDPG, QMIX, VDN, COMA, IPPO, QTRAN, MAT...","archived":false,"fork":false,"pushed_at":"2024-04-19T14:50:01.000Z","size":856,"stargazers_count":142,"open_issues_count":5,"forks_count":19,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-09-06T13:41:51.741Z","etag":null,"topics":["coma","ia2c","idqn","ippo","maddpg","mappo","mat","qatten","qmix","qtran","vdn"],"latest_commit_sha":null,"homepage":"https://jianzhnie.github.io/llmtech/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jianzhnie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2022-08-08T04:29:57.000Z","updated_at":"2025-08-10T14:40:11.000Z","dependencies_parsed_at":null,"dependency_job_id":"d28631d4-5215-4a5e-9972-cdca41cbc663","html_url":"https://github.com/jianzhnie/deep-marl-toolkit","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jianzhnie/deep-marl-toolkit","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-marl-toolkit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-marl-toolkit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-marl-toolkit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-marl-toolkit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jianzhnie","download_url":"https://codeload.github.com/jianzhnie/deep-marl-toolkit/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jianzhnie%2Fdeep-marl-toolkit/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279018122,"owners_count":26086280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-14T02:00:06.444Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coma","ia2c","idqn","ippo","maddpg","mappo","mat","qatten","qmix","qtran","vdn"],"created_at":"2024-10-10T18:14:24.664Z","updated_at":"2025-10-14T06:09:17.597Z","avatar_url":"https://github.com/jianzhnie.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch2 align=\"center\"\u003e MARLToolkit: The  Multi Agent Rainforcement Learning Toolkit\u003c/h2\u003e\n\n**MARLToolkit** is a *Multi-Agent Reinforcement Learning Toolkit* based on **Pytorch**.\nIt provides MARL research community a unified platform for developing and evaluating the new ideas in various multi-agent environments.\nThere are four core features of **MARLToolkit**.\n\n- it collects most of the existing MARL algorithms widely acknowledged by the community and unifies them under one framework.\n- it gives a solution that enables different multi-agent environments using the same interface to interact with the agents.\n- it guarantees excellent efficiency in both the training and sampling process.\n- it provides trained results, including learning curves and pretrained models specific to each task and algorithm's combination, with finetuned hyper-parameters to guarantee credibility.\n\n## Overview\n\nWe collected most of the existing multi-agent environment and multi-agent reinforcement learning algorithms and unified them under one framework based on \\[**Pytorch**\\] to boost the MARL research.\n\nThe MARL baselines include **independence learning (IQL, A2C, DDPG, TRPO, PPO)**, **centralized critic learning (COMA, MADDPG, MAPPO, HATRPO)**, and **value decomposition (QMIX, VDN, FACMAC, VDA2C)** are all implemented.\n\nPopular environments like **SMAC, MaMujoco, and Google Research Football** are provided with a unified interface.\n\nThe algorithm code and environment code are fully separated. Changing the environment needs no modification on the algorithm side and vice versa.\n\n|                              Benchmark                               |                                                                    Github Stars                                                                     | Learning Mode | Available Env | Algorithm Type | Algorithm Number | Continues Control  | Asynchronous Interact | Distributed Training |                                       Framework                                        |                                                     Last Update                                                     |\n| :------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------------------------------------------------: | :-----------: | :-----------: | :------------: | :--------------: | :----------------: | :-------------------: | :------------------: | :------------------------------------------------------------------------------------: | :-----------------------------------------------------------------------------------------------------------------: |\n|             [PyMARL](https://github.com/oxwhirl/pymarl)              |                 [![GitHub stars](https://img.shields.io/github/stars/oxwhirl/pymarl)](https://github.com/oxwhirl/pymarl/stargazers)                 |      CP       |       1       |       VD       |        5         |                    |                       |                      |                                           \\*                                           |         ![GitHub last commit](https://img.shields.io/github/last-commit/oxwhirl/pymarl?label=last%20update)         |\n|            [PyMARL2](https://github.com/hijkzzz/pymarl2)             |                [![GitHub stars](https://img.shields.io/github/stars/hijkzzz/pymarl2)](https://github.com/hijkzzz/pymarl2/stargazers)                |      CP       |       1       |       VD       |        12        |                    |                       |                      |                      [PyMARL](https://github.com/oxwhirl/pymarl)                       |        ![GitHub last commit](https://img.shields.io/github/last-commit/hijkzzz/pymarl2?label=last%20update)         |\n|      [off-policy](https://github.com/marlbenchmark/off-policy)       |                                    ![GitHub stars](https://img.shields.io/github/stars/marlbenchmark/off-policy)                                    |      CP       |       4       |    IL+VD+CC    |        4         |                    |                       |                      |               [off-policy](https://github.com/marlbenchmark/off-policy)                |    ![GitHub last commit](https://img.shields.io/github/last-commit/marlbenchmark/off-policy?label=last%20update)    |\n|       [on-policy](https://github.com/marlbenchmark/on-policy)        |                                    ![GitHub stars](https://img.shields.io/github/stars/marlbenchmark/on-policy)                                     |      CP       |       4       |    IL+VD+CC    |        1         |                    |                       |                      |                [on-policy](https://github.com/marlbenchmark/on-policy)                 |    ![GitHub last commit](https://img.shields.io/github/last-commit/marlbenchmark/on-policy?label=last%20update)     |\n| [MARL-Algorithms](https://github.com/starry-sky6688/MARL-Algorithms) | [![GitHub stars](https://img.shields.io/github/stars/starry-sky6688/MARL-Algorithms)](https://github.com/starry-sky6688/MARL-Algorithms/stargazers) |      CP       |       1       |    VD+Comm     |        9         |                    |                       |                      |                                           \\*                                           | ![GitHub last commit](https://img.shields.io/github/last-commit/starry-sky6688/MARL-Algorithms?label=last%20update) |\n|           [EPyMARL](https://github.com/uoe-agents/epymarl)           |         [![GitHub stars](https://img.shields.io/github/stars/uoe-agents/epymarl)](https://github.com/hijkzzz/uoe-agents/epymarl/stargazers)         |      CP       |       4       |    IL+VD+CC    |        10        |                    |                       |                      |                      [PyMARL](https://github.com/oxwhirl/pymarl)                       |       ![GitHub last commit](https://img.shields.io/github/last-commit/uoe-agents/epymarl?label=last%20update)       |\n|     [Marlbenchmark](https://github.com/marlbenchmark/on-policy)      |        [![GitHub stars](https://img.shields.io/github/stars/marlbenchmark/on-policy)](https://github.com/marlbenchmark/on-policy/stargazers)        |     CP+CL     |       4       |     VD+CC      |        5         | :heavy_check_mark: |                       |                      | [pytorch-a2c-ppo-acktr-gail](https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail) |    ![GitHub last commit](https://img.shields.io/github/last-commit/marlbenchmark/on-policy?label=last%20update)     |\n|             [MAlib](https://github.com/sjtu-marl/malib)              |            [![GitHub stars](https://img.shields.io/github/stars/sjtu-marl/malib)](https://github.com/hijkzzz/sjtu-marl/malib/stargazers)            |      SP       |       8       |       SP       |        9         | :heavy_check_mark: |                       |                      |                                           \\*                                           |        ![GitHub last commit](https://img.shields.io/github/last-commit/sjtu-marl/malib?label=last%20update)         |\n|        [MARLlib](https://github.com/Replicable-MARL/MARLlib)         |        [![GitHub stars](https://img.shields.io/github/stars/Replicable-MARL/MARLlib)](https://github.com/Replicable-MARL/MARLlib/stargazers)        |  CP+CL+CM+MI  |      10       |    IL+VD+CC    |        18        | :heavy_check_mark: |  :heavy_check_mark:   |  :heavy_check_mark:  |                  [Ray/RLlib](https://docs.ray.io/en/releases-1.8.0/)                   |    ![GitHub last commit](https://img.shields.io/github/last-commit/Replicable-MARL/MARLlib?label=last%20update)     |\n|                                                                      |                                                                                                                                                     |               |               |                |                  |                    |                       |                      |                                                                                        |                                                                                                                     |\n\nCP, CL, CM, and MI represent cooperative, collaborative, competitive, and mixed task learning modes.\nIL, VD, and CC represent independent learning, value decomposition, and centralized critic categorization. SP represents self-play.\nComm represents communication-based learning.\nAsterisk denotes that the benchmark uses its framework.\n\n## Environment\n\n#### Supported Multi-agent Environments / Tasks\n\nMost of the popular environment in MARL research has been incorporated in this benchmark:\n\n| Env Name                                                          | Learning Mode | Observability | Action Space | Observations |\n| ----------------------------------------------------------------- | ------------- | ------------- | ------------ | ------------ |\n| [LBF](https://github.com/semitable/lb-foraging)                   | Mixed         | Both          | Discrete     | Discrete     |\n| [RWARE](https://github.com/semitable/robotic-warehouse)           | Collaborative | Partial       | Discrete     | Discrete     |\n| [MPE](https://github.com/openai/multiagent-particle-envs)         | Mixed         | Both          | Both         | Continuous   |\n| [SMAC](https://github.com/oxwhirl/smac)                           | Cooperative   | Partial       | Discrete     | Continuous   |\n| [MetaDrive](https://github.com/decisionforce/metadrive)           | Collaborative | Partial       | Continuous   | Continuous   |\n| [MAgent](https://www.pettingzoo.ml/magent)                        | Mixed         | Partial       | Discrete     | Discrete     |\n| [Pommerman](https://github.com/MultiAgentLearning/playground)     | Mixed         | Both          | Discrete     | Discrete     |\n| [MaMujoco](https://github.com/schroederdewitt/multiagent_mujoco)  | Cooperative   | Partial       | Continuous   | Continuous   |\n| [GRF](https://github.com/google-research/football)                | Collaborative | Full          | Discrete     | Continuous   |\n| [Hanabi](https://github.com/deepmind/hanabi-learning-environment) | Cooperative   | Partial       | Discrete     | Discrete     |\n\nEach environment has a readme file, standing as the instruction for this task, talking about env settings, installation, and some important notes.\n\n## Algorithm\n\nWe provide three types of MARL algorithms as our baselines including:\n\n**Independent Learning:**\nIQL\nDDPG\nPG\nA2C\nTRPO\nPPO\n\n**Centralized Critic:**\nCOMA\nMADDPG\nMAAC\nMAPPO\nMATRPO\nHATRPO\nHAPPO\n\n**Value Decomposition:**\nVDN\nQMIX\nFACMAC\nVDAC\nVDPPO\n\nHere is a chart describing the characteristics of each algorithm:\n\n| Algorithm                                                                                                                  | Support Task Mode | Need Global State | Action     | Learning Mode        | Type       |\n| -------------------------------------------------------------------------------------------------------------------------- | ----------------- | ----------------- | ---------- | -------------------- | ---------- |\n| IQL                                                                                                                        | Mixed             | No                | Discrete   | Independent Learning | Off Policy |\n| [PG](https://papers.nips.cc/paper/1713-policy-gradient-methods-for-reinforcement-learning-with-function-approximation.pdf) | Mixed             | No                | Both       | Independent Learning | On Policy  |\n| [A2C](https://arxiv.org/abs/1602.01783)                                                                                    | Mixed             | No                | Both       | Independent Learning | On Policy  |\n| [DDPG](https://arxiv.org/abs/1509.02971)                                                                                   | Mixed             | No                | Continuous | Independent Learning | Off Policy |\n| [TRPO](http://proceedings.mlr.press/v37/schulman15.pdf)                                                                    | Mixed             | No                | Both       | Independent Learning | On Policy  |\n| [PPO](https://arxiv.org/abs/1707.06347)                                                                                    | Mixed             | No                | Both       | Independent Learning | On Policy  |\n| [COMA](https://ojs.aaai.org/index.php/AAAI/article/download/11794/11653)                                                   | Mixed             | Yes               | Both       | Centralized Critic   | On Policy  |\n| [MADDPG](https://arxiv.org/abs/1706.02275)                                                                                 | Mixed             | Yes               | Continuous | Centralized Critic   | Off Policy |\n| MAA2C                                                                                                                      | Mixed             | Yes               | Both       | Centralized Critic   | On Policy  |\n| MATRPO                                                                                                                     | Mixed             | Yes               | Both       | Centralized Critic   | On Policy  |\n| [MAPPO](https://arxiv.org/abs/2103.01955)                                                                                  | Mixed             | Yes               | Both       | Centralized Critic   | On Policy  |\n| [HATRPO](https://arxiv.org/abs/2109.11251)                                                                                 | Cooperative       | Yes               | Both       | Centralized Critic   | On Policy  |\n| [HAPPO](https://arxiv.org/abs/2109.11251)                                                                                  | Cooperative       | Yes               | Both       | Centralized Critic   | On Policy  |\n| [VDN](https://arxiv.org/abs/1706.05296)                                                                                    | Cooperative       | No                | Discrete   | Value Decomposition  | Off Policy |\n| [QMIX](https://arxiv.org/abs/1803.11485)                                                                                   | Cooperative       | Yes               | Discrete   | Value Decomposition  | Off Policy |\n| [FACMAC](https://arxiv.org/abs/2003.06709)                                                                                 | Cooperative       | Yes               | Continuous | Value Decomposition  | Off Policy |\n| [VDAC](https://arxiv.org/abs/2007.12306)                                                                                   | Cooperative       | Yes               | Both       | Value Decomposition  | On Policy  |\n| VDPPO\\*                                                                                                                    | Cooperative       | Yes               | Both       | Value Decomposition  | On Policy  |\n\n*IQL* is the multi-agent version of Q learning.\n*MAA2C* and *MATRPO* are the centralized version of A2C and TRPO.\n*VDPPO* is the value decomposition version of PPO.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjianzhnie%2Fdeep-marl-toolkit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjianzhnie%2Fdeep-marl-toolkit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjianzhnie%2Fdeep-marl-toolkit/lists"}