{"id":48292876,"url":"https://github.com/jaggbow/saf","last_synced_at":"2026-04-04T23:18:30.564Z","repository":{"id":76993585,"uuid":"499151733","full_name":"jaggbow/saf","owner":"jaggbow","description":"This repository contains code for the paper \"Stateful Active Facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning\". https://arxiv.org/abs/2210.03022 ","archived":false,"fork":false,"pushed_at":"2023-04-27T23:49:12.000Z","size":100215,"stargazers_count":5,"open_issues_count":0,"forks_count":5,"subscribers_count":2,"default_branch":"main","last_synced_at":"2024-04-20T20:21:45.226Z","etag":null,"topics":["coordination","multi-agent-reinforcement-learning","reinforcement-learning","reinforcement-learning-environments"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jaggbow.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-06-02T13:42:55.000Z","updated_at":"2024-04-20T20:21:45.226Z","dependencies_parsed_at":"2023-09-28T07:18:20.793Z","dependency_job_id":null,"html_url":"https://github.com/jaggbow/saf","commit_stats":{"total_commits":84,"total_committers":6,"mean_commits":14.0,"dds":0.5238095238095238,"last_synced_commit":"5c388a5cc43dbe2266d7cd01b2be42321f4c5130"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/jaggbow/saf","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaggbow%2Fsaf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaggbow%2Fsaf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaggbow%2Fsaf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaggbow%2Fsaf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jaggbow","download_url":"https://codeload.github.com/jaggbow/saf/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jaggbow%2Fsaf/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31418290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T20:09:54.854Z","status":"ssl_error","status_checked_at":"2026-04-04T20:09:44.350Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["coordination","multi-agent-reinforcement-learning","reinforcement-learning","reinforcement-learning-environments"],"created_at":"2026-04-04T23:18:30.028Z","updated_at":"2026-04-04T23:18:30.559Z","avatar_url":"https://github.com/jaggbow.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Stateful Active Facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning\n\nThis is the official repository to the paper [\"Stateful Active Facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning\"](https://arxiv.org/abs/2210.03022)\n\nIn this paper, we tackle the coordination and environmental heterogeneity characteristics that are present in real-life scenarios. To be able to understand how existing MARL algorithms fare in environments with high levels of coordination and/or environmental heterogeneity, we introduce a suite of environments called **HECOGrid**, where users can manually tune the level of coordination and environmental heterogeneity in the provided environments.\n\nTo tackle the difficulty in learning that comes with high levels of coordination and/or environmental heterogeneity, we introduce a new model: the **Stateful Active Facilitator** which has a differentiable communication channel that allows agents to efficiently communicate during training to improve coordination, as well as a pool of policies that they can choose from in order to be resilient to increasing levels of environmental heterogeneity\n![Stateful Active Faciliator](/assets/saf.jpg \"Stateful Active Faciliator\")\n![HECOGrid](/assets/hecogrid.jpg \"HECOGrid\")\n\n# Setup\nStart by installing the required modules:\n```\npip install -r requirements.txt\n```\nNext, install the marlgrid environment by executing the following lines:\n```\ncd src/envs/marlgrid_env/\npip install -e .\n```\n\n# HECOGrid Environment\nThe code and installation guide for HECOGrid can be found [here](https://github.com/veds12/hecogrid).\n\n# Experiments\nYou can find scripts to run experiments in the folder ``scripts``. It is structured as follows:\n```\nscripts/\n├── jobs_master_keyfortreasure.sh\n├── jobs_master_OOD.sh\n├── jobs_master_teamsupportenv.sh\n├── jobs_master_teamtogetherenv.sh\n├── OOD_tests\n│   ├── OOD_coordination_test_compoundenv.sh\n│   ├── OOD_coordination_test.sh\n│   ├── OOD_heco_test.sh\n│   ├── OOD_heterogeneity_test_compoundenv.sh\n│   ├── OOD_heterogeneity_test.sh\n│   ├── OOD_num_agents_test.sh\n│   ├── OOD_num_treasures_test.sh\n│   └── OOD_size_env_test.sh\n├── testing.sh\n└── training\n    ├── baselines.sh\n    └── saf.sh\n```\nIn order to reproduce experimental results in the paper, you can run the appropriate ``jobs_master`` for the desired environment. For example, for the ``KeyForTreasure``, that would be ``jobs_master_keyfortreasure.sh``.\n\nIf you want to run our algorithm on your environments, you can check the structure of ``saf.sh`` in ``scripts/training`` folder.\n# Comet Configuration\n[comet.ml](https://www.comet.com/site/) is a great tool for tracking and logging experiments as well as running hyperparameter sweeps.\n\nIn order to get started, make sure you [create an account](https://www.comet.com/signup) on their website (you can sign up using your github account!). Once that is done, you'll receive your ``API_KEY``.\n\nNext, install ``comet`` using the following command:\n\n```\npip install comet-ml\n```\n\nNext, go to your ``$HOME`` folder an create a file named ``.comet.config`` as follows (this works on Linux):\n\n```\ntouch .comet.config\n```\nNext, open your config file and start editing:\n\n```\nnano .comet.config\n```\n\nFinally, copy-paste the following:\n\n```\n[comet]\napi_key=API_KEY\nworkspace=WORKSPACE\n```\n\nNow, you can kick-start a comet.ml experiment as follows:\n\n```\nfrom comet_ml import Experiment\n\nexperiment = Experiment(project_name=\"pytorch\") # No need to explicitly provide the API_KEY as .comet.config has it already\n```\n\nFor more information, you can check the [documentation](https://www.comet.com/docs/python-sdk/pytorch/).\n\n# Citation\n\nTo cite this project, please use:\n\n```\n@article{Liu2022StatefulAF,\n  title={Stateful active facilitator: Coordination and Environmental Heterogeneity in Cooperative Multi-Agent Reinforcement Learning},\n  author={Dianbo Liu and Vedant Shah and Oussama Boussif and Cristian Meo and Anirudh Goyal and Tianmin Shu and Michael Curtis Mozer and Nicolas Manfred Otto Heess and Yoshua Bengio},\n  journal={ArXiv},\n  year={2022},\n  volume={abs/2210.03022}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaggbow%2Fsaf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaggbow%2Fsaf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaggbow%2Fsaf/lists"}