https://github.com/proroklab/hetgppo
Heterogeneous Multi-Robot Reinforcement Learning
https://github.com/proroklab/hetgppo
heterogeneity multi-agent-reinforcement-learning multi-agent-systems robotics
Last synced: 16 days ago
JSON representation
Heterogeneous Multi-Robot Reinforcement Learning
- Host: GitHub
- URL: https://github.com/proroklab/hetgppo
- Owner: proroklab
- Created: 2022-09-13T22:59:36.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-09-10T07:05:16.000Z (about 1 year ago)
- Last Synced: 2024-09-10T09:22:06.331Z (about 1 year ago)
- Topics: heterogeneity, multi-agent-reinforcement-learning, multi-agent-systems, robotics
- Language: Python
- Homepage: https://arxiv.org/abs/2301.07137
- Size: 112 KB
- Stars: 31
- Watchers: 5
- Forks: 13
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# HetGPPO & SND
This repository contains the code for the papers:
- [Heterogeneous Multi-Agent Reinforcement Learning](https://arxiv.org/abs/2301.07137)
- [System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learning](https://arxiv.org/abs/2305.02128)
### Cite
If you use HetGPPO in your research, **cite** it using:
```
@inproceedings{bettini2023hetgppo,
title = {Heterogeneous Multi-Robot Reinforcement Learning},
author = {Bettini, Matteo and Shankar, Ajay and Prorok, Amanda},
year = {2023},
booktitle = {Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems},
publisher = {International Foundation for Autonomous Agents and Multiagent Systems},
series = {AAMAS '23}
}
```
If you use SND in your research, **cite** it using:
```
@article{bettini2023snd,
title={System Neural Diversity: Measuring Behavioral Heterogeneity in Multi-Agent Learning},
author={Bettini, Matteo and Shankar, Ajay and Prorok, Amanda},
journal={arXiv preprint arXiv:2305.02128},
year={2023}
}
```
### Videos
Watch the presentation video of HetGPPO.
[](https://www.youtube.com/watch?v=J81IVQEy-zw)
Watch the talk about HetGPPO.
[](https://youtu.be/a4md0es3kuo)
### How to use
#### Install
Clone the repository using
```bash
git clone --recursive https://github.com/proroklab/HetGPPO.git
cd HetGPPO
```
Create a conda environment and install the dependencies
```bash
pip install "ray[rllib]"==2.1.0
pip install -r requirements.txt
```
#### Run
The training scripts to run HetGPPO in the various [VMAS](https://github.com/proroklab/VectorizedMultiAgentSimulator) scenarios can be found in the `train` folder.
These scripts use the HetGPPO model in `models/gppo.py` to train multiple agents in different scenarios. The scripts log to wandb.
You can run them with:
```bash
python train/train_flocking.py
```
The hyperparameters for each script can be changed according to the user needs.
Several util tools can be found in `utils.py`, including the callback to compute the SND heterogeneity metric.
Several evaluation tools can be found in the `evaluate` folder.