https://github.com/christianlin0420/universalmarl
This project mainly focus on how to transfer agents' knowledge in order to improve the flexibility of the MARL model and the training efficiency.
https://github.com/christianlin0420/universalmarl
multi-agent-reinforcement-learning reinforcement-learning transfer-learning
Last synced: 5 months ago
JSON representation
This project mainly focus on how to transfer agents' knowledge in order to improve the flexibility of the MARL model and the training efficiency.
- Host: GitHub
- URL: https://github.com/christianlin0420/universalmarl
- Owner: ChristianLin0420
- License: mit
- Created: 2022-04-03T03:27:54.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2025-01-13T02:42:31.000Z (over 1 year ago)
- Last Synced: 2025-10-08T19:55:39.974Z (10 months ago)
- Topics: multi-agent-reinforcement-learning, reinforcement-learning, transfer-learning
- Language: Python
- Homepage:
- Size: 131 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# universalMARL
# Python MARL framework
universalMARL is [WhiRL](http://whirl.cs.ox.ac.uk)'s framework for deep multi-agent reinforcement learning and includes implementations of the following algorithms:
- [**QMIX**: QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://arxiv.org/abs/1803.11485)
- [**COMA**: Counterfactual Multi-Agent Policy Gradients](https://arxiv.org/abs/1705.08926)
- [**VDN**: Value-Decomposition Networks For Cooperative Multi-Agent Learning](https://arxiv.org/abs/1706.05296)
- [**IQL**: Independent Q-Learning](https://arxiv.org/abs/1511.08779)
- [**QTRAN**: QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learning](https://arxiv.org/abs/1905.05408)
PyMARL is written in PyTorch and uses [SMAC](https://github.com/oxwhirl/smac) as its environment.
## Installation instructions
Set up StarCraft II and SMAC:
```shell
bash install_sc2.sh
```
This will download SC2 into the 3rdparty folder and copy the maps necessary to run over.
The requirements.txt file can be used to install the necessary packages into a virtual environment (not recomended).
## Run an experiment
```shell
python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z
```
The config files act as defaults for an algorithm or environment.
They are all located in `src/config`.
`--config` refers to the config files in `src/config/algs`
`--env-config` refers to the config files in `src/config/envs`
To run experiments using the Docker container:
```shell
bash run.sh $GPU python3 src/main.py --config=qmix --env-config=sc2 with env_args.map_name=2s3z
```
All results will be stored in the `Results` folder.
The previous config files used for the SMAC Beta have the suffix `_beta`.
## Saving and loading learnt models
### Saving models
You can save the learnt models to disk by setting `save_model = True`, which is set to `False` by default. The frequency of saving models can be adjusted using `save_model_interval` configuration. Models will be saved in the result directory, under the folder called *models*. The directory corresponding each run will contain models saved throughout the experiment, each within a folder corresponding to the number of timesteps passed since starting the learning process.
### Loading models
Learnt models can be loaded using the `checkpoint_path` parameter, after which the learning will proceed from the corresponding timestep.
## Watching StarCraft II replays
`save_replay` option allows saving replays of models which are loaded using `checkpoint_path`. Once the model is successfully loaded, `test_nepisode` number of episodes are run on the test mode and a .SC2Replay file is saved in the Replay directory of StarCraft II. Please make sure to use the episode runner if you wish to save a replay, i.e., `runner=episode`. The name of the saved replay file starts with the given `env_args.save_replay_prefix` (map_name if empty), followed by the current timestamp.
The saved replays can be watched by double-clicking on them or using the following command:
```shell
python -m pysc2.bin.play --norender --rgb_minimap_size 0 --replay NAME.SC2Replay
```
**Note:** Replays cannot be watched using the Linux version of StarCraft II. Please use either the Mac or Windows version of the StarCraft II client.
## Citing universalMARL
If you use universalMARL in your research, please cite the [universalMARL paper].