https://github.com/douglasrizzo/hcanet

Neural Network for Heterogemous Communicative Agents
https://github.com/douglasrizzo/hcanet

Last synced: 10 days ago
JSON representation

Neural Network for Heterogemous Communicative Agents

• Host: GitHub
• URL: https://github.com/douglasrizzo/hcanet
• Owner: douglasrizzo
• License: apache-2.0
• Created: 2021-06-07T11:19:45.000Z (over 3 years ago)
• Default Branch: master
• Last Pushed: 2021-12-07T00:04:16.000Z (about 3 years ago)
• Last Synced: 2024-12-17T19:28:30.418Z (23 days ago)
• Language: Python
• Size: 142 KB
• Stars: 2
• Watchers: 3
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# HCA-Net/HMAG-QNet

This repository contains one of the projects I did during my PhD in Artificial Intelligence Applied to Automation and Robotics.  The idea was to propose a neural network architecture for cooperative multi-agent systems with heterogeneous agents, in which agents could be grouped into classes of homogeneous agents and policies as well as communication protocols could be learned faster by doing so.

- HMAG-QNet stands for __Heterogeneous Multi-Agent Graph Q-Network__. Paper and code information is available in the [v8 tag](https://github.com/douglasrizzo/hcanet/releases/tag/v8).

- HCA-Net stands for __Neural Network for Heterogeneous Communicative Agents__. Paper and code information is available in the [v9 tag](https://github.com/douglasrizzo/hcanet/releases/tag/v9).

To configure an environment, use `conda env create` in the root folder of the repository, which will configure a conda environment ready for use.

## Training time

On a PC with the following specs:

- 16~32 GB RAM

- i7 7th generation

- NVIDIA GTX 1070 GPU

It takes anywhere from 4h30 to 14h to train a neural network for 1 million to 1.5 million steps. In some cases, training has to run for around 5 to 6 million steps for some models to learn (see paper referenced in the [v9 tag](https://github.com/douglasrizzo/hcanet/releases/tag/v9)).

## Weights & Biases

I log my experiments on [Weights & Biases](https://wandb.ai/). I don't know how this code behaves if ran on another account or if W&B is not installed or not logged in.

## Maps I decide to use

- `3m` - a scenario with a single type of agent, to test all networks.

- `3s5z` - an easy scenario, but there is still room for VDN to improve.

- `MMM2` - an asymmetric heterogeneous environment in which VDN fails to win.

- `1c3s5z` - an easy scenario with high heterogeneity.

- `MMM` - a symmetric heterogeneous environment, should be easier than MMM2.