https://github.com/neka-nat/distributed_rl

Pytorch implementation of distributed deep reinforcement learning
https://github.com/neka-nat/distributed_rl

amazon-web-services ape-x deep-q-network distributed-systems double-dqn dueling-dqn openai-gym prioritized-experience-replay pytorch r2d2 reinforcement-learning

Last synced: 20 days ago
JSON representation

Pytorch implementation of distributed deep reinforcement learning

• Host: GitHub
• URL: https://github.com/neka-nat/distributed_rl
• Owner: neka-nat
• License: mit
• Created: 2018-10-25T16:06:24.000Z (about 6 years ago)
• Default Branch: master
• Last Pushed: 2022-07-04T14:09:14.000Z (over 2 years ago)
• Last Synced: 2024-10-14T19:42:55.506Z (about 1 month ago)
• Topics: amazon-web-services, ape-x, deep-q-network, distributed-systems, double-dqn, dueling-dqn, openai-gym, prioritized-experience-replay, pytorch, r2d2, reinforcement-learning
• Language: Python
• Homepage:
• Size: 972 KB
• Stars: 72
• Watchers: 6
• Forks: 8
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# distributed_rl

This is pytorch implementation of distributed deep reinforcement learning.

* ape-x

    * [Distributed Prioritized Experience Replay](https://arxiv.org/abs/1803.00933)

* r2d2 (Recurrent Replay Distributed DQN)(experimental)

    * [Recurrent Experience Replay in Distributed Reinforcement Learning](https://openreview.net/forum?id=r1lyTjAqYX)

![image](images/image.gif)

![actors](images/actors.gif)

## System

In our system, there are two processes, Actor and Learner.

In Learner process, thread of the replay memory runs at the same time,

and these processes communicate using Redis.

![system](images/system.png)

## Install

```

git clone https://github.com/neka-nat/distributed_rl.git

cd distributed_rl

poetry install

```

Install redis-server.

```

sudo apt-get install redis-server

```

Setting Atari.

https://github.com/openai/atari-py#roms

## Run

The following command is running all actors and learner in localhost.

The number of actor's processes is given as an argument.

```

poetry shell

./run.sh 4

```

Run r2d2 mode.

```

./run.sh 4 config/all_r2d2.conf

```

## Docker build

```

cd distributed_rl

docker-compose up -d

```

## Use EKS

Create EKS resource.

```

cd terraform

terraform init

terraform plan

terraform apply

```