https://github.com/reubenjohn/distributed-tf-keras-rl

Demonstration of distributed Tensorflow Reinforcement Learning with Keras
https://github.com/reubenjohn/distributed-tf-keras-rl

distributed-computing keras keras-tensorflow machine-learning python reinforcement-learning tensorflow

Last synced: 9 days ago
JSON representation

Demonstration of distributed Tensorflow Reinforcement Learning with Keras

• Host: GitHub
• URL: https://github.com/reubenjohn/distributed-tf-keras-rl
• Owner: reubenjohn
• License: mit
• Created: 2019-07-11T01:56:29.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2022-04-13T19:47:23.000Z (almost 3 years ago)
• Last Synced: 2024-11-20T10:57:14.907Z (2 months ago)
• Topics: distributed-computing, keras, keras-tensorflow, machine-learning, python, reinforcement-learning, tensorflow
• Language: Python
• Homepage:
• Size: 19.5 KB
• Stars: 0
• Watchers: 0
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Distributed-TF-Keras-RL

An simple working example of distributed reinforcement learning with Keras backed by tensorflow using the Dataset api, multiprocessing and more!

In a nutshell

---

We bring up a cluster of processes:  

**6 Explorers**: Applies the learned model to explore the environment and collect data to learn from.  

**1 Learner**: Improves the model by learning from the collected data.

Usage

---

Launch two horse.py with task_index arguments 0 and 1 respectively, and similarly two rider.py.

Launch them in separate windows for a better demonstration.

  

    python3.5 -m distributed_horse_rider.start horse 0 # In window 1

    python3.5 -m distributed_horse_rider.start horse 1 # In window 2

    python3.5 -m distributed_horse_rider.start rider 0 # In window 3

    python3.5 -m distributed_horse_rider.start rider 1 # In window 4

Or, for more convenience, `python3.5 -m distributed_horse_rider.launch multiwindow` (`multiwindow` is optional).

Under the ~~hood~~ analogies

---

 - The `whip` is shared by all drivers and horses => Shared FIFO queue of capacity 16.

 - Each whipping of a particular intensity => An integer that is enqueued by a driver

 - `flick_intensity` => TF placeholder

 - `flick_whip` => TF op to enqueue the flick_intensity

 - `feel_whip` => TF op to dequeue the flick_intensity

 - `steps_taken` => TF variable shared by all horses and riders storing total steps taken by all horses

 - `take_step` => TF op to add the dequeued flick_intensity to the shared `steps_taken` variable

 - `measure_distance` => Just a TF op representing half the number of steps taken

![distributed_horse_rider Architecture](https://image.ibb.co/nQkHvz/distributed_horse_rider_architecture.png)

Feeling adventurous? Play with it!

---

With just a little bit of tweaking,

you can play with the number of horses and drivers,

remove the input statements to benchmark the performance (spoiler I found it pretty impressive for a master-slave based IPC!),

go full distributed - running it beyond your localhost and across multiple machine on the same LAN,

or... whatever else is in these days. Enjoy!

Troubleshooting

---

Due to the simplicity of this demo, there isn't any graceful shutdown, you just kill the processes when you're done.

Make sure to kill these processes. You'll be unable to launch this project until the previous launches have been shut down.

Please [post issues](https://github.com/Project-MANAS/distributed_horse_rider/issues) if you face any!