Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/futurecrew/DeepRL

Deep Reinforcement Learning
https://github.com/futurecrew/DeepRL

Last synced: 3 months ago
JSON representation

Deep Reinforcement Learning

Awesome Lists containing this project

README

        

# DeepRL

This project implements deep reinforcement learning algorithms including following papers.
- Deep Q Network (Human-level control through deep reinforcement learning)
- Deep Reinforcement Learning with Double Q-learning
- Asynchronous Methods for Deep Reinforcement Learning
- Prioritized Experience Replay
- Continuous control with deep reinforcement learning



## Test scores
In my PC (i7 CPU, Titan-X Maxwell),




- A3C FF took 20 hours for 80M global steps (nips network)


- A3C LSTM took 44 hours for 80M global steps (nips network)





- DQN took 96 hours for 80M steps (shown 11M steps, nature network)


- Double-Q took 112 hours for 80M steps (shown 11M steps, nature network)


- Prioritized took 112 hours for 80M steps (shown 11M steps, nature network)

## Torcs

After training in simulator Torcs, it learns how to accelerate, brake and turn the steering wheel.


Click the image to watch the video.

## Requirements
- Python-2.7
- pip, scipy, matplotlib, numpy
- Tensorflow-0.11
- Arcade-Learning-Environment
- Torcs (optional)
- Vizdoom (in working)



See this for installation.

## How to train
```
DQN : python train.py /path/to/rom --drl dqn
Double DQN : python train.py /path/to/rom --drl double_dqn
Prioritized : python train.py /path/to/rom --drl prioritized_rank
A3C FF : python train.py /path/to/rom --drl a3c --thread-no 8
A3C LSTM : python train.py /path/to/rom --drl a3c_lstm --thread-no 8
DDPG : python train.py torcs --ddpg
```

## How to retrain
```
python train.py /path/to/rom --drl a3c --thread-no 8 --snapshot path/to/snapshot_file
ex) python train.py /rom/breakout.bin --drl a3c --thread-no 8 --snapshot snapshot/breakout/20161114_003838/a3c_6250000
```

## How to play
```
python play.py path/to/snapshot_file
ex) python play.py snapshot/space_invaders/20161114_003838/a3c_79993828
```

## Debug console commands
While training you can send several debug commands in the console.
- p : print debug logs or not
- u : pause training or not
- quit : finish running
- d : show the current running screen or not. You can see how the training is going on in the game screen.
- \- : show the screen more fast
- \+ : show the screen more slowly

## Reference projects
- https://github.com/tambetm/simple_dqn
- https://github.com/miyosuda/async_deep_reinforce
- https://github.com/muupan/async-rl
- https://github.com/yanpanlau/DDPG-Keras-Torcs