Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/takuseno/cpp-dqn
Blazingly Fast Implementation of Deep Q-Network in C++ with NNabla
https://github.com/takuseno/cpp-dqn
cpp deep-reinforcement-learning nnabla
Last synced: 28 days ago
JSON representation
Blazingly Fast Implementation of Deep Q-Network in C++ with NNabla
- Host: GitHub
- URL: https://github.com/takuseno/cpp-dqn
- Owner: takuseno
- License: mit
- Created: 2019-09-15T05:26:42.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2020-03-03T06:36:47.000Z (almost 5 years ago)
- Last Synced: 2024-10-23T04:06:10.772Z (about 2 months ago)
- Topics: cpp, deep-reinforcement-learning, nnabla
- Language: C++
- Homepage:
- Size: 105 KB
- Stars: 14
- Watchers: 3
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-blazingly-fast - cpp-dqn - Blazingly Fast Implementation of Deep Q-Network in C++ with NNabla (C++)
README
# cpp-dqn
Deep Q-Network implementation written in C++ with NNabla.
This project aims for the :zap: fastest and :smile: readable DQN implementation.
macOS and Linux are currently supported.
## TODO
- [ ] reproduce [the Nature paper](https://www.nature.com/articles/nature14236).
- [ ] add more DQN-based algorithms (Double DQN, Prioritized DQN, ...)
- [ ] use [CULE](https://github.com/NVlabs/cule) for further speed up.
## third party
- [nnabla](https://github.com/sony/nnabla)
- [nnabla-ext-cuda](https://github.com/sony/nnabla-ext-cuda)
- [Arcade Learning Environment](https://github.com/mgbellemare/Arcade-Learning-Environment)
- [googletest](https://github.com/google/googletest)
- [gflags](https://github.com/gflags/gflags)
- [atari-py](https://github.com/openai/atari-py) (only for extracting ROMs)
## pull prebuilt docker container
If you want to play with this implementation on docker container, please use the prebuilt container.
```
$ docker pull takuseno/cpp-dqn
$ docker run -it --rm --runtime nvidia --name cpp-dqn takuseno/cpp-dqn:latest bash
root@834182ee578b:/cpp-dqn# ./bin/train -rom atari_roms/breakout.bin
```
DockerHub: https://hub.docker.com/r/takuseno/cpp-dqn
## play with a pretrained model
```
root@834182ee578b:/cpp-dqn# ./bin/play -rom atari_roms/breakout.bin -load logs/experiment_xxxx/10000000.param
```
If you want to see GUI window through the container, try the following commands.
```
$ xhost + # this is required only once
$ docker run -it --rm --runtime nvidia \
-v /tmp/.X11-unix/:/tmp/.X11-unix \
--shm-size=256m \
-e QT_X11_NO_MITSHM=1 \
-e DISPLAY=$DISPLAY \
--name cpp-dqn takuseno/cpp-dqn:latest bash
root@834182ee578b:/cpp-dqn# ./bin/play -rom atari_roms/breakout.bin -load logs/experiment_xxxx/10000000.param -gui
```
## build with docker
To skip manual build, use prebuilt container and mount the current directory by running the following commands.
```
$ ./scripts/up.sh --runtime nvidia
root@834182ee578b:/cpp-dqn# mkdir build
root@834182ee578b:/cpp-dqn# cd build
root@834182ee578b:/cpp-dqn/build# cmake .. -DGPU=ON
root@834182ee578b:/cpp-dqn/build# make
root@834182ee578b:/cpp-dqn/build# cd ..
root@834182ee578b:/cpp-dqn# ./bin/train -rom atari_roms/breakout.bin
```
As `scripts/up.sh` will enable X11 to show GUI window, you can use `-gui` option to see rendered screens.
## manual build
### nnabla
Before building this repository, you need to install NNabla.
See [official instruction](https://github.com/sony/nnabla/blob/master/doc/build/build_cpp_utils.md).
Note that arguments of cmake must be as follows.
```
$ cmake -DBUILD_CPP_UTILS=ON -DBUILD_PYTHON_PACKAGE=OFF ..
```
If you use GPU, you additionally need to install CUDA extension of NNabla.
See [official instruction](https://github.com/sony/nnabla-ext-cuda/blob/master/doc/build/build.md).
### SDL
By default, SDL libraries are used to build to render GUI. Then you need to install related libraries.
If you need to omit this, you have to set `-DUSE_SDL=OFF`.
```
# macOS
$ brew install sdl sdl_gfx sdl_image
# Ubuntu
$ sudo apt-get install libsdl1.2-dev libsdl-gfx1.2-dev libsdl-image1.2-dev
```
### build DQN
Finally, run the following codes to build DQN.
```
$ mkdir build
$ cd build
$ cmake .. # add -DGPU=ON option to build with cuda extension
$ make
```
## test
```
$ cd build
$ make
$ cd ..
$ ./scripts/test.sh
```
## format codes
`clang-format` is used to format entire codes with `llvm` style.
```
$ ./scripts/autoformat.sh
```