Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mattbui/dd_qnet
A Double Dueling Deep Q learning model for turtle bot to navigate in custom gym-gazebo maze
https://github.com/mattbui/dd_qnet
double-dqn dueling-dqn keras qlearning
Last synced: 13 days ago
JSON representation
A Double Dueling Deep Q learning model for turtle bot to navigate in custom gym-gazebo maze
- Host: GitHub
- URL: https://github.com/mattbui/dd_qnet
- Owner: mattbui
- Created: 2018-05-16T10:29:10.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2018-05-26T07:05:32.000Z (over 6 years ago)
- Last Synced: 2024-10-30T02:54:31.543Z (about 2 months ago)
- Topics: double-dqn, dueling-dqn, keras, qlearning
- Language: Python
- Homepage:
- Size: 92.3 MB
- Stars: 4
- Watchers: 0
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Double Dueling Q Net
## Table of contents
- [Requirements](#requirements)
- [Environment setup](#environment-setup)
- [Project structure](#project-structure)
- [Environments](#environments)
- [Models](#models)
- [Usage](#usage)
- [Image CNN model](#image-cnn-model)
- [DNN laser model](#dnn-laser-model)
- [Use laser model with hint detect strategy](#use-laser-model-with-hint-detect-strategy)
- [Pretrain models](#pretrain-models)
## Requirements
Basic requirements:
- python2
- [gym-gazebo](https://github.com/erlerobot/gym-gazebo)
- keras
- tensorflow
## Environment setup
The gym-gazebo environments can be found [here](https://drive.google.com/open?id=19ldjgpsS303IGfPCBrT-WEf18FYy4quL).
- Copy `custom` folder to `gym-gazebo/gym_gazebo/envs`.
- Copy `launch`, `models`, `worlds` to `gym-gazebo/gym_gazebo/envs/assets` and skip duplicated files.
- Replace `__init__.py` file in the `gym-gazebo/gym_gazebo` folder with the `__init__.py` file here.
- Replace the line `` in the `gym-gazebo/gym_gazebo/envs/assets/launch/MazeColor.launch` file with your own path to the `maze_color.world` file
## Project structure
### Environments
The environment files used for different training and tesing situation, they are all stored in `custom_envs` folder. To use one environment, copy the code in `gazebo_turtlebot_maze_color*.py` to `gazebo_turtlebot_maze_color.py` in `gym-gazebo/gym_gazebo/envs/custom`
List environments:
- `gazebo_turtlebot_maze_color.py`: enviroment for training CNN model which use both image and laser as state input. To change target position, change line 52 `self.num_target = 1` to another index (from 0 to 2).
- `gazebo_turtlebot_maze_color_laser_only.py`: enviroment for training laser model. The environment have 5 target position to be random at the start or after one episode end.
- `gazebo_turtlebot_maze_color_laser_only_ver2.py`: environment for training laser model. The environment have 5 hint at 5 corner of the maze, the laser model can learn to go to the corner and turn back.
- `gazebo_turtlebot_maze_color_laser-image.py`: environment for testing laser model with the image processing strategy. It have 5 different reward and hints positions which can be change by line 59 `self.num_target = 1` in the code to another index (from 0 to 4).
### Models
All the main codes are stored in `src` folder
We have 2 type of model:
- `ddq_model.py`: the model used CNN architecture and image with laser as input state. The model use the architecture proposed in this [paper](https://storage.googleapis.com/deepmind-data/assets/papers/DeepMindNature14236Paper.pdf)
- `laser_model.py`: The model use DNN and laser as input state.
## Usage
### Image CNN model
Replace code in `gym-gazebo/.../custom/gazebo_turtlebot_maze_color.py` with the code in `gazebo_turtlebot_maze_color.py` to use this model.
Train:
```
python qlearning.py
```
For more parameters: `python qlearning --help`
Test model: `python test.py `
```
python test.py ddq_model 0.01
```
### DNN laser model
Replace code in `gym-gazebo/.../custom/gazebo_turtlebot_maze_color.py` with the code in `gazebo_turtlebot_maze_color_laser_only.py` or `gazebo_turtlebot_maze_color_laser_only_ver2.py` to use this model.
Train:
```
python laser_learning.py
```
For more parameters: `python qlearning --help`
Test model: `python test_laser_only.py `
```
python test_laser_only.py laser-only 0.1
```
### Use laser model with hint detect strategy
Replace code in `gym-gazebo/.../custom/gazebo_turtlebot_maze_color.py` with the code in `gazebo_turtlebot_maze_color_laser-image.py` to use this model.
Run: `python test_laser_image.py `
```
python test_laser_image.py laser-only 0.0
```
## Pretrain models
Some of our pretrain model can be found [here](https://drive.google.com/open?id=1g8cAxaL6CcqavwaLLHuiR6_4DxYZ_WFq)
It can be use as pretrain model or continue to train with the parameters `--from_pretrain` or `--continue_from` in the learning files. For examples: `python laser_learn --continue_from laser-only --output_dir laser-only`