{"id":17632460,"url":"https://github.com/mattbui/dd_qnet","last_synced_at":"2025-04-14T04:50:56.059Z","repository":{"id":90700376,"uuid":"133649604","full_name":"mattbui/dd_qnet","owner":"mattbui","description":"A Double Dueling Deep Q learning model for turtle bot to navigate in custom gym-gazebo maze","archived":false,"fork":false,"pushed_at":"2018-05-26T07:05:32.000Z","size":96735,"stargazers_count":4,"open_issues_count":0,"forks_count":2,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-01-07T10:30:57.069Z","etag":null,"topics":["double-dqn","dueling-dqn","keras","qlearning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mattbui.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-05-16T10:29:10.000Z","updated_at":"2020-08-19T07:16:30.000Z","dependencies_parsed_at":null,"dependency_job_id":"bcf98b27-5ab7-49e4-97b2-ced99622a067","html_url":"https://github.com/mattbui/dd_qnet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattbui%2Fdd_qnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattbui%2Fdd_qnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattbui%2Fdd_qnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mattbui%2Fdd_qnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mattbui","download_url":"https://codeload.github.com/mattbui/dd_qnet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":233690492,"owners_count":18714782,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["double-dqn","dueling-dqn","keras","qlearning"],"created_at":"2024-10-23T01:08:02.962Z","updated_at":"2025-01-13T03:34:35.988Z","avatar_url":"https://github.com/mattbui.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Double Dueling Q Net\n\n![running](screenshots/running.gif)\n\n## Table of contents\n- [Requirements](#requirements)\n- [Environment setup](#environment-setup)\n- [Project structure](#project-structure)\n    - [Environments](#environments)\n    - [Models](#models)\n- [Usage](#usage)\n    - [Image CNN model](#image-cnn-model)\n    - [DNN laser model](#dnn-laser-model)\n    - [Use laser model with hint detect strategy](#use-laser-model-with-hint-detect-strategy)\n- [Pretrain models](#pretrain-models)\n\n## Requirements\nBasic requirements:\n- python2\n- [gym-gazebo](https://github.com/erlerobot/gym-gazebo)\n- keras\n- tensorflow\n\n## Environment setup\n\nThe gym-gazebo environments can be found [here](https://drive.google.com/open?id=19ldjgpsS303IGfPCBrT-WEf18FYy4quL).\n\n- Copy `custom` folder to `gym-gazebo/gym_gazebo/envs`.\n- Copy `launch`, `models`, `worlds` to `gym-gazebo/gym_gazebo/envs/assets` and skip duplicated files.\n- Replace `__init__.py` file in the `gym-gazebo/gym_gazebo` folder with the `__init__.py` file here.\n- Replace the line `\u003carg name=\"world_file\"  default=\"/home/cloud/gym-gazebo/gym_gazebo/envs/assets/worlds/maze_color.world\"/\u003e` in the `gym-gazebo/gym_gazebo/envs/assets/launch/MazeColor.launch` file with your own path to the `maze_color.world` file\n\n## Project structure\n### Environments\nThe environment files used for different training and tesing situation, they are all stored in `custom_envs` folder. To use one environment, copy the code in `gazebo_turtlebot_maze_color*.py` to `gazebo_turtlebot_maze_color.py` in `gym-gazebo/gym_gazebo/envs/custom`\nList environments:\n\n- `gazebo_turtlebot_maze_color.py`: enviroment for training CNN model which use both image and laser as state input. To change target position, change line 52 `self.num_target = 1` to another index (from 0 to 2).\n\n![evn1](screenshots/env1.png)\n- `gazebo_turtlebot_maze_color_laser_only.py`: enviroment for training laser model. The environment have 5 target position to be random at the start or after one episode end. \n\n![env2](screenshots/env2.png)\n- `gazebo_turtlebot_maze_color_laser_only_ver2.py`: environment for training laser model. The environment have 5 hint at 5 corner of the maze, the laser model can learn to go to the corner and turn back. \n\n![evn3](screenshots/env3.png)\n- `gazebo_turtlebot_maze_color_laser-image.py`: environment for testing laser model with the image processing strategy. It have 5 different reward and hints positions which can be change by line 59 `self.num_target = 1` in the code to another index (from 0 to 4). \n\n![evn4](screenshots/env4.png)\n\n### Models\nAll the main codes are stored in `src` folder\nWe have 2 type of model:\n- `ddq_model.py`: the model used CNN architecture and image with laser as input state. The model use the architecture proposed in this [paper](https://storage.googleapis.com/deepmind-data/assets/papers/DeepMindNature14236Paper.pdf)\n- `laser_model.py`: The model use DNN and laser as input state.\n\n## Usage\n\n### Image CNN model\nReplace code in `gym-gazebo/.../custom/gazebo_turtlebot_maze_color.py` with the code in `gazebo_turtlebot_maze_color.py` to use this model.\n\nTrain:\n```\npython qlearning.py\n```\n\nFor more parameters: `python qlearning --help`\n\nTest model: `python test.py \u003cfrom_pretrain_dir\u003e \u003cepsilon\u003e`\n```\npython test.py ddq_model 0.01\n```\n\n### DNN laser model\nReplace code in `gym-gazebo/.../custom/gazebo_turtlebot_maze_color.py` with the code in `gazebo_turtlebot_maze_color_laser_only.py` or `gazebo_turtlebot_maze_color_laser_only_ver2.py` to use this model.\nTrain:\n```\npython laser_learning.py\n```\n\nFor more parameters: `python qlearning --help`\n\nTest model: `python test_laser_only.py \u003cfrom_pretrain_dir\u003e \u003cepsilon\u003e`\n```\npython test_laser_only.py laser-only 0.1\n```\n\n### Use laser model with hint detect strategy\nReplace code in `gym-gazebo/.../custom/gazebo_turtlebot_maze_color.py` with the code in `gazebo_turtlebot_maze_color_laser-image.py` to use this model.\n\nRun: `python test_laser_image.py \u003cfrom_pretrain_dir\u003e \u003cepsilon\u003e`\n```\npython test_laser_image.py laser-only 0.0\n```\n\n## Pretrain models\nSome of our pretrain model can be found [here](https://drive.google.com/open?id=1g8cAxaL6CcqavwaLLHuiR6_4DxYZ_WFq)\n\nIt can be use as pretrain model or continue to train with the parameters `--from_pretrain` or `--continue_from` in the learning files. For examples: `python laser_learn --continue_from laser-only --output_dir laser-only`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmattbui%2Fdd_qnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmattbui%2Fdd_qnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmattbui%2Fdd_qnet/lists"}