https://github.com/johnny1188/deep-q-learning-driver

Implementation of Deep reinforcement learning (Q-learning) in a self-made game environment of a highway driver.
https://github.com/johnny1188/deep-q-learning-driver

deep-q-learning reinforcement-learning

Last synced: 3 months ago
JSON representation

Implementation of Deep reinforcement learning (Q-learning) in a self-made game environment of a highway driver.

• Host: GitHub
• URL: https://github.com/johnny1188/deep-q-learning-driver
• Owner: Johnny1188
• Created: 2021-07-01T19:19:33.000Z (about 5 years ago)
• Default Branch: main
• Last Pushed: 2023-10-31T19:13:27.000Z (over 2 years ago)
• Last Synced: 2025-06-03T19:50:23.261Z (about 1 year ago)
• Topics: deep-q-learning, reinforcement-learning
• Language: Python
• Homepage:
• Size: 32.4 MB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Deep Q-Learning Driver

[Intro](#intro) - [Game](#game) - [Model used](#model-used) - [How to run](#how-to-run)

---

## Intro

Implementation of Deep reinforcement learning (Q-learning) in a self-made game environment of a highway driver. Inspiration was taken from the [DeepTraffic competition](https://github.com/lexfridman/deeptraffic) with a few tweaks in the rules and features of the game (eg. the Agent driver can not know the exact speed of surrounding cars). At each time step, the agent chooses to speed up, slow down, turn left or turn right. The goal of the agent is to achieve the highest average speed while avoiding accidents with surrounding cars.

## Game

The game is built with the Python library [PyGame](https://www.pygame.org/news) and is meant to represent a highway with multiple lanes divided into segments and unpredictable surrounding cars. Among the main parameters of the game are road difficulty (i.e. how many cars there are), number of lanes, number of segments per lane (i.e. discretization of the q-learning model's input), and range of agent's vision (i.e. input the Q-learning model gets).

![game preview](./game/imgs/game_intro.png)

## Model used

The agent uses [Deep Q-learning](https://arxiv.org/abs/1312.5602) model with two layers (default is 64 and 128 units, respectively), the ReLU activation function, and one output layer with 4 units (action space). Pre-trained weights can be found in `weights/q_learner_latest_weights.pt`. The input of this model (state) consists of an array with the agent's visible segments on the road (0 if vacant, 1 for the place of the agent, 10 for surrounding car, -10 for out of the road) and agent's current speed.

![game gif preview](./game/gif/game_animation.gif)

## How to run

`python run.py [-h] [--mode MODE_TO_RUN] [--episodes NUM_OF_EPISODES] [--difficulty DIFFICULTY_OF_ROAD] [--weights PATH_TO_WEIGHTS] [--cpu_only] [--slow] [--silent]`

This script runs the highway agent with the Deep Q-learning model.

**Optional arguments:**  

        `-h`, `--help`            show this help message and exit  

        `--mode MODE_TO_RUN`           Mode of the script to run (Either 'train' or 'test'), default: 'train'  

        `--episodes NUM_OF_EPISODES`   Number of road runs to run, default: 1000  

        `--difficulty DIFFICULTY_OF_ROAD`

                        Difficulty of the road (number of surrounding cars),

                        default: 2  

        `--weights PATH_TO_WEIGHTS`     Path to pre-trained weights. When not specified, the

                        the model learns from scratch.                       

        `--cpu_only`            Include for using only CPU  

        `--slow`                Include for slowing down the road simulation  

        `--silent`              Include for keeping the logs from the progress of agent's

                        road run silent