https://github.com/burakahmet/autonomous-car-with-deep-q-learning
Self driving car with Deep Q-Learning
https://github.com/burakahmet/autonomous-car-with-deep-q-learning
airsim airsim-simulator autonomous-agents autonomous-car autonomous-driving autonomous-vehicles convolutional-neural-networks deep-learning deep-neural-networks deep-q-learning deep-q-network deep-reinforcement-learning otonom-arac python pytorch pytorch-cnn reinforcement-learning reinforcement-learning-agent self-driving-car self-learning
Last synced: 6 months ago
JSON representation
Self driving car with Deep Q-Learning
- Host: GitHub
- URL: https://github.com/burakahmet/autonomous-car-with-deep-q-learning
- Owner: BurakAhmet
- License: apache-2.0
- Created: 2025-01-28T14:03:47.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2025-01-28T15:41:51.000Z (8 months ago)
- Last Synced: 2025-02-12T15:49:11.037Z (8 months ago)
- Topics: airsim, airsim-simulator, autonomous-agents, autonomous-car, autonomous-driving, autonomous-vehicles, convolutional-neural-networks, deep-learning, deep-neural-networks, deep-q-learning, deep-q-network, deep-reinforcement-learning, otonom-arac, python, pytorch, pytorch-cnn, reinforcement-learning, reinforcement-learning-agent, self-driving-car, self-learning
- Language: Python
- Homepage:
- Size: 5.91 MB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Autonomous Car with Deep Q-Learning
This repository demonstrates how to train and run a Deep Q-learning (DQN) model for controlling an autonomous car using the [AirSim](https://github.com/microsoft/AirSim/releases/download/v1.8.1-windows/AirSimNH.zip) simulation environment. The primary goal is to teach the car to navigate an environment without collisions, maximizing a reward function that encourages safe and efficient driving.
## Table of Contents
1. [Project Overview](https://github.com/BurakAhmet/Autonomous-Car-with-Deep-Q-Learning/tree/main?tab=readme-ov-file#project-overview)
2. [Installation](https://github.com/BurakAhmet/Autonomous-Car-with-Deep-Q-Learning/tree/main?tab=readme-ov-file#installation)
- Requirements
3. [Usage](https://github.com/BurakAhmet/Autonomous-Car-with-Deep-Q-Learning/tree/main?tab=readme-ov-file#usage)
4. [How It Works?](https://github.com/BurakAhmet/Autonomous-Car-with-Deep-Q-Learning/tree/main?tab=readme-ov-file#how-it-works)
- Neural Network Architecture
- Reward Function
5. [Results](https://github.com/BurakAhmet/Autonomous-Car-with-Deep-Q-Learning/tree/main?tab=readme-ov-file#results)
## Project Overview
In this project, a deep Q-learning algorithm is employed to train a car to navigate autonomously. The car receives observations from the environment—such as camera view of the obstacles, velocity, coordinates, and sensor readings—and uses a DQN to decide the best possible steering and acceleration actions. The car trained for 1000 episodes (it took about 12 hours) to reach the end of the street and turn left without any collisions.
## Installation
### Requirements
* Python
* PyTorch
* NumPy
* OpenCV-python
* Matplotlib
* AirSim (You can download it from https://github.com/microsoft/AirSim/releases/download/v1.8.1-windows/AirSimNH.zip)
You can download the necessary libraries from the [requirements.txt](https://github.com/BurakAhmet/Autonomous-Car-with-Deep-Q-Learning/blob/main/requirements.txt) with this command:
```pip install -r requirements.txt```.
## Usage
* Using an Existing Model: ```python drive.py```. Do not forget to change the model path.
* Training a model: ```python train.py```.
## How It Works
### Neural Network Architecture
* The DQN model takes the current image observation from the simulation, resized to 84×84, as input. It consists of three convolutional layers that extract features from the image. The filter sizes in these layers decrease from 8×8 to 3×3, with strides adjusting from 4×4 to 1×1, and each layer is followed by ReLU activation. After the last convolutional layer, the outputs are concatenated with the current state of the simulation, represented by the X, Y , and Z coordinates, to inform the network about the vehicle’s current position. The concatenated features are passed through a fully connected layer with 512 neurons, followed by another ReLU activation. The final layer outputs five actions, representing the possible navigation decisions for the vehicle.
* The last layer outputs Q-values for each possible action.
* By selecting the action with the highest Q-value, the agent makes a decision at each step.
## Reward Function
* The reward function provides critical feedback to guide the agent's learning in the environment.
* It heavily penalizes collisions (-100 reward) by ending the episode upon collision, discouraging unsafe actions.
* It rewards the agent for reaching waypoints (+100 reward) and gives an additional bonus for completing all waypoints (+500 reward), encouraging goal-oriented behavior.
* It offers continuous feedback based on the reduction of distance to the target waypoint, rewarding movement towards the goal and penalizing moving away.
* By combining penalties and rewards, the function guides the agent to learn safe navigation while efficiently reaching its objectives.
## Results
**Reference Path**
* 
The numbered red point are represents the waypoints on the path.
**Reached Path**
* 
**Autonomous Driving Video (accelerated x2)**
https://github.com/user-attachments/assets/0d804259-b4cf-4437-98af-8a9bf7f66803
**Mean Max Q-value per Episode**
* 
**Total Reward Per Episode**
* 