https://github.com/mozturan/autonomousdrive2d-drl
Autonomous Driving W/ Deep Reinforcement Learning in Lane Keeping - DDQN and SAC with kinematics/birdview-images
https://github.com/mozturan/autonomousdrive2d-drl
autonomous-driving ddqn deep-learning deep-reinforcement-learning highway-env lane-keeping prioritized-experience-replay sac simulation soft-actor-critic
Last synced: about 1 year ago
JSON representation
Autonomous Driving W/ Deep Reinforcement Learning in Lane Keeping - DDQN and SAC with kinematics/birdview-images
- Host: GitHub
- URL: https://github.com/mozturan/autonomousdrive2d-drl
- Owner: mozturan
- Created: 2023-10-04T18:14:27.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-17T10:27:05.000Z (almost 2 years ago)
- Last Synced: 2025-03-21T05:43:05.573Z (over 1 year ago)
- Topics: autonomous-driving, ddqn, deep-learning, deep-reinforcement-learning, highway-env, lane-keeping, prioritized-experience-replay, sac, simulation, soft-actor-critic
- Language: Jupyter Notebook
- Homepage:
- Size: 116 MB
- Stars: 11
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Autonomous Driving W/ Deep Reinforcement Learning in Lane Keeping
A comprehensive comparative analysis of two formidable deep reinforcement learning algorithms: **Soft Actor-Critic (SAC)** and **Double Deep Q-Network with Prioritized Experience Replay (DDQN with PER)**. Our primary goal was to discern how the choice of observation space influences the performance of these algorithms.
Our primary goal was to discern how the choice of observation space influences the performance of these algorithms and to offer an alternative to end-to-end deep learning studies carried out with raw sensor data and to show that processed data is much more successful in terms of reinforcement learning algorithms in the autonomous driving system, compared to raw data.
# **NOTE!**
This is work in progress so there are things to do:
- [ ] Modulate the code
- [ ] Implement some other DRL algorithms
- [ ] Expand project with collision avoidence task (This needs a new reward function)
- [x] Implement more advanced experiment tracking tool (Wandb instead of tensorboard is better)
## Simulation Environment
- Using Highway-Env simulation.
- The simulated environment was designed to mimic a racetrack scenario.
- Vehicle tasked with lane-keeping and maintaining target speed on a racetrack.
- Testing two different deep reinforcement learning algorithms (SAC and DDQN-PER) with two different observation types (Kinematics and Birdview Images)
## **Action Space :**
Both steering and throttle can be controlled. In fact, *"one_act"* file contains code for the situation where agents control steering only, and *"two_acts"* file contains code for the situation where agents control both steering and throttle. This doc focused on "*two_acts*".
Action spaces are continuous between **[-1,1]** values. Continuous action space is supported in SAC. For DDQN-PER, we discretize action space to 55 different action.
## **Observation Spaces**
Two different observation types are testes:
1. Kinematics
3. Birdview Images
## **Reward Funtion**
Designed to Promote:
- On-road behavior
- Distance to lane centering
- Target speed maintenance
** **FOR TARGET SPEED MAINTENANCE WE USE GAUSSIAN FUNCTION**
Terminal conditions:
- Agent is off road
- Agent reaches maximum number of steps
- Agent reaches maximum time to run
## **Deep Networks for Algorithms**
#### For Kinematics Input
#### For Birdview Input
# RESULS
### **Performance Graphs**
## **SAC with KINEMATICS INPUT TRAINING RESULTS**
https://github.com/rafiqollective/thesis/assets/89272933/30c2c8f4-209d-44f6-98f9-c89bc9620ce3
## **DDQN-PER with KINEMATICS INPUT TRAINING RESULTS**
https://github.com/rafiqollective/thesis/assets/89272933/87b5acfc-98a9-4f52-90ca-cf9d84edcb64