{"id":18301468,"url":"https://github.com/mozturan/autonomousdrive2d-drl","last_synced_at":"2025-04-05T14:30:59.622Z","repository":{"id":222526039,"uuid":"700508706","full_name":"mozturan/AutonomousDrive2D-DRL","owner":"mozturan","description":"Autonomous Driving W/ Deep Reinforcement Learning in Lane Keeping - DDQN and SAC with kinematics/birdview-images","archived":false,"fork":false,"pushed_at":"2024-07-17T10:27:05.000Z","size":121363,"stargazers_count":11,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-21T05:43:05.573Z","etag":null,"topics":["autonomous-driving","ddqn","deep-learning","deep-reinforcement-learning","highway-env","lane-keeping","prioritized-experience-replay","sac","simulation","soft-actor-critic"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mozturan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2023-10-04T18:14:27.000Z","updated_at":"2025-03-19T03:08:04.000Z","dependencies_parsed_at":"2024-02-15T18:25:26.053Z","dependency_job_id":"76bdc4bb-2258-4204-9e51-091f86143f7b","html_url":"https://github.com/mozturan/AutonomousDrive2D-DRL","commit_stats":null,"previous_names":["rafiqollective/thesis","9xxi8q4f/thesis","mozturan/autonomousdrive2d-drl"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozturan%2FAutonomousDrive2D-DRL","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozturan%2FAutonomousDrive2D-DRL/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozturan%2FAutonomousDrive2D-DRL/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mozturan%2FAutonomousDrive2D-DRL/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mozturan","download_url":"https://codeload.github.com/mozturan/AutonomousDrive2D-DRL/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247352359,"owners_count":20925253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["autonomous-driving","ddqn","deep-learning","deep-reinforcement-learning","highway-env","lane-keeping","prioritized-experience-replay","sac","simulation","soft-actor-critic"],"created_at":"2024-11-05T15:16:01.520Z","updated_at":"2025-04-05T14:30:56.475Z","avatar_url":"https://github.com/mozturan.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Autonomous Driving W/ Deep Reinforcement Learning in Lane Keeping\n\nA comprehensive comparative analysis of two formidable deep reinforcement learning algorithms: **Soft Actor-Critic (SAC)** and **Double Deep Q-Network with Prioritized Experience Replay (DDQN with PER)**. Our primary goal was to discern how the choice of observation space influences the performance of these algorithms.\n\nOur primary goal was to discern how the choice of observation space influences the performance of these algorithms and to offer an alternative to end-to-end deep learning studies carried out with raw sensor data and to show that processed data is much more successful in terms of reinforcement learning algorithms in the autonomous driving system, compared to raw data.\n\n# **NOTE!**\nThis is work in progress so there are things to do:\n- [ ] Modulate the code\n- [ ] Implement some other DRL algorithms\n- [ ] Expand project with collision avoidence task (This needs a new reward function)\n- [x] Implement more advanced experiment tracking tool (Wandb instead of tensorboard is better)\n\n## Simulation Environment\n![Screenshot from 2024-02-14 19-03-17](https://github.com/rafiqollective/thesis/assets/89272933/10953242-8b1b-415b-862d-fda8bbbfb8d4)\n\n- Using Highway-Env simulation.\n- The simulated environment was designed to mimic a racetrack scenario.\n- Vehicle tasked with lane-keeping and maintaining target speed on a racetrack.\n- Testing two different deep reinforcement learning algorithms (SAC and DDQN-PER) with two different observation types (Kinematics and Birdview Images)\n\n## **Action Space :**\nBoth steering and throttle can be controlled. In fact, *\"one_act\"* file contains code for the situation where agents control steering only, and *\"two_acts\"* file contains code for the situation where agents control both steering and throttle. This doc focused on \"*two_acts*\". \n\nAction spaces are continuous between **[-1,1]** values. Continuous action space is supported in SAC. For DDQN-PER, we discretize action space to 55 different action.  \n\n## **Observation Spaces**\nTwo different observation types are testes:\n  1. Kinematics\n     \n![Screenshot from 2024-02-14 19-17-12](https://github.com/rafiqollective/thesis/assets/89272933/351696b0-f9bc-4191-a4cd-c03673fafb3e)\n\n  3. Birdview Images\n     \n![Screenshot from 2024-02-14 19-18-06](https://github.com/rafiqollective/thesis/assets/89272933/61d7f780-25f8-49e8-b055-bc3cd4379ff6)\n\n## **Reward Funtion**\nDesigned to Promote:\n  - On-road behavior\n  - Distance to lane centering\n  - Target speed maintenance\n\n![Screenshot from 2024-02-14 19-22-00](https://github.com/rafiqollective/thesis/assets/89272933/779d131e-acb4-4783-a316-c8cda0826848)\n\n** **FOR TARGET SPEED MAINTENANCE WE USE GAUSSIAN FUNCTION**\n\nTerminal conditions:\n  - Agent is off road\n  - Agent reaches maximum number of steps\n  - Agent reaches maximum time to run \n\n## **Deep Networks for Algorithms**\n\n#### For Kinematics Input\n![Screenshot from 2024-02-14 19-25-04](https://github.com/rafiqollective/thesis/assets/89272933/b8130389-55fd-441f-8465-d58e53807147)\n\n#### For Birdview Input\n![Screenshot from 2024-02-14 19-26-59](https://github.com/rafiqollective/thesis/assets/89272933/15c4ae4f-f9fa-4b3b-a109-1294e4b1fa1e)\n\n# RESULS\n\n### **Performance Graphs**\n\n![avegare_100](https://github.com/rafiqollective/thesis/assets/89272933/0c8a128f-5db6-4226-8065-db022594c656)\n![episode_reward](https://github.com/rafiqollective/thesis/assets/89272933/181cbf88-4631-4834-a8ef-04b97084d89f)\n![episode_len](https://github.com/rafiqollective/thesis/assets/89272933/913e36e3-ac57-49c0-894a-67c385a61563)\n\n\n## **SAC with KINEMATICS INPUT TRAINING RESULTS**\nhttps://github.com/rafiqollective/thesis/assets/89272933/30c2c8f4-209d-44f6-98f9-c89bc9620ce3\n\n## **DDQN-PER with KINEMATICS INPUT TRAINING RESULTS**\n\n\nhttps://github.com/rafiqollective/thesis/assets/89272933/87b5acfc-98a9-4f52-90ca-cf9d84edcb64\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmozturan%2Fautonomousdrive2d-drl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmozturan%2Fautonomousdrive2d-drl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmozturan%2Fautonomousdrive2d-drl/lists"}