{"id":14958904,"url":"https://github.com/gautam-j/self-driving-car","last_synced_at":"2025-07-19T23:35:48.233Z","repository":{"id":39365399,"uuid":"122729461","full_name":"Gautam-J/Self-Driving-Car","owner":"Gautam-J","description":"Automated Driving in NFS using CNN.","archived":false,"fork":false,"pushed_at":"2021-06-18T03:54:17.000Z","size":9558,"stargazers_count":150,"open_issues_count":0,"forks_count":23,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-07-09T03:09:49.130Z","etag":null,"topics":["alexnet","automatic-game-role-managment","automation","autonomous-driving","car","convolutional-neural-networks","game","keypresses","neural-network","nfs","python","self-driving-car","tensorflow","tensorflow-examples","tensorflow-experiments","tensorflow-tutorial","tflearn"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Gautam-J.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-02-24T10:18:32.000Z","updated_at":"2025-04-09T11:04:27.000Z","dependencies_parsed_at":"2022-09-09T21:50:34.131Z","dependency_job_id":null,"html_url":"https://github.com/Gautam-J/Self-Driving-Car","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/Gautam-J/Self-Driving-Car","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gautam-J%2FSelf-Driving-Car","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gautam-J%2FSelf-Driving-Car/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gautam-J%2FSelf-Driving-Car/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gautam-J%2FSelf-Driving-Car/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Gautam-J","download_url":"https://codeload.github.com/Gautam-J/Self-Driving-Car/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Gautam-J%2FSelf-Driving-Car/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266041961,"owners_count":23867959,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["alexnet","automatic-game-role-managment","automation","autonomous-driving","car","convolutional-neural-networks","game","keypresses","neural-network","nfs","python","self-driving-car","tensorflow","tensorflow-examples","tensorflow-experiments","tensorflow-tutorial","tflearn"],"created_at":"2024-09-24T13:18:28.840Z","updated_at":"2025-07-19T23:35:48.174Z","avatar_url":"https://github.com/Gautam-J.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Self Driving Car\n\n\u003e This project is mainly focused on End-to-End Deep Learning for Self-Driving-Cars. Uses raw image data, a convolutional neural network, and an autoencoder for autonomous driving in Need For Speed Most Wanted (2005) v1.3 game.\n\n## Sections:\n* [What's New](#changelog)\n* [Demo](#demo)\n* [Installation](#installation)\n* [Working](#working)\n* [DriveNet](#DriveNet)\n* [CrashNet](#CrashNet)\n* [LaneFinder](#Lanefinder)\n* [Citations](#citations)\n* [Todo](#todo)\n\n---\n## Changelog\nVersion 2.0\n* Updated from tflearn to Keras API on top of tensorflow 2.0\n* Updated from alexnet to new architecture.\n* Using minimap also as an input, along with road images.\n\n## Demo\nLink to Reddit Post: [Click here](https://www.reddit.com/r/Python/comments/iwt09a/endtoend_self_driving_car_need_for_speed/?utm_source=share\u0026utm_medium=web2x\u0026context=3)\n\n![demo](nfs.gif)\n\n## Installation\n`python -m pip install --user --requirement requirements.txt`\n\nAll required modules (except the built-ins) are listed below.\n```\nopencv-python\nnumpy\npsutil\npandas\nsklearn\ntensorflow\nmatplotlib\n```\n\n## Working\n1. Visualizing Region of Interest - to make sure that we are capturing the areas that we want and nothing extra.\n    * [Visualize screen](visualize_screen.py) can be used to see the area of the road that is being captured when recording training data.\n    * [Visualize map](visualize_map.py) can be used to see the area of the minimap that is located in the bottom-left corner, that is captured when recording training data.\n\n1. Getting Training Data - capturing raw frames along with player's inputs.\n    * [Get data](get_data.py) is used to capture the ROI found in step 1. We capture both the road and the minimap per observation as features, along with the player's input as label.\n    * The captured road frame is resized to (80, 200, 3)\n    * The captured minimap frame is resized to (50, 50, 1)\n\n1. Balancing the data - The raw data is balanced to avoid bias.\n    * [Balance data](balance_data.py) removes the unwanted bias in the training data.\n    * The raw data has most of the observations with labels for _forward_ with few observations with labels for _left_ or _right_.\n    * We thus discard the excess amount of unwanted data that has label as _forward_. Keep in mind that we do __lose a lot of data__.\n\n1. Combining the data - The balanced files can now be joined together to form one final data file.\n    * [Combine data](combine_data.py) is used to join all balanced data for easier loading of data during training process.\n    * All batches of balanced data is now combined together to form _final\\_data.npy_ file.\n    * This file has 2 images as features, with shape (80, 200, 3) and (50, 50, 1), respectively, and a one-hot-encoded label with 3 classes.\n\n1. Training the Neural Network - [DriveNet](#drivenet) is used as the convolutional neural network for autonomous driving. [CrashNet](#crashnet), an autoencoder, is used to for anomaly detection during autonomous driving.\n    * [Train model](train_model.py) is used to train DriveNet over a max of 100 epochs.\n    * [Train CrashNet](train_crashnet.py) is used to train CrashNet.\n    * Training is regulated by EarlyStopping Callback, monitoring validation_loss with a patience of 3 epochs.\n    * Adam optimizer is used, with learning rate set to 0.001\n    * No data augmentation is done.\n\n1. Testing the model - Final testing done in the game.\n    * [Test model](test_model.py) is used to actually run the trained model and control the car real-time. For CrashNet, a threshold of 0.0095 is used for anomaly detection. (Reconstruction loss - Mean Squared Error)\n\n## DriveNet\nGraph of [DriveNet](drivenet.py), rendered using plot_model function.\n\n![Image](DriveNet.png)\n\n* **NOTE:** This architecture is heavily inspired by the paper \"Variational End-to-End Navigation and Localization\" by Alexander Amini and others. Refer to the [citation](#citations) section for more details.\n\n## CrashNet\nGraph of [CrashNet](crashnet.py), rendered using plot_model function.\n\n![Image](CrashNet_autoencoder.png)\n\n## LaneFinder\nPipeline is as follows:\n* Convert image to grayscale\n* Apply Gaussian Blur\n* Canny Edge Detection (threshold values calculated automatically)\n* Masking region of interest\n* Probabilistic Hough Transform\n* Selecting 2 lines(lanes) averaged over\n    * If both lines have negative slope, go right\n    * If both lines have positive slope, go left\n    * If both lines have different slope, go straight\n\n**NOTE:** This pipeline is almost similar to the one used in Sentdex's Python plays GTA-V. Refer to the [citations](#citations) section for more details.\n\n## Citations\n1. [MIT - Intro to Deep Learning Course](https://introtodeeplearning.com/ \"Go to HomePage\")\n1. [Variational End-to-End Navigation and Localization](https://arxiv.org/abs/1811.10119v2 \"Go to arxiv page\")\n1. [Sentex's Python Plays GTA-V](https://github.com/Sentdex/pygta5 \"Go to GitHub\")\n1. [AlexNet](https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf \"Go to pdf\")\n\n## Todo\n* Use .hdf5 files instead of .npy files for better memory utilization.\n    * Switch to PyTorch if dealing with .hdf5 datasets.\n* Implement some sort of reinforcement learning algorithm to avoid collecting data.\n* Merge LaneFinder, DriveNet and CrashNet for better driving.\n* Carla\n* TORCS\n* VDrift\n* Beam.ng\n* Add a control filter for the output, maybe a low pass filter\n* Vagrant multiple VM - run the game with different camera views\n* Use a cheat engine that tracks car's relative position, speed, angle, etc.\n* Have analog control with a PID controller\n* Have a pre-trained depth estimator model, use the depth map as features, and keypresses as labels.\n\n---\nOpen to suggestions. Feel free to fork this repository. If you would to use some code from here, please do give the required citations and references.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgautam-j%2Fself-driving-car","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgautam-j%2Fself-driving-car","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgautam-j%2Fself-driving-car/lists"}