{"id":15643242,"url":"https://github.com/nikolasent/vehicle-detection-and-tracking","last_synced_at":"2025-04-30T11:15:14.043Z","repository":{"id":205011078,"uuid":"83070726","full_name":"NikolasEnt/Vehicle-Detection-and-Tracking","owner":"NikolasEnt","description":"Udacity Self-Driving Car Engineer Nanodegree. Project: Vehicle Detection and Tracking","archived":false,"fork":false,"pushed_at":"2018-01-10T03:16:36.000Z","size":51545,"stargazers_count":58,"open_issues_count":0,"forks_count":40,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-26T06:54:37.008Z","etag":null,"topics":["carnd","classifier","computer-vision","opencv","self-driving-car","svm"],"latest_commit_sha":null,"homepage":null,"language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NikolasEnt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-02-24T18:26:46.000Z","updated_at":"2024-07-05T03:10:22.000Z","dependencies_parsed_at":null,"dependency_job_id":"71c854e2-825f-4fcb-9797-62a393d60368","html_url":"https://github.com/NikolasEnt/Vehicle-Detection-and-Tracking","commit_stats":null,"previous_names":["nikolasent/vehicle-detection-and-tracking"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FVehicle-Detection-and-Tracking","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FVehicle-Detection-and-Tracking/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FVehicle-Detection-and-Tracking/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FVehicle-Detection-and-Tracking/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NikolasEnt","download_url":"https://codeload.github.com/NikolasEnt/Vehicle-Detection-and-Tracking/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251687589,"owners_count":21627591,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["carnd","classifier","computer-vision","opencv","self-driving-car","svm"],"created_at":"2024-10-03T11:59:39.822Z","updated_at":"2025-04-30T11:15:14.015Z","avatar_url":"https://github.com/NikolasEnt.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vehicle-Detection-and-Tracking\n### Udacity Self-Driving Car Engineer Nanodegree. Project: Vehicle Detection and Tracking\n\nThis Project is the fifth task of the Udacity Self-Driving Car Nanodegree program. The main goal of the project is to create a software pipeline to identify vehicles in a video from a front-facing camera on a car. Additionally, an [Advanced Lane Line](https://github.com/NikolasEnt/Advanced-Lane-Lines) finding algorithm was added from the fourth task of the Nanodegree program.\n\n![Title .gif animation](readme_img/title.gif)\n\n**Result:** [video](https://youtu.be/waYJjmkRZfw)\n\nFor details on this and other projects, see my [website](https://nikolasent.github.io/proj/proj2)\n\n## Content of this repo\n\n- `VehicheDetect.ipynb` - Jupyter notebook with code for the project\n- `laneline.py` - python program for lane line detection from the [project 4](https://github.com/NikolasEnt/Advanced-Lane-Lines).\n- `test_images` - a directory with test images\n- `camera_cal` - a directory with camera calibration images from the [project 4](https://github.com/NikolasEnt/Advanced-Lane-Lines).\n- `project_video_proc.mp4` - the result video\n- `project_video.mp4` - the original raw video from [Udacity](https://github.com/udacity/CarND-Vehicle-Detection)\n\n**Note:** The repository does not contain any training images. You have to download and unzip the image datasets of vehicles and non-vehicles provided by Udacity and place them in appropriate directories on your own.\nPlease, see links under \"Data loading\" header in the [VehicheDetect.ipynb](./VehicheDetect.ipynb) notebook.\n\n## Classifier\n\nFeatures are needed to train a classifier and make predictions on the test or real-world images.\n\nThe project required to build a classifier that is able to answer if there is a car in a given image (subset of the whole image). To address this task three types of features were used: HOG (Histogram of Oriented Gradients) (shape features), binned color (color and shape features) and color histogram features (color only features). This combination of features can provide enough information for image classification.\n\nFirstly, an automated approach was applied to tune the HOG parameters (`orientations, pixels_per_cell, cells_per_block`).\n\nSomething like:\n```Python\nfrom skopt import gp_minimize\nspace  = [(8, 64),                  # nbins\n          (6, 12),                  # orient\n          (4, 16),                   # pix_per_cell\n          (1, 2)]                   # cell_per_block\ni = 0\ndef obj(params):\n    global i\n    nbins, orient, pix_per_cell, cell_per_block = params\n    car_features = extract_features(cars[0:len(cars):10], nbins, orient, pix_per_cell, cell_per_block)\n    notcar_features = extract_features(notcars[0:len(notcars):10], nbins, orient, pix_per_cell, cell_per_block)\n    y = np.hstack((np.ones(len(cars[0:len(cars):10])), np.zeros(len(notcars[0:len(notcars):10]))))\n    X = np.vstack((car_features, notcar_features)).astype(np.float64)                        \n    # Fit a per-column scaler\n    X_scaler = StandardScaler().fit(X)\n    # Apply the scaler to X\n    scaled_X = X_scaler.transform(X)\n    X_train, X_test, y_train, y_test = train_test_split(scaled_X, y, test_size=0.2, random_state=22)\n    svc = LinearSVC()\n    svc.fit(X_train, y_train)\n    test_acc = svc.score(X_test, y_test)\n    print i, params, test_acc\n    i+=1\n    return 1.0-test_acc\n    \nres = gp_minimize(obj, space, n_calls=20, random_state=22)\n\"Best score=%.4f\" % res.fun\n```\n\nHowever, results were not very good because it ended with high numbers for HOG parameters which results in very slow feature extraction with comparable to less computational-expensive parameters set accuracy. That is why, the parameters for HOG as well as parameters for other features extractors were finetuned manually by try and error process so that it optimizes accuracy and computation time.\n\nHere is an example of a train image and its HOG:\n\n![Example image](readme_img/ex.jpg) ![HOG of example image](readme_img/hog.jpg)\n\nFinal parameter for feature extraction:\n\n```Python\ncolor_space = 'LUV' # Can be RGB, HSV, LUV, HLS, YUV, YCrCb\norient = 8  # HOG orientations\npix_per_cell = 8 # HOG pixels per cell\ncell_per_block = 2 # HOG cells per block\nhog_channel = 0 # Can be 0, 1, 2, or \"ALL\"\nspatial_size = (16, 16) # Spatial binning dimensions\nhist_bins = 32    # Number of histogram bins\n```\n\nNormalizing ensures that a classifier's behavior isn't dominated by just a subset of the features, and that the training process is as efficient as possible. That is why, feature list was normolized by the `StandardScaler()` method from `sklearn`. The data is splitted into thaining and testing subsets (80% and 20%). The classifier is a linear SVM. It was found that it performs well enough and quite fast for the task. The code chunk under *Classifier* represents these operations.\n\n\n## Sliding Window and the classifier testing\n\nBasic sliding window algoritm was implemented in the same way to one presented in Udacity's lectures (See the code chunks under *Slide window* header). It allows to search a car in a desired region of the frame with a desired window size (each subsamled window is rescaled to 64x64 px before classifing by the SVC).\n\nThe window size and overlap should be wisely selected. Size of the window should be compared to the size of an expected car. These parameters were set to mimic perspective.\n\nThere are some sample results for a fixed window size (128x128 px) and overlap for the provided test images:\n\n![Test image 1](output_images/test1.jpg)\n![Test image 2](output_images/test2.jpg)\n![Test image 3](output_images/test3.jpg)\n![Test image 4](output_images/test4.jpg)\n![Test image 5](output_images/test5.jpg)\n![Test image 6](output_images/test6.jpg)\n\nAs we can see on examples above, the classifier successfully finds cars on the test images. However, there is a false positive example, so, we will need to apply a kind of filter (such as heat map) and the classifier failed to find a car on th 3rd image because it is too small for it. That is why, we will need to use multi scale windows.\n\n### Implemented ideas:\n\n- Images were preprocessed by undistortion process from the [Advanced Lane Line](https://github.com/NikolasEnt/Advanced-Lane-Lines) finding project\n\n- To increase the classifier accuracy, feature extraction parameters were tuned. The data was augmented by flipped images.\n\n- To reduce number of false positives a heatmap with a threshold approach was implemented in the same to the suggested in the lectures way. For video the heatmap is accumulated by two frames which reduces number of outliers false positives.\n\n- To increase performance it is needed to analize the smallest possible number of windows. That is why, one can scan with a search window not across the whole image, but only areas where a new car can appear and also we are going to scan areas where a car was detected (track cars) \n\n*There is an example of a new car detection ROI:*\n\n![New car detection ROI](readme_img/new_car_windows.jpg)\n\n*And a car tracking ROI:*\n\n![New car detection ROI](readme_img/car_track_window.jpg)\n\n- It is important to use different scale of the classifiers window on different parts of the image due to perspective. So, different ROI  window sizes were applied on different areas (realized in the `frame_proc` function).\n\n- In order to reduce jitter the function `filt` applies a simple low-pass filter on the new and the previous cars boxes coordinates and sizes (see under the *Frames processing* header) with weight `ALPHA=0.75` of the previous data. This makes car boundaries on video quite smooth.\n\n- To increase performance the analizys was skiped for every 2nd frame because we do not expect very fast moving of the detected cars. Known cars boundaries from the previous frame is used in such cases.\n\n## The pipeline visualization\n\nAreas of interest for tracking of detected cars are marked green. Hot windows (which were classified as cars) are yellow.\n\n![Image proc](readme_img/image_proc.jpg)\n\nThe heatmap of found hot windows overlap:\n\n![Heatmap](readme_img/heatmap.jpg)\n\nThe final result of an image with cars boundaries and lane detection.\n\n![Image proc](readme_img/image_fin.jpg)\n\n\n## Results and discussion\n\nThe pipeline is able to correctly lable cars areas on a video frames. The final video is [here](https://github.com/NikolasEnt/Vehicle-Detection-and-Tracking/blob/master/project_video_proc.mp4). The [Advanced Lane Line](https://github.com/NikolasEnt/Advanced-Lane-Lines) finding algorithm was added for the lane marking.\n\n- Of course, the algorithm may fail in case of difficult light conditions, which could be partly resolved by the classifier improvement.\n\n- It is possible to improve the classifier by additional data augmentation, hard negative mining, classifier parameters tuning etc.\n\n- The algorithm may have some problems in case of car overlaps another. To resolve this problem one may introduce long term memory of car position and a kind of predictive algorithm which can predict where occluded car can be and where it is worth to look for it.\n\n- To eliminate false positives on areas out of the road, one can deeply combine results from the Advanced Lane Line finding project to correctly determine the wide ROI on the whole frame by the road boundaries. Unfortunately, it was not correctly implemented (just hard coded, which is enought for the project but not a good implementation for a real-world application) due to time limitation.\n\n- The pipeline is not a real-time (about 4 fps with Lane line detection, which independently performs at 9 fps). One can further optimize number of features and feature extraction parameters as well as number of analyzed windows to increase the rate because lane line detection is quite fast.  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikolasent%2Fvehicle-detection-and-tracking","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnikolasent%2Fvehicle-detection-and-tracking","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikolasent%2Fvehicle-detection-and-tracking/lists"}