{"id":15640369,"url":"https://github.com/nikolasent/road-semantic-segmentation","last_synced_at":"2025-04-30T07:49:59.001Z","repository":{"id":205011073,"uuid":"102219581","full_name":"NikolasEnt/Road-Semantic-Segmentation","owner":"NikolasEnt","description":"Udacity Self-Driving Car Engineer Nanodegree. Project: Road Semantic Segmentation","archived":false,"fork":false,"pushed_at":"2017-12-28T21:57:57.000Z","size":44521,"stargazers_count":87,"open_issues_count":1,"forks_count":33,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-30T07:49:53.041Z","etag":null,"topics":["carnd","classification","computer-vision","deep-learning","fcn","self-driving-car","semantic-segmentation","tensorflow"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NikolasEnt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-09-02T19:42:14.000Z","updated_at":"2025-03-18T15:33:42.000Z","dependencies_parsed_at":null,"dependency_job_id":"92630f26-20d7-43e6-97ae-a50290bd56e4","html_url":"https://github.com/NikolasEnt/Road-Semantic-Segmentation","commit_stats":null,"previous_names":["nikolasent/road-semantic-segmentation"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FRoad-Semantic-Segmentation","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FRoad-Semantic-Segmentation/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FRoad-Semantic-Segmentation/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NikolasEnt%2FRoad-Semantic-Segmentation/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NikolasEnt","download_url":"https://codeload.github.com/NikolasEnt/Road-Semantic-Segmentation/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251666227,"owners_count":21624291,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["carnd","classification","computer-vision","deep-learning","fcn","self-driving-car","semantic-segmentation","tensorflow"],"created_at":"2024-10-03T11:35:02.198Z","updated_at":"2025-04-30T07:49:58.967Z","avatar_url":"https://github.com/NikolasEnt.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Semantic Segmentation\n\n### Udacity Self-Driving Car Engineer Nanodegree. Project: Vehicle Detection and Tracking\n\nThis Project is the twelfth task of the Udacity Self-Driving Car Nanodegree program. The main goal of the project is to train an artificial neural network for semantic segmentation of a video from a front-facing camera on a car in order to mark road pixels using Tensorflow.\n\n## Results\n\nKITTI Road segmentation (main task of the project):\n\n![title.gif animation](readme_img/title.gif)\n\nCityscapes multiclass segmentation (optional task):\n\n![title_cityscapes.gif animation](readme_img/title_cityscapes.gif)\n\n## Content of this repo\n\n- `Segmentation.ipynb` - Jupyter notebook with the main code for the project\n- `helper.py` - python program for images pre- and  post- processing.\n- `runs` - directory with processed images\n- `cityscapes.ipynb` - Jupyter notebook with some visualization and preprocessing of the Cityscape dataset. Please, see the notebook for correct dataset directories placing. \n- `Segmentation_cityscapes.ipynb` - Jupyter notebook with the main code for the Cityscape dataset.\n- `helper_cityscapes.py` - python program for images pre- and  post- processing for the Cityscape dataset.\n\n**Note:** The repository does not contain any training images. You have to download the image datasetsplace them in appropriate directories on your own.\n\n### Architecture\n\nA Fully Convolutional Network (FCN-8 Architecture developed at Berkeley, see [paper](https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf) ) was applied for the project. It uses VGG16 pretrained on ImageNet as an encoder.\nDecoder is used to upsample features, extracted by the VGG16 model, to the original image size. The decoder is based on transposed convolution layers.\n\nThe goal is to assign each pixel of the input image to the appropriate class (road, backgroung, etc). So, it is a classification problem, that is why, cross entropy loss was applied.\n\n### Setup\n\nHyperparameters were chosen by the try-and-error process. Adam optimizer was used as a well-established optimizer. Weights were initialized by a random normal initializer. Some benefits of L2 weights regularization were observed, therefore, it was applied in order to reduce grainy edges of masks.\n\n### Augmentation\n\nResized input images were also treated by random contrast and brightness augmentation (as linear function of the input image). It helps to produce reasonable predictions in difficult light conditions.\n\n```Python\ndef bc_img(img, s = 1.0, m = 0.0):\n    img = img.astype(np.int)\n    img = img * s + m\n    img[img \u003e 255] = 255\n    img[img \u003c 0] = 0\n    img = img.astype(np.uint8)\n    return img   \n```\n\n![1.png](readme_img/1.png)\n\n![2.png](readme_img/2.png)\n\n![3.png](readme_img/3.png)\n\n_Deep shadows and contrast variations are not a problem because of rich augmentation on the training stage._\n\n## Cityscapes\n\nTwo classes (roads and cars) were chosen from the Cityscapes dataset for the optional task. The classes are unbalanced (roads are prevalent), so, a weighted loss function was involved (see [Segmentation_cityscapes.ipynb](Segmentation_cityscapes.ipynb\n) for details). Interestingly, RMSProp optimizer performed better for the imageset.\n\nUnfortunately, accord to the Cityscapes dataset licence I can not publish all produced images, however, there are some of them.\n\n![cityscapes_1.png](readme_img/cityscapes_1.png)\n\n_It correctly do not label a cyclist as a car, but mark small partly occluded cars._\n\n![cityscapes_1.png](readme_img/cityscapes_2.png)\n\n_It does not have problems with recognizing a cobbled street as a road._\n\n![cityscapes_1.png](readme_img/cityscapes_3.png)\n\n_And the ANN is able to mark cars in different projections._\n\nReferences:\n\n1. [KITTI](http://www.cvlibs.net/datasets/kitti/) dataset\n2. [Cityscapes](https://www.cityscapes-dataset.com/) dataset\n3. [FCN](https://people.eecs.berkeley.edu/~jonlong/long_shelhamer_fcn.pdf) ANN\n\n_____________________ Udacity Readme.md ____________________\n\n\n### Setup\n##### Frameworks and Packages\nMake sure you have the following is installed:\n - [Python 3](https://www.python.org/)\n - [TensorFlow](https://www.tensorflow.org/)\n - [NumPy](http://www.numpy.org/)\n - [SciPy](https://www.scipy.org/)\n##### Dataset\nDownload the [Kitti Road dataset](http://www.cvlibs.net/datasets/kitti/eval_road.php) from [here](http://www.cvlibs.net/download.php?file=data_road.zip).  Extract the dataset in the `data` folder.  This will create the folder `data_road` with all the training a test images.\n\n### Start\n##### Implement\nImplement the code in the `main.py` module indicated by the \"TODO\" comments.\nThe comments indicated with \"OPTIONAL\" tag are not required to complete.\n##### Run\nRun the following command to run the project:\n```\npython main.py\n```\n**Note** If running this in Jupyter Notebook system messages, such as those regarding test status, may appear in the terminal rather than the notebook.\n\n### Submission\n1. Ensure you've passed all the unit tests.\n2. Ensure you pass all points on [the rubric](https://review.udacity.com/#!/rubrics/989/view).\n3. Submit the following in a zip file.\n - `helper.py`\n - `main.py`\n - `project_tests.py`\n - Newest inference images from `runs` folder\n \n ## How to write a README\nA well written README file can enhance your project and portfolio.  Develop your abilities to create professional README files by completing [this free course](https://www.udacity.com/course/writing-readmes--ud777).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikolasent%2Froad-semantic-segmentation","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnikolasent%2Froad-semantic-segmentation","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnikolasent%2Froad-semantic-segmentation/lists"}