{"id":26758275,"url":"https://github.com/ksm26/occupancygrid-predictions","last_synced_at":"2025-04-15T05:25:12.536Z","repository":{"id":40639362,"uuid":"485406172","full_name":"ksm26/OccupancyGrid-Predictions","owner":"ksm26","description":"Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning","archived":false,"fork":false,"pushed_at":"2022-07-04T13:58:47.000Z","size":4234,"stargazers_count":33,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-28T16:19:01.398Z","etag":null,"topics":["deep-learning","future-prediction","occupancy-grid-map","prediction","spatio-temporal"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ksm26.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-04-25T14:30:24.000Z","updated_at":"2025-02-03T09:26:38.000Z","dependencies_parsed_at":"2022-09-18T15:20:17.060Z","dependency_job_id":null,"html_url":"https://github.com/ksm26/OccupancyGrid-Predictions","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FOccupancyGrid-Predictions","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FOccupancyGrid-Predictions/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FOccupancyGrid-Predictions/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ksm26%2FOccupancyGrid-Predictions/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ksm26","download_url":"https://codeload.github.com/ksm26/OccupancyGrid-Predictions/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249011891,"owners_count":21198114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deep-learning","future-prediction","occupancy-grid-map","prediction","spatio-temporal"],"created_at":"2025-03-28T16:19:05.984Z","updated_at":"2025-04-15T05:25:12.516Z","avatar_url":"https://github.com/ksm26.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"##  \u003cdiv align=\"center\"\u003e Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning  \u003c/div\u003e                      \n\n##  \u003cdiv align=\"center\"\u003e \u003cimg src=\"https://github.com/ksm26/SpatioTemporal-Predictions/blob/master/images/Inria.jpg\" alt=\"INRIA\" width=\"200\" height=\"50\"/\u003e \u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n \u003ca href=\"https://arxiv.org/pdf/2205.03212.pdf\"\u003ePaper\u003c/a\u003e | \u003ca href=\"https://ksm26.github.io/OccupancyGrid-Predictions/\"\u003eProject website\u003c/a\u003e | \u003ca href=\"https://www.youtube.com/watch?v=4W7dT-HfQPQ\"\u003eVideo\u003c/a\u003e\n\u003c/p\u003e\n\n\u003ch3 align=\"center\" id=\"heading\"\u003e\n    Abstract\n\u003c/h3\u003e\n\n\u003cp align=\"justify\" width=\"800\"\u003e\nReliably predicting future occupancy of highly dynamic urban environments is an important precursor for safe autonomous navigation. Common challenges in the prediction include forecasting the relative position of other vehicles, modelling the dynamics of vehicles subjected to different traffic conditions, and vanishing surrounding objects. To tackle these challenges, we propose a spatio-temporal prediction network pipeline that takes the past information from the environment and semantic labels separately for generating future occupancy predictions. Compared to the current SOTA, our approach predicts occupancy for a longer horizon of 3 seconds and in a relatively complex environment from the nuScenes dataset. Our experimental results demonstrate the ability of spatio-temporal networks to understand scene dynamics without the need for HD-Maps and explicit modeling dynamic objects. We publicly release our occupancy grid dataset based on nuScenes to support further research. \n\u003c/p\u003e\n\n\n\u003ca align=\"justify\" href=\"url\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177139170-bfd37bd5-9324-4392-b565-faad2138b19e.png\" align=\"center\" height=\"380\"\u003e\n\u003c/a\u003e\n\n\n\u003ch3 align=\"center\" id=\"heading\"\u003e\n    Generating Occupancy grids dataset\n\u003c/h3\u003e\n\n\u003cp align=\"justify\"\u003e\nOccupancy grid maps are generated from the \u003ca href=\"https://arxiv.org/pdf/1903.11027.pdf\"\u003enuScenes dataset\u003c/a\u003e. Agents belonging to the ’Vehicles’ category are of interest and being marked by ’green’ semantic pixel labels using the projections of ground truth 3D bounding boxes. Objects of any other type including the static environment are marked in ’blue’. Different road crossing motion scenarios from the dataset are presented below: \n\u003c/p\u003e\n\n\u003ca href=\"url\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177139351-ae4486d2-c493-434f-bba2-bb63fcec0c82.png\" align=\"center\" height=\"180\"  width=\"200\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177139372-3db4e24e-f8a4-4380-aa39-7849b92617d7.png\" align=\"center\" height=\"180\"  width=\"200\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177139409-bb56e1f3-26ed-4d6a-a30a-814718f96659.png\" align=\"center\" height=\"180\"  width=\"200\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177139431-a9630317-3c7f-4164-ad70-66e657d2e73f.png\" align=\"center\" height=\"180\"  width=\"200\"\u003e\n\u003c/a\u003e\n\n\u003cp align=\"justify\"\u003e\nWe publicly release the Occupancy Grid Maps dataset consisting of static environment and semantic labels for ease in long-term prediction. The dataset is available \u003ca href=\"https://archive.org/details/nuscenes-occupancy-grids-dataset\"\u003ehere\u003c/a\u003e \n\nThe paper demonstrate the performance of two state-of-art video prediction networks - \u003ca href=\"https://arxiv.org/pdf/2103.09504.pdf\"\u003ePredRNN\u003c/a\u003e and \u003ca href=\"https://papers.nips.cc/paper/2015/file/07563a3fe3bbe7e3ba84431ad9d055af-Paper.pdf\"\u003eConvLSTM\u003c/a\u003e for this dataset. Under the training scheme we observe that the models have consistent performance even for long-term predictions.\n\u003c/p\u003e\n\n\n\u003ch3 align=\"center\" id=\"heading\"\u003e\n    Spatio-temporal architecture\n\u003c/h3\u003e\n\n\u003cp align=\"justify\"\u003e\nWe present an Spatio-Temporal Network Pipeline for long-term future occupancy grid prediction. Our approach uses semantic labels of the vehicle in OGMs to model the specific motion type of agents in the prediction rather than using a generic combined prediction of static and dynamic cells that contains the environment and various types of agents such as cars, pedestrians, cyclists, etc.\n\nSemantic occupancy grids consisting of environment and vehicles over the time frames of 0.5 sec. Grids are converted into binary images and separately fed to spatio-temporal networks. We evaluate two spatio-temporal networks: PredRNN and ConvLSTM\n\u003c/p\u003e\n \n![architecture](https://user-images.githubusercontent.com/24546547/177139739-ebd21b21-4644-48a7-bee2-97b32dc8c3d8.png)\n\n\n\u003ch3 align=\"center\" id=\"heading\"\u003e\n    Qualitative Results\n\u003c/h3\u003e\n\n\u003cp align=\"justify\"\u003e\nA scene depicting few static vehicles and a vehicle going in reverse direction. Comparison between two spatio-temporal learning networks over the future predictions of 1 sec, 2 sec and 3 sec.\n\u003c/p\u003e\n\n\u003ca href=\"url\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177150383-e4e77415-1f9e-4ed8-8932-4ffb55ff3e59.jpg\" align=\"center\" height=\"180\"\u003e\n\u003c/a\u003e\n(a) Ground truth \n\n\u003ca href=\"url\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177150412-cc8c02b8-4ae9-40a7-9726-76b611254ed3.jpg\" align=\"center\" height=\"180\"\u003e\n\u003c/a\u003e\n(b) PredRNN predicitons\n\n\u003ca href=\"url\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177150364-58998cf4-04a4-44d9-b709-3df36412cd94.jpg\" align=\"center\" height=\"180\"\u003e\n\u003c/a\u003e\n(c) ConvLSTM predictions\n\n\n\u003ch3 align=\"center\" id=\"heading\"\u003e\n    Quantitative Results\n\u003c/h3\u003e\n\n\u003cp align=\"justify\"\u003e\nFrame-wise PSNR(↑), SSIM(↑), Static MSE (↓), and Semantic MSE (↓) results on the generated Occupancy grid map dataset are presented below. The prediction horizon is 3 sec during training and testing phases. Note that PredRNN (combined) and ConvLSTM (combined) predict an entire OGM, thus separate Static and Semantic MSE cannot be reported for these cases. The Semantic MSE result from the \u003ca href=\"https://hal.inria.fr/hal-03416222/document\"\u003elinear projection\u003c/a\u003e of vehicle’s bounding boxes is also presented in (d).\n\u003c/p\u003e\n\n\u003ca href=\"url\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177152101-6ed53611-84f5-4290-9853-086f9290a7c8.jpg\" align=\"center\" height=\"180\" width=\"200\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177152127-6526887c-4bb3-4305-8116-d418359aa13d.jpg\" align=\"center\" height=\"180\" width=\"200\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177152144-99bd9fc0-bae6-4edd-b56c-4f3ea9f26732.jpg\" align=\"center\" height=\"180\" width=\"200\"\u003e\n  \u003cimg src=\"https://user-images.githubusercontent.com/24546547/177152178-636a03b6-b636-429d-a3f9-c3fa5d377df9.jpg\" align=\"center\" height=\"180\" width=\"200\"\u003e\n\u003c/a\u003e\n\n\n\u003ch3 align=\"center\" id=\"heading\"\u003e\n    Bibtex\n\u003c/h3\u003e\n\n``` \n@article{mann2022predicting,\n  title={Predicting Future Occupancy Grids in Dynamic Environment with Spatio-Temporal Learning},\n  author={Mann, Khushdeep Singh and Tomy, Abhishek and Paigwar, Anshul and Renzaglia, Alessandro and Laugier, Christian},  \n  journal={arXiv preprint arXiv:2205.03212},  \n  year={2022}\n}\n``` \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fksm26%2Foccupancygrid-predictions","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fksm26%2Foccupancygrid-predictions","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fksm26%2Foccupancygrid-predictions/lists"}