{"id":18792439,"url":"https://github.com/prbonn/point-cloud-prediction","last_synced_at":"2025-08-20T06:32:03.098Z","repository":{"id":45544218,"uuid":"410846060","full_name":"PRBonn/point-cloud-prediction","owner":"PRBonn","description":"Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks","archived":false,"fork":false,"pushed_at":"2024-04-20T09:16:31.000Z","size":4322,"stargazers_count":143,"open_issues_count":0,"forks_count":25,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-11-26T01:33:59.203Z","etag":null,"topics":["cloud","forecasting","lidar","point","point-cloud","point-cloud-forecasting","point-cloud-prediction","point-forecasting","prediction","range-image","self-supervised-learning","video","video-prediction"],"latest_commit_sha":null,"homepage":"https://www.ipb.uni-bonn.de/wp-content/papercite-data/pdf/mersch2021corl.pdf","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PRBonn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2021-09-27T10:51:58.000Z","updated_at":"2024-11-05T12:19:44.000Z","dependencies_parsed_at":"2024-04-20T10:42:46.784Z","dependency_job_id":null,"html_url":"https://github.com/PRBonn/point-cloud-prediction","commit_stats":{"total_commits":11,"total_committers":2,"mean_commits":5.5,"dds":0.2727272727272727,"last_synced_commit":"805526c02abcfea603fbefca9b9e95b45c6b45f4"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fpoint-cloud-prediction","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fpoint-cloud-prediction/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fpoint-cloud-prediction/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2Fpoint-cloud-prediction/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PRBonn","download_url":"https://codeload.github.com/PRBonn/point-cloud-prediction/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":230400606,"owners_count":18219830,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","forecasting","lidar","point","point-cloud","point-cloud-forecasting","point-cloud-prediction","point-forecasting","prediction","range-image","self-supervised-learning","video","video-prediction"],"created_at":"2024-11-07T21:19:52.260Z","updated_at":"2024-12-19T08:07:44.683Z","avatar_url":"https://github.com/PRBonn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks\nThis is a Pytorch-Lightning implementation of the paper \"Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks\".\n\n![](docs/introduction.png)\n*Given a sequence of P past point clouds (left in red) at time T, the goal is to predict the F future scans (right in blue).*\n\n## Table of Contents\n1. [Publication](#Publication)\n2. [Data](#Data)\n3. [Installation](#Installation)\n4. [Training](#Training)\n5. [Testing](#Testing)\n6. [Visualization](#Visualization)\n7. [Download](#Dwnload)\n8. [License](#License)\n\n![](docs/architecture.png)\n*Overview of our architecture*\n\n## Publication\nIf you use our code in your academic work, please cite the corresponding [paper](https://www.ipb.uni-bonn.de/wp-content/papercite-data/pdf/mersch2021corl.pdf):\n    \n```latex\n@inproceedings{mersch2021corl,\n  author = {B. Mersch and X. Chen and J. Behley and C. Stachniss},\n  title = {{Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks}},\n  booktitle = {Proc.~of the Conf.~on Robot Learning (CoRL)},\n  year = {2021},\n}\n```\n\n## Data\nDownload the Kitti Odometry data from the official [website](http://www.cvlibs.net/datasets/kitti/eval_odometry.php).\n\n## Installation\n\n### Source Code\nClone this repository and run \n```bash\ncd point-cloud-prediction\ngit submodule update --init\n```\nto install the Chamfer distance submodule. The Chamfer distance submodule is originally taken from [here](https://github.com/chrdiller/pyTorchChamferDistance) with some modifications to use it as a submodule. All parameters are stored in ```config/parameters.yaml```.\n\n### Dependencies\nIn this project, we use CUDA 10.2. All other dependencies are managed with Python Poetry and can be found in the ```poetry.lock``` file. If you want to use Python Poetry (recommended), install it with:\n```bash\ncurl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -\n```\n\nInstall Python dependencies with Python Poetry\n```bash\npoetry install\n```\n\nand activate the virtual environment in the shell with\n```bash\npoetry shell\n```\n\n### Export Environment Variables to dataset\nWe process the data in advance to speed up training. The preprocessing is automatically done if ```GENERATE_FILES``` is set to true in ```config/parameters.yaml```. The environment variable ```PCF_DATA_RAW``` points to the directory containing the train/val/test sequences specified in the config file. It can be set with\n\n```bash\nexport PCF_DATA_RAW=/path/to/kitti-odometry/dataset/sequences\n```\n\nand the destination of the processed files ```PCF_DATA_PROCESSED``` is set with\n\n```bash\nexport PCF_DATA_PROCESSED=/desired/path/to/processed/data/\n```\n\n## Training\n*Note* If you have not pre-processed the data yet, you need to set ```GENERATE_FILES: True``` in ```config/parameters.yaml```. After that, you can set ```GENERATE_FILES: False``` to skip this step.\n\nThe training script can be run by\n```bash\npython pcf/train.py\n```\nusing the parameters defined in ```config/parameters.yaml```. Pass the flag ```--help``` if you want to see more options like resuming from a checkpoint or initializing the weights from a pre-trained model. A directory will be created in ```pcf/runs``` which makes it easier to discriminate between different runs and to avoid overwriting existing logs. The script saves everything like the used config, logs and checkpoints into a path ```pcf/runs/COMMIT/EXPERIMENT_DATE_TIME``` consisting of the current git commit ID (this allows you to checkout at the last git commit used for training), the specified experiment ID (```pcf``` by default) and the date and time.\n\n*Example:*\n```pcf/runs/7f1f6d4/pcf_20211106_140014```\n\n```7f1f6d4```: Git commit ID\n\n```pcf_20211106_140014```: Experiment ID, date and time\n\n## Testing\nTest your model by running\n```bash\npython pcf/test.py -m COMMIT/EXPERIMENT_DATE_TIME\n```\nwhere ```COMMIT/EXPERIMENT_DATE_TIME``` is the relative path to your model in ```pcf/runs```. *Note*: Use the flag ```-s``` if you want to save the predicted point clouds for visualiztion and ```-l``` if you want to test the model on a smaller amount of data.\n\n*Example*\n```bash\npython pcf/test.py -m 7f1f6d4/pcf_20211106_140014\n```\nor \n```bash\npython pcf/test.py -m 7f1f6d4/pcf_20211106_140014 -l 5 -s\n```\nif you want to test the model on 5 batches and save the resulting point clouds.\n\n## Visualization\nAfter passing the ```-s``` flag to the testing script, the predicted range images will be saved as .svg files in ```pcf/runs/COMMIT/EXPERIMENT_DATE_TIME/range_view_predictions```. The predicted point clouds are saved to ```pcf/runs/COMMIT/EXPERIMENT_DATE_TIME/test/point_clouds```. You can visualize them by running\n```bash\npython pcf/visualize.py -p pcf/runs/COMMIT/EXPERIMENT_DATE_TIME/test/point_clouds\n```\n\n![](docs/predictions.gif)\n*Five past and five future ground truth and our five predicted future range images.*\n\n![](docs/qualitative.png)\n*Last received point cloud at time T and the predicted next 5 future point clouds. Ground truth points\nare shown in red and predicted points in blue.*\n\n## Download\nYou can download our best performing model from the paper [here](https://www.ipb.uni-bonn.de/html/projects/point-cloud-prediction/pretrained.zip). Just extract the zip file into ```pcf/runs```.\n\n## License\nThis project is free software made available under the MIT License. For details see the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fpoint-cloud-prediction","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprbonn%2Fpoint-cloud-prediction","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Fpoint-cloud-prediction/lists"}