{"id":15116180,"url":"https://github.com/PRBonn/LiDiff","last_synced_at":"2025-09-27T21:32:33.805Z","repository":{"id":226034827,"uuid":"761136364","full_name":"PRBonn/LiDiff","owner":"PRBonn","description":"[CVPR'24] Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion","archived":false,"fork":false,"pushed_at":"2024-10-28T14:21:20.000Z","size":58864,"stargazers_count":234,"open_issues_count":0,"forks_count":21,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-01-17T17:09:35.603Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PRBonn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-21T10:01:06.000Z","updated_at":"2025-01-11T14:35:49.000Z","dependencies_parsed_at":"2024-05-22T10:46:42.708Z","dependency_job_id":"7ed1db90-286f-420e-ad31-f857d35ec3ad","html_url":"https://github.com/PRBonn/LiDiff","commit_stats":null,"previous_names":["prbonn/lidiff"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PRBonn","download_url":"https://codeload.github.com/PRBonn/LiDiff/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":234461889,"owners_count":18837185,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-26T01:44:12.434Z","updated_at":"2025-09-27T21:32:33.798Z","avatar_url":"https://github.com/PRBonn.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion\n\n**[Paper](http://www.ipb.uni-bonn.de/pdfs/nunes2024cvpr.pdf)** **|** **[Sup. material](http://www.ipb.uni-bonn.de/pdfs/nunes2024cvpr-supmaterial.pdf)** **|** **[Video](https://www.youtube.com/watch?v=XWu8svlMKUo)**\n\nThis repo contains the code for the scene completion diffusion method proposed in the CVPR'24 paper: \"Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion\".\n\nOur method leverages diffusion process as a point-wise local problem, disentangling the scene data distribution during in the diffusion process, learning only the point local neighborhood distribution. From our formulation we\ncan achieve a complete scene representation from a single LiDAR scan directly operating over the 3D points.\n\n![](media/diff_steps.png)\n\n## Dependencies\n\nInstalling python (we have used python 3.8) packages pre-requisites:\n\n`sudo apt install build-essential python3-dev libopenblas-dev`\n\n`pip3 install -r requirements.txt`\n\nInstalling MinkowskiEngine:\n\n`pip3 install -U MinkowskiEngine==0.5.4 --install-option=\"--blas=openblas\" -v --no-deps`\n\nTo setup the code run the following command on the code main directory:\n\n`pip3 install -U -e .`\n\n## SemanticKITTI Dataset\n\nThe SemanticKITTI dataset has to be download from the official [site](http://www.semantic-kitti.org/dataset.html#download) and extracted in the following structure:\n\n```\n./lidiff/\n└── Datasets/\n    └── SemanticKITTI\n        └── dataset\n          └── sequences\n            ├── 00/\n            │   ├── velodyne/\n            |   |       ├── 000000.bin\n            |   |       ├── 000001.bin\n            |   |       └── ...\n            │   └── labels/\n            |       ├── 000000.label\n            |       ├── 000001.label\n            |       └── ...\n            ├── 08/ # for validation\n            ├── 11/ # 11-21 for testing\n            └── 21/\n                └── ...\n```\n\n## Ground truth generation\n\nTo generate the ground complete scenes you can run the `map_from_scans.py` script. This will use the dataset scans and poses to generate the sequence map to be used as ground truth during training:\n\n```\npython3 map_from_scans.py --path Datasets/SemanticKITTI/dataset/sequences/\n```\n\nOnce the sequences map is generated you can then train the model.\n\n## Training the diffusion model\n\nFor training the diffusion model, the configurations are defined in `config/config.yaml`, and the training can be started with:\n\n`python3 train.py`\n\nFor training the refinement network, the configurations are defined in `config/config_refine.yaml`, and the training can be started with:\n\n`python3 train_refine.py`\n\n## Trained model\n\nYou can download the trained model weights and save then to `lidiff/checkpoints/`:\n\n- Diffusion model [weights](https://www.ipb.uni-bonn.de/html/projects/lidiff/diff_net.ckpt)\n- Refinement model [weights](https://www.ipb.uni-bonn.de/html/projects/lidiff/refine_net.ckpt)\n\n## Evaluation\n\nBased on this [issue](https://github.com/PRBonn/LiDiff/issues/45), we found a minor bug in our completion metric code where the metric was not computed over the entire point cloud. We have updated the code to fix it, and below we provide the updated results from Table 3 of the paper:\n\n| Method | 0.5 cm | 0.2 cm | 0.1 cm |\n| ----------| -----------|-------------| ----------|\n| PVD | 15.91 | 3.97 | 0.6 | \n| LMSCNet| 30.83 | 12.09 | 3.65 | \n|LODE    |  **33.81**  | 16.39      | 5.0      |\n|MID      | 31.58   | 22.72      | 13.14   |\n|LiDiff    | 31.47   | 16.79     | 4.67   |\n|LiDiff (Refined)    | 32.43   | **22.99**      | **13.40**   |\n\n## Diffusion Scene Completion Pipeline\n\nFor running the scene completion inference we provide a pipeline where both the diffusion and refinement network are loaded and used to complete the scene from an input scan. You can run the pipeline with the command:\n\n`python3 tools/diff_completion_pipeline.py --diff DIFF_CKPT --refine REFINE_CKPT -T DENOISING_STEPS -s CONDITIONING_WEIGHT`\n\nWe provide one scan as example in `lidiff/Datasets/test/` so you can directly test it out with our trained model by just running the code above.\n\n## Citation\n\nIf you use this repo, please cite as :\n\n```bibtex\n@inproceedings{nunes2024cvpr,\n    author = {Lucas Nunes and Rodrigo Marcuzzi and Benedikt Mersch and Jens Behley and Cyrill Stachniss},\n    title = {{Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion}},\n    booktitle = {{Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)}},\n    year = {2024}\n}\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPRBonn%2FLiDiff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FPRBonn%2FLiDiff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FPRBonn%2FLiDiff/lists"}