{"id":18792413,"url":"https://github.com/prbonn/lidiff","last_synced_at":"2025-04-13T05:07:30.642Z","repository":{"id":226034827,"uuid":"761136364","full_name":"PRBonn/LiDiff","owner":"PRBonn","description":"[CVPR'24] Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion","archived":false,"fork":false,"pushed_at":"2024-10-28T14:21:20.000Z","size":58864,"stargazers_count":243,"open_issues_count":1,"forks_count":21,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-04-13T05:07:26.759Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PRBonn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-02-21T10:01:06.000Z","updated_at":"2025-04-11T13:02:10.000Z","dependencies_parsed_at":"2024-05-22T10:46:42.708Z","dependency_job_id":"7ed1db90-286f-420e-ad31-f857d35ec3ad","html_url":"https://github.com/PRBonn/LiDiff","commit_stats":null,"previous_names":["prbonn/lidiff"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PRBonn%2FLiDiff/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PRBonn","download_url":"https://codeload.github.com/PRBonn/LiDiff/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248665747,"owners_count":21142123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-07T21:19:46.713Z","updated_at":"2025-04-13T05:07:30.624Z","avatar_url":"https://github.com/PRBonn.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion\n\n**[Paper](http://www.ipb.uni-bonn.de/pdfs/nunes2024cvpr.pdf)** **|** **[Sup. material](http://www.ipb.uni-bonn.de/pdfs/nunes2024cvpr-supmaterial.pdf)** **|** **[Video](https://www.youtube.com/watch?v=XWu8svlMKUo)**\n\nThis repo contains the code for the scene completion diffusion method proposed in the CVPR'24 paper: \"Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion\".\n\nOur method leverages diffusion process as a point-wise local problem, disentangling the scene data distribution during in the diffusion process, learning only the point local neighborhood distribution. From our formulation we\ncan achieve a complete scene representation from a single LiDAR scan directly operating over the 3D points.\n\n![](media/diff_steps.png)\n\n## Dependencies\n\nInstalling python (we have used python 3.8) packages pre-requisites:\n\n`sudo apt install build-essential python3-dev libopenblas-dev`\n\n`pip3 install -r requirements.txt`\n\nInstalling MinkowskiEngine:\n\n`pip3 install -U MinkowskiEngine==0.5.4 --install-option=\"--blas=openblas\" -v --no-deps`\n\nTo setup the code run the following command on the code main directory:\n\n`pip3 install -U -e .`\n\n## SemanticKITTI Dataset\n\nThe SemanticKITTI dataset has to be download from the official [site](http://www.semantic-kitti.org/dataset.html#download) and extracted in the following structure:\n\n```\n./lidiff/\n└── Datasets/\n    └── SemanticKITTI\n        └── dataset\n          └── sequences\n            ├── 00/\n            │   ├── velodyne/\n            |   |       ├── 000000.bin\n            |   |       ├── 000001.bin\n            |   |       └── ...\n            │   └── labels/\n            |       ├── 000000.label\n            |       ├── 000001.label\n            |       └── ...\n            ├── 08/ # for validation\n            ├── 11/ # 11-21 for testing\n            └── 21/\n                └── ...\n```\n\n## Ground truth generation\n\nTo generate the ground complete scenes you can run the `map_from_scans.py` script. This will use the dataset scans and poses to generate the sequence map to be used as ground truth during training:\n\n```\npython3 map_from_scans.py --path Datasets/SemanticKITTI/dataset/sequences/\n```\n\nOnce the sequences map is generated you can then train the model.\n\n## Training the diffusion model\n\nFor training the diffusion model, the configurations are defined in `config/config.yaml`, and the training can be started with:\n\n`python3 train.py`\n\nFor training the refinement network, the configurations are defined in `config/config_refine.yaml`, and the training can be started with:\n\n`python3 train_refine.py`\n\n## Trained model\n\nYou can download the trained model weights and save then to `lidiff/checkpoints/`:\n\n- Diffusion model [weights](https://www.ipb.uni-bonn.de/html/projects/lidiff/diff_net.ckpt)\n- Refinement model [weights](https://www.ipb.uni-bonn.de/html/projects/lidiff/refine_net.ckpt)\n\n## Diffusion Scene Completion Pipeline\n\nFor running the scene completion inference we provide a pipeline where both the diffusion and refinement network are loaded and used to complete the scene from an input scan. You can run the pipeline with the command:\n\n`python3 tools/diff_completion_pipeline.py --diff DIFF_CKPT --refine REFINE_CKPT -T DENOISING_STEPS -s CONDITIONING_WEIGHT`\n\nWe provide one scan as example in `lidiff/Datasets/test/` so you can directly test it out with our trained model by just running the code above.\n\n## Citation\n\nIf you use this repo, please cite as :\n\n```bibtex\n@inproceedings{nunes2024cvpr,\n    author = {Lucas Nunes and Rodrigo Marcuzzi and Benedikt Mersch and Jens Behley and Cyrill Stachniss},\n    title = {{Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion}},\n    booktitle = {{Proc. of the IEEE/CVF Conf. on Computer Vision and Pattern Recognition (CVPR)}},\n    year = {2024}\n}\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Flidiff","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprbonn%2Flidiff","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprbonn%2Flidiff/lists"}