{"id":18573547,"url":"https://github.com/prs-eth/overlappredator","last_synced_at":"2025-05-15T16:06:15.259Z","repository":{"id":43666654,"uuid":"315675611","full_name":"prs-eth/OverlapPredator","owner":"prs-eth","description":"[CVPR 2021, Oral] PREDATOR: Registration of 3D Point Clouds with Low Overlap. ","archived":false,"fork":false,"pushed_at":"2025-02-20T10:39:38.000Z","size":5745,"stargazers_count":541,"open_issues_count":16,"forks_count":80,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-05-13T06:45:40.059Z","etag":null,"topics":["3dvision","attention-mechanism","point-cloud","registration","transformer"],"latest_commit_sha":null,"homepage":"https://shengyuh.github.io/predator/index.html","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prs-eth.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-24T15:34:16.000Z","updated_at":"2025-05-13T02:32:51.000Z","dependencies_parsed_at":"2024-10-28T02:10:24.327Z","dependency_job_id":"4d16d9f6-ea73-4849-a353-1f5910363a8b","html_url":"https://github.com/prs-eth/OverlapPredator","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FOverlapPredator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FOverlapPredator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FOverlapPredator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prs-eth%2FOverlapPredator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prs-eth","download_url":"https://codeload.github.com/prs-eth/OverlapPredator/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254374470,"owners_count":22060611,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3dvision","attention-mechanism","point-cloud","registration","transformer"],"created_at":"2024-11-06T23:10:47.638Z","updated_at":"2025-05-15T16:06:15.242Z","avatar_url":"https://github.com/prs-eth.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"## PREDATOR: Registration of 3D Point Clouds with Low Overlap (CVPR 2021, Oral)\nThis repository represents the official implementation of the paper:\n\n### [PREDATOR: Registration of 3D Point Clouds with Low Overlap](https://arxiv.org/abs/2011.13005)\n\n\\*[Shengyu Huang](https://shengyuh.github.io), \\*[Zan Gojcic](https://zgojcic.github.io/), [Mikhail Usvyatsov](https://aelphy.github.io), [Andreas Wieser](https://gseg.igp.ethz.ch/people/group-head/prof-dr--andreas-wieser.html), [Konrad Schindler](https://prs.igp.ethz.ch/group/people/person-detail.schindler.html)\\\n|[ETH Zurich](https://igp.ethz.ch/) | \\* Equal contribution\n\nFor implementation using MinkowskiEngine backbone, please check [this](https://github.com/ShengyuH/OverlapPredator.Mink)\n\nFor more information, please see the [project website](https://overlappredator.github.io)\n\n![Predator_teaser](assets/teaser_predator.jpg?raw=true)\n\n\n\n### Contact\nIf you have any questions, please let us know: \n- Shengyu Huang {shengyu.huang@geod.baug.ethz.ch}\n- Zan Gojcic {zan.gojcic@geod.baug.ethz.ch}\n\n## News\n- 2021-08-09: We've updated arxiv version of our [paper](https://arxiv.org/abs/2011.13005) with improved performance!\n- 2021-06-02: Fix feature gathering bug in k-nn graph, please see improved performance in this [issue](https://github.com/overlappredator/OverlapPredator/issues/15). Stay tunned for updates on other experiments!\n- 2021-05-31: Check our video and poster on [project page](https://overlappredator.github.io)! \n- 2021-03-25: Camera ready is on arXiv! I also gave a talk on Predator(中文), you can find the recording here: [Bilibili](https://www.bilibili.com/video/BV1UK4y1U7Gs), [Youtube](https://www.youtube.com/watch?v=AZQGJa6R_4I\u0026t=1563s) \n- 2021-02-28: MinkowskiEngine-based PREDATOR [release](https://github.com/ShengyuH/OverlapPredator.Mink.git)\n- 2020-11-30: Code and paper release\n\n\n## Instructions\nThis code has been tested on \n- Python 3.8.5, PyTorch 1.7.1, CUDA 11.2, gcc 9.3.0, GeForce RTX 3090/GeForce GTX 1080Ti\n\n**Note**: We observe random data loader crashes due to memory issues, if you observe similar issues, please consider reducing the number of workers or increasing CPU RAM. We now released a sparse convolution-based Predator, have a look [here](https://github.com/ShengyuH/OverlapPredator.Mink.git)!\n\n### Requirements\nTo create a virtual environment and install the required dependences please run:\n```shell\ngit clone https://github.com/overlappredator/OverlapPredator.git\nvirtualenv predator; source predator/bin/activate\ncd OverlapPredator; pip install -r requirements.txt\ncd cpp_wrappers; sh compile_wrappers.sh; cd ..\n```\nin your working folder.\n\n### Datasets and pretrained models\nFor KITTI dataset, please follow the instruction on [KITTI Odometry website](http://www.cvlibs.net/datasets/kitti/eval_odometry.php) to download the KITTI odometry training set.\n\nWe provide \n- preprocessed 3DMatch pairwise datasets (voxel-grid subsampled fragments together with their ground truth transformation matrices)\n- raw dense 3DMatch datasets\n- modelnet dataset\n- pretrained models on 3DMatch, KITTI and Modelnet\n\nThe preprocessed data and models can be downloaded by running:\n```shell\nsh scripts/download_data_weight.sh\n```\n\nTo download raw dense 3DMatch data, please run:\n```shell\nwget --no-check-certificate --show-progress https://share.phys.ethz.ch/~gsg/pairwise_reg/3dmatch.zip\nunzip 3dmatch.zip\n```\n\nThe folder is organised as follows:\n\n- `3dmatch`\n    - `train`\n        - `7-scenes-chess`\n            - `fragments`\n                - `cloud_bin_*.ply`\n                - ...\n            - `poses`\n                - `cloud_bin_*.txt`\n                - ...\n        - ...\n    - `test`\n\n### 3DMatch(Indoor)\n#### Train\nAfter creating the virtual environment and downloading the datasets, Predator can be trained using:\n```shell\npython main.py configs/train/indoor.yaml\n```\n\n#### Evaluate\nFor 3DMatch, to reproduce Table 2 in our main paper, we first extract features and overlap/matachability scores by running: \n```shell\npython main.py configs/test/indoor.yaml\n```\nthe features together with scores will be saved to ```snapshot/indoor/3DMatch```. The estimation of the transformation parameters using RANSAC can then be carried out using:\n```shell\nfor N_POINTS in 250 500 1000 2500 5000\ndo\n  python scripts/evaluate_predator.py --source_path snapshot/indoor/3DMatch --n_points $N_POINTS --benchmark 3DMatch --exp_dir snapshot/indoor/est_traj --sampling prob\ndone\n```\ndependent on ```n_points``` used by RANSAC, this might take a few minutes. The final results are stored in ```snapshot/indoor/est_traj/{benchmark}_{n_points}_prob/result```. To evaluate PREDATOR on 3DLoMatch benchmark, please also change ```3DMatch``` to ```3DLoMatch``` in ```configs/test/indoor.yaml```.\n\n#### Demo\nWe prepared a small demo, which demonstrates the whole Predator pipeline using two random fragments from the 3DMatch dataset. To carry out the demo, please run:\n```shell\npython scripts/demo.py configs/test/indoor.yaml\n```\n\nThe demo script will visualize input point clouds, inferred overlap regions, and point cloud aligned with the estimated transformation parameters:\n\n\u003cimg src=\"assets/demo.png\" alt=\"demo\" width=\"750\"/\u003e\n\n### ModelNet(Synthetic)\n#### Train\nTo train PREDATOR on ModelNet, please run:\n```\npython main.py configs/train/modelnet.yaml\n```\n\nWe provide a small script to evaluate Predator on ModelNet test set, please run:\n```\npython main.py configs/test/modelnet.yaml\n```\nThe rotation and translation errors could be better/worse than the reported ones due to randomness in RANSAC. \n\n### KITTI(Outdoor)\nWe provide a small script to evaluate Predator on KITTI test set, after configuring KITTI dataset, please run:\n```\npython main.py configs/test/kitti.yaml\n```\nthe results will be saved to the log file.\n\n\n### Custom dataset\nWe have a few tips for train/test on custom dataset\n\n- If it's similar indoor scenes, please run ```demo.py``` first to check the generalisation ability before retraining\n- Remember to voxel-downsample the data in your data loader, see ```kitti.py``` for reference \n\n### Citation\nIf you find this code useful for your work or use it in your project, please consider citing:\n\n```shell\n@InProceedings{Huang_2021_CVPR,\n    author    = {Huang, Shengyu and Gojcic, Zan and Usvyatsov, Mikhail and Wieser, Andreas and Schindler, Konrad},\n    title     = {Predator: Registration of 3D Point Clouds With Low Overlap},\n    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},\n    month     = {June},\n    year      = {2021},\n    pages     = {4267-4276}\n}\n```\n\n### Acknowledgments\nIn this project we use (parts of) the official implementations of the followin works: \n\n- [FCGF](https://github.com/chrischoy/FCGF) (KITTI preprocessing)\n- [D3Feat](https://github.com/XuyangBai/D3Feat.pytorch) (KPConv backbone)\n- [3DSmoothNet](https://github.com/zgojcic/3DSmoothNet) (3DMatch preparation)\n- [MultiviewReg](https://github.com/zgojcic/3D_multiview_reg) (3DMatch benchmark)\n- [SuperGlue](https://github.com/magicleap/SuperGluePretrainedNetwork) (Transformer part)\n- [DGCNN](https://github.com/WangYueFt/dgcnn) (self-gnn)\n- [RPMNet](https://github.com/yewzijian/RPMNet) (ModelNet preprocessing and evaluation)\n\n We thank the respective authors for open sourcing their methods. We would also like to thank reviewers, especially reviewer 2 for his/her valuable inputs. \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprs-eth%2Foverlappredator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprs-eth%2Foverlappredator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprs-eth%2Foverlappredator/lists"}