{"id":13442876,"url":"https://github.com/Haiyang-W/CAGroup3D","last_synced_at":"2025-03-20T15:31:20.309Z","repository":{"id":63396225,"uuid":"547742673","full_name":"Haiyang-W/CAGroup3D","owner":"Haiyang-W","description":"[NeurIPS2022] This is the official code of \"CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds\".  ","archived":false,"fork":false,"pushed_at":"2023-05-31T06:40:49.000Z","size":7543,"stargazers_count":91,"open_issues_count":1,"forks_count":9,"subscribers_count":7,"default_branch":"main","last_synced_at":"2024-10-28T06:00:04.298Z","etag":null,"topics":["3d-object-detection","pytorch","scannet-dataset","sunrgbd"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Haiyang-W.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-10-08T07:48:17.000Z","updated_at":"2024-10-22T11:41:51.000Z","dependencies_parsed_at":"2024-01-18T14:41:23.699Z","dependency_job_id":"d58f2877-6f77-4b13-b209-1bb82ae55844","html_url":"https://github.com/Haiyang-W/CAGroup3D","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haiyang-W%2FCAGroup3D","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haiyang-W%2FCAGroup3D/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haiyang-W%2FCAGroup3D/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Haiyang-W%2FCAGroup3D/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Haiyang-W","download_url":"https://codeload.github.com/Haiyang-W/CAGroup3D/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244640025,"owners_count":20485967,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","pytorch","scannet-dataset","sunrgbd"],"created_at":"2024-07-31T03:01:52.562Z","updated_at":"2025-03-20T15:31:19.541Z","avatar_url":"https://github.com/Haiyang-W.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/cagroup3d-class-aware-grouping-for-3d-object/3d-object-detection-on-scannetv2)](https://paperswithcode.com/sota/3d-object-detection-on-scannetv2?p=cagroup3d-class-aware-grouping-for-3d-object)\n[![PWC](https://img.shields.io/endpoint.svg?url=https://paperswithcode.com/badge/cagroup3d-class-aware-grouping-for-3d-object/3d-object-detection-on-sun-rgbd-val)](https://paperswithcode.com/sota/3d-object-detection-on-sun-rgbd-val?p=cagroup3d-class-aware-grouping-for-3d-object)\n\n# CAGroup3D\n\nThis repo is the official implementation of the paper:\n#### CAGroup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds\n[PaperLink](https://arxiv.org/abs/2210.04264)\n\n\u003cimg src=\"CAGroup3D.jpg\"\u003e\n\n## NEWS\n- Official implementation based on [OpenPCDet](https://github.com/open-mmlab/OpenPCDet) is released.\n- 🔥 CAGroup3D is accepted at NeurIPS 2022.\n\n## TODO\n\n- [x] Implement CAGroup3D on OpenPCDet\n- [ ] Fix the bug of slow inference (180ms-\u003e400ms) in our OpenPCDet version. Paper report is mmdet3d version.\n\n## Introduction\nThis paper presents a novel two-stage fully sparse convolutional 3D object detection framework, named CAGroup3D. The proposed method first generates some high-quality 3D proposals by leveraging the class-aware local group strategy on the object surface voxels with the same semantic predictions, which considers semantic consistency and diverse locality abandoned in previous bottom-up approaches. Then, to recover the features of missed voxels due to incorrect voxel-wise segmentation, we build a fully sparse convolutional RoI pooling module to directly aggregate fine-grained spatial information from backbone for further proposal refinement.\n## Requirements\nThe code is tested on the following environment:\n\n- Unbuntu 18.04\n- Python 3.7\n- Pytorch 1.10\n- CUDA 11.1\n## Installation\n\n- Clone this repo and install the `pcdet` library\n```bash\ngit clone https://github.com/Haiyang-W/CAGroup3D.git\n# install spconv\npip install spconv-cu113\ncd CAGroup3D/\npython setup.py develop\n# if you meet some pakage not matched errors, just pip install them individually before install pcdet\n```\n\n- Compile additional CUDA ops\n```bash\n# rotate iou ops\ncd CAGroup3D/pcdet/ops/rotated_iou/cuda_op\npython setup.py install\n# knn ops\ncd ../../knn\npython setup.py develop\n```\n\n- Install [MinkowskiEngine](https://github.com/NVIDIA/MinkowskiEngine)\n```bash\napt-get install -y python3-dev libopenblas-dev\npip install ninja==1.10.2.3\npip install \\\n  -U git+https://github.com/NVIDIA/MinkowskiEngine@v0.5.4 \\\n  --install-option=\"--blas=openblas\" \\\n  --install-option=\"--force_cuda\" \\\n  -v \\\n  --no-deps\n# Or if you can not install MinkowskiEngine with pip sucessfully,\ngit clone https://github.com/NVIDIA/MinkowskiEngine.git\ncd MinkowskiEngine\npython setup.py install --blas=openblas --force_cuda\n```\n\n## Data preparation\nWe haven't achieved compatibility with the generated data of OpenPCDet yet and use the same data format as [MMdeteciton3d](https://github.com/open-mmlab/mmdetection3d) for now. We will try to implement indoor data pre-processing based on OpenPCDet as soon as possible.\n- follow  [MMdetection3D(0.15)](https://github.com/open-mmlab/mmdetection3d) to create data (ScanNetV2, SunRGBD). Note that the coordinate system adopted in MMdetection3D(0.15) is very different from MMdetection3D(\u003e=1.0). We also provide processed data in [GoogleDrive](https://drive.google.com/drive/folders/1sKvq4WBSEb4CWMdCTN6lCHLXnn3NwUv_) (highly recommended). For people from mainland China, we also provide [BaiduDrive](https://pan.baidu.com/s/1SO1YhvC9BZh5eFXYjt-a3g?pwd=8mfk).\n\n\n- remember to modify the `DATA_PATH` in **tools/cfgs/dataset_configs/scannet_dataset.yaml**, **sunrgbd_dataset.yaml** or link the generated data as follows:\n```shell\nln -s ${mmdet3d_scannet_dir} ./CAGroup3D/data/scannet\nln -s ${mmdet3d_sunrgbd_dir} ./CAGroup3D/data/sunrgbd\n``` \n\n## Get started\n### ScanNetV2\n- Training, `num_gpus` x `batch_size` can be set to 2x8 or 4x4.\n```bash\ncd tools/\nCUDA_VISIBLE_DEVICES={} ./scripts/dist_train.sh {num_gpus} --cfg_file cfgs/scannet_models/CAGroup3D.yaml --ckpt_save_interval 1 --extra_tag {your name} --fix_random_seed\n```\n\n- Testing\n```bash\ncd tools/\n# test single pth\npython test.py --cfg_file cfgs/scannet_models/CAGroup3D.yaml --ckpt {your pth}\n# or test all checkpoints\npython test.py --cfg_file cfgs/scannet_models/CAGroup3D.yaml --extra_tag {your name} --eval_all\n# dist test is also supported\nCUDA_VISIBLE_DEVICES={} ./scripts/dist_test.sh {num_gpus} --cfg_file cfgs/scannet_models/CAGroup3D.yaml --ckpt {your pth}\n```\n### Sun RGB-D\n- Training, `num_gpus` x `batch_size` can be set to 2x8 or 4x4.\n```bash\ncd tools/\nCUDA_VISIBLE_DEVICES={} ./scripts/dist_train.sh {num_gpus} --cfg_file cfgs/sunrgbd_models/CAGroup3D.yaml --ckpt_save_interval 1 --extra_tag {your name} --fix_random_seed\n```\n\n- Testing\n```bash\ncd tools/\n# test single pth\npython test.py --cfg_file cfgs/sunrgbd_models/CAGroup3D.yaml --ckpt {your pth}\n# or test all checkpoints\npython test.py --cfg_file cfgs/sunrgbd_models/CAGroup3D.yaml --extra_tag {your name} --eval_all\n# dist test is also supported\nCUDA_VISIBLE_DEVICES={} ./scripts/dist_test.sh {num_gpus} --cfg_file cfgs/sunrgbd_models/CAGroup3D.yaml --ckpt {your pth}\n```\n\n### Main Results\nWe reproduce CAGroup3D based on [OpenPCDet](https://github.com/open-mmlab/OpenPCDet)(Paper reported is MMDet3D version). All models are trained with 4 3090 GPUs. The batch size of each card is 4 (2x8 is better). Please note that the pcdet version of scannet pretrained model has a higher mAP@50 (61.1 vs 60.3), but a lower mAP@25 (74.0 vs 74.5) than the original paper. Additionally, the sunrgbd pretrained model has a higher mAP@25 (67.1 vs 66.4) than the original paper. Since these two datasets fluctuate significantly, training more times should yield better results than the pre-trained models we provide.\n\n|   Dataset | mAP@0.25 | mAP0.50 | mAP@0.25(repro) | mAP0.50(repro) | Pretrain Model \u0026\u0026 Log |\n|----------|----------:|:-------:|:-------:|:-------:|:-------:|\n| [ScanNet](tools/cfgs/scannet_models/CAGroup3D.yaml) | 74.5  |\t60.3 | 74.0 | 61.1 | [model](https://drive.google.com/drive/folders/1lhd3kx-G-6vMVj51Ryjp20cibwOD1txq), [log](https://drive.google.com/drive/folders/1lhd3kx-G-6vMVj51Ryjp20cibwOD1txq) |\n| [Sun RGB-D](tools/cfgs/sunrgbd_models/CAGroup3D.yaml) | 66.4   |\t49.5 | 67.1| 49.1| [model](https://drive.google.com/drive/folders/1mSmAxzHxEvXLd3IpxBsycdx9aigl7Wb3), [log](https://drive.google.com/drive/folders/1mSmAxzHxEvXLd3IpxBsycdx9aigl7Wb3) |\n\n## Citation\nPlease consider citing our work as follows if it is helpful.\n```\n@inproceedings{\nwang2022cagroupd,\ntitle={{CAG}roup3D: Class-Aware Grouping for 3D Object Detection on Point Clouds},\nauthor={Haiyang Wang and Lihe Ding and Shaocong Dong and Shaoshuai Shi and Aoxue Li and Jianan Li and Zhenguo Li and Liwei Wang},\nbooktitle={Advances in Neural Information Processing Systems},\neditor={Alice H. Oh and Alekh Agarwal and Danielle Belgrave and Kyunghyun Cho},\nyear={2022},\nurl={https://openreview.net/forum?id=nLKkHwYP4Au}\n}\n```\n\n## Acknowledgments\nThis project is based on the following codebases.\n* [OpenPCDet](https://github.com/open-mmlab/OpenPCDet)\n* [FCAF3D](https://github.com/SamsungLabs/fcaf3d)\n* [RBGNet](https://github.com/Haiyang-W/RBGNet)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHaiyang-W%2FCAGroup3D","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHaiyang-W%2FCAGroup3D","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHaiyang-W%2FCAGroup3D/lists"}