{"id":13442858,"url":"https://github.com/CVMI-Lab/FS3D","last_synced_at":"2025-03-20T15:31:21.678Z","repository":{"id":113401420,"uuid":"544904429","full_name":"CVMI-Lab/FS3D","owner":"CVMI-Lab","description":"(NeurlPS 2022) Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detection","archived":false,"fork":false,"pushed_at":"2023-01-03T14:05:20.000Z","size":13404,"stargazers_count":55,"open_issues_count":3,"forks_count":6,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-28T06:00:02.597Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/CVMI-Lab.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-10-03T12:45:37.000Z","updated_at":"2024-06-28T16:24:58.000Z","dependencies_parsed_at":"2023-07-24T07:01:29.958Z","dependency_job_id":null,"html_url":"https://github.com/CVMI-Lab/FS3D","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVMI-Lab%2FFS3D","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVMI-Lab%2FFS3D/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVMI-Lab%2FFS3D/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/CVMI-Lab%2FFS3D/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/CVMI-Lab","download_url":"https://codeload.github.com/CVMI-Lab/FS3D/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244640021,"owners_count":20485966,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:52.244Z","updated_at":"2025-03-20T15:31:21.666Z","avatar_url":"https://github.com/CVMI-Lab.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detection\n\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://nips.cc/virtual/2022/poster/55053\"\u003e\u003cimg src=\"https://img.shields.io/badge/-NeurIPS%202022-68488b\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://arxiv.org/abs/2210.05593\"\u003e\u003cimg src=\"https://img.shields.io/badge/arXiv-2210.05593-b31b1b\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/CVMI-Lab/SlotCon/blob/master/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/License-Apache%202.0-blue.svg\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n\tPrototypical VoteNet for Few-Shot 3D Point Cloud Object Detection (NeurIPS 2022)\u003cbr\u003e\n  By\n  Shizhen Zhao\n  and \n  Xiaojuan Qi.\n\u003c/p\u003e\n\n## Introduction\n\nMost existing 3D point cloud object detection approaches heavily rely on large amounts of labeled training data. However, the labeling process is costly and time-consuming. This paper considers few-shot 3D point cloud object detection, where only a few annotated samples of novel classes are needed with abundant samples of base classes. To this end, we propose Prototypical VoteNet to recognize and localize novel instances, which incorporates two new modules: Prototypical Vote Module (PVM) and Prototypical Head Module (PHM). Specifically, as the 3D basic geometric structures can be shared among categories, PVM is designed to leverage class-agnostic geometric prototypes, which are learned from base classes, to refine local features of novel categories.Then PHM is proposed to utilize class prototypes to enhance the global feature of each object, facilitating subsequent object localization and classification, which is trained by the episodic training strategy. To evaluate the model in this new setting, we contribute two new benchmark datasets, FS-ScanNet and FS-SUNRGBD. We conduct extensive experiments to demonstrate the effectiveness of Prototypical VoteNet, and our proposed method shows significant and consistent improvements compared to baselines on two benchmark datasets.\n\n\u003cp align=\"center\"\u003e\n\t\u003cimg src=pic/method.png width=80% /\u003e\n\u003cp align=\"center\"\u003e\n\n\t\n\n## Environments\n\nPlease make sure that you have installed all dependencies. Our implementation has been tested on one NVIDIA 3090 GPU with cuda 11.2.\n\n**Step 1.** (Create virtual env using conda)\n```\nconda create --name prototypical_votenet python=3.8 -y\nconda activate prototypical_votenet\n```\n\n**Step 2.** (Intall [Pytorch](https://pytorch.org/))\n```\npip3 install torch==1.10.1+cu113 torchvision==0.11.2+cu113 torchaudio===0.10.1+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html\n```\nYou may change the above command according to your cuda version. Please refer to [official website](https://pytorch.org/) of Pytorch. \n\n\n**Step 3.** (Install mmdet, mmcv and mmsegmentation)\n```\npip install mmdet==2.19.0\npip install mmcv-full==1.3.18 -f https://download.openmmlab.com/mmcv/dist/cu113/torch1.10.0/index.html\npip install mmsegmentation==0.20.0\n```\nYou may change the above command according to your Pytorch version. Please refer to [official website](https://github.com/open-mmlab/mmdetection3d/blob/master/docs/en/getting_started.md) of MMDetection3D. \n\n\n**Step 4.** (Setup the code base in your env)\n```\npip install setuptools==58.0.4\npip install -v -e .\n```\n\n\n## Dataset Preparation\n\nRequest the datasets (FS-ScanNet and FS-SUNRGBD) from zhaosz@eee.hku.hk (academic only). Due to licensing issues, please send me your request using your university email.\n\nAfter downloading FS-ScanNet and FS-SUNRGBD, you should unzip and put it in your project folder. The datasets have been processed so that you can directly use them to train your own models. \n\n\n## Train and Test the Model\n\nFor example, train and test the model on FS-SUNRGBD 1-shot, 2-shot, 3-shot, 4-shot, and 5-shot. \n```\nCUDA_VISIBLE_DEVICES=0 python tools/train.py \\\n./configs/prototypical_votenet/train_together_sun/prototypical_votenet_16x8_sunrgbd-3d-10class_1_1.py --sample_num 16 --work_path work_path/sunrgbd_split1_shot1_1\n\nCUDA_VISIBLE_DEVICES=1 python tools/train.py \\\n./configs/prototypical_votenet/train_together_sun/prototypical_votenet_16x8_sunrgbd-3d-10class_1_2.py --sample_num 16 --work_path work_path/sunrgbd_split1_shot2_1\n\nCUDA_VISIBLE_DEVICES=2 python tools/train.py \\\n./configs/prototypical_votenet/train_together_sun/prototypical_votenet_16x8_sunrgbd-3d-10class_1_3.py --sample_num 16 --work_path work_path/sunrgbd_split1_shot3_1\n\nCUDA_VISIBLE_DEVICES=3 python tools/train.py \\\n./configs/prototypical_votenet/train_together_sun/prototypical_votenet_16x8_sunrgbd-3d-10class_1_4.py --sample_num 16 --work_path work_path/sunrgbd_split1_shot4_1\n\nCUDA_VISIBLE_DEVICES=4 python tools/train.py \\\n./configs/prototypical_votenet/train_together_sun/prototypical_votenet_16x8_sunrgbd-3d-10class_1_5.py --sample_num 16 --work_path work_path/sunrgbd_split1_shot5_1\n```\n\nYou can find the commands in the folder named \"tools/\".\n\n## ModelZoo\n### FS-ScanNet \n#### Split-1\n\n| 1-shot             | 3-shot             | 5-shot             |\n|--------------------|--------------------|--------------------|\n| [model](https://drive.google.com/file/d/1WTnSUCdJu_dkwb7jjLcoR4791kgyUChW/view?usp=sharing) | [model](https://drive.google.com/file/d/1oJAS1x91ICPCNEEFvOKRFnO1TwOjWNXA/view?usp=sharing) | [model](https://drive.google.com/file/d/1QtBBrdyAxKiUBP3ZhGB8CRCDkRWXUF4H/view?usp=sharing)|\n\n#### Split-2\n\n| 1-shot             | 3-shot             | 5-shot             |\n|--------------------|--------------------|--------------------|\n| [model](https://drive.google.com/file/d/1UVCJRm-ABbqTcWt5g4r8EqfuANUjFpYT/view?usp=sharing) | [model](https://drive.google.com/file/d/1hAXfdyYE4zD5vaTZf_gWGBfi1pGA3XJF/view?usp=sharing)  | [model](https://drive.google.com/file/d/1B_oN9ogbmGIN-he6dgKbOkPS86Eyuqpi/view?usp=sharing) |\n\n### FS-SUNRGBD\n\n| 1-shot             | 2-shot             | 3-shot             | 4-shot             | 5-shot             |\n|--------------------|--------------------|--------------------|--------------------|--------------------|\n| [model](https://drive.google.com/file/d/16omLmv1laapqv4mGrER3LEPzeFerG7FI/view?usp=sharing)| [model](https://drive.google.com/file/d/1ohbk8efkYrMjOlBCHx_U-SXOyStrU5Pf/view?usp=sharing) | [model](https://drive.google.com/file/d/1h9OhzPwgR9xoVnw8-59udW23-u06M-iO/view?usp=sharing)| [model](https://drive.google.com/file/d/1qpUVrTsUEzNhWOsshUu-GSCb6rtbPCDy/view?usp=sharing)  | [model](https://drive.google.com/file/d/1N1pEqmcfC2bIC1GFuOl94HnOdRKgYq3q/view?usp=sharing) |\n\n## Citation\nPlease consider :grimacing: staring this repository and citing the following paper if you feel this repository useful. \n\n```\n@inproceedings{zhao2022fs3d,\n  title={Prototypical VoteNet for Few-Shot 3D Point Cloud Object Detection},\n  author={Zhao, Shizhen and Qi, Xiaojuan},\n  booktitle={Advances in Neural Information Processing Systems},\n  year={2022}\n}\n```\n\n\n## Acknowledgement\n\nOur code is largely based on [MMDetection3D](https://github.com/open-mmlab/mmdetection3d), and we thank the authors for their implementation. Please also consider citing their wonderful code base. \n\n```\n@misc{mmdet3d2020,\n    title={{MMDetection3D: OpenMMLab} next-generation platform for general {3D} object detection},\n    author={MMDetection3D Contributors},\n    howpublished = {\\url{https://github.com/open-mmlab/mmdetection3d}},\n    year={2020}\n}\n```\n\n## License\nThis project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.\n\t\n## Contact\nIf you have any questions, you can email me (zhaosz@eee.hku.hk). \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCVMI-Lab%2FFS3D","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FCVMI-Lab%2FFS3D","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FCVMI-Lab%2FFS3D/lists"}