{"id":13442909,"url":"https://github.com/qq456cvb/CanonicalVoting","last_synced_at":"2025-03-20T15:31:25.828Z","repository":{"id":70755578,"uuid":"315586576","full_name":"qq456cvb/CanonicalVoting","owner":"qq456cvb","description":"Canonical Voting: Towards Robust Oriented Bounding Box Detection in 3D Scenes (CVPR2022)","archived":false,"fork":false,"pushed_at":"2022-07-09T13:37:54.000Z","size":5388,"stargazers_count":46,"open_issues_count":2,"forks_count":7,"subscribers_count":6,"default_branch":"main","last_synced_at":"2024-08-01T03:42:20.208Z","etag":null,"topics":["3d-object-detection","3d-scenes","cvpr2022","deep-learning","hough-transform","pytorch"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/qq456cvb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-11-24T09:48:07.000Z","updated_at":"2024-03-04T09:07:48.000Z","dependencies_parsed_at":null,"dependency_job_id":"5c45a834-e721-4b00-a341-b2d37b463bf9","html_url":"https://github.com/qq456cvb/CanonicalVoting","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qq456cvb%2FCanonicalVoting","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qq456cvb%2FCanonicalVoting/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qq456cvb%2FCanonicalVoting/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/qq456cvb%2FCanonicalVoting/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/qq456cvb","download_url":"https://codeload.github.com/qq456cvb/CanonicalVoting/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221772601,"owners_count":16878137,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","3d-scenes","cvpr2022","deep-learning","hough-transform","pytorch"],"created_at":"2024-07-31T03:01:53.174Z","updated_at":"2024-10-28T03:31:18.719Z","avatar_url":"https://github.com/qq456cvb.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\r\nCanonical Voting: Towards Robust Oriented Bounding Box Detection in 3D Scenes\r\n\u003c/h1\u003e\r\n\r\n\u003cp align='center'\u003e\r\n\u003cimg align=\"center\" src='images/intro.jpg' width='70%'\u003e \u003c/img\u003e\r\n\u003c/p\u003e\r\n\r\n\u003cdiv align=\"center\"\u003e\r\n\u003ch3\u003e\r\n\u003ca href=\"https://qq456cvb.github.io\"\u003eYang You\u003c/a\u003e, Zelin Ye, Yujing Lou, Chengkun Li, Yong-Lu Li, Lizhuang Ma, Weiming Wang, Cewu Lu\r\n\u003cbr\u003e\r\n\u003cbr\u003e\r\nCVPR 2022\r\n\u003cbr\u003e\r\n\u003cbr\u003e\r\n\u003ca href='https://arxiv.org/pdf/2011.12001.pdf'\u003e\r\n  \u003cimg src='https://img.shields.io/badge/Paper-PDF-orange?style=flat\u0026logo=arxiv\u0026logoColor=orange' alt='Paper PDF'\u003e\r\n\u003c/a\u003e\r\n\u003ca href='https://qq456cvb.github.io/projects/canonical-voting'\u003e\r\n  \u003cimg src='https://img.shields.io/badge/Project-Page-green?style=flat\u0026logo=googlechrome\u0026logoColor=green' alt='Project Page'\u003e\r\n\u003c/a\u003e\r\n\u003ca href='https://youtu.be/W7ZYEES5cLY'\u003e\r\n\u003cimg src='https://img.shields.io/badge/Youtube-Video-red?style=flat\u0026logo=youtube\u0026logoColor=red' alt='Video'/\u003e\r\n\u003c/a\u003e\r\n\u003cbr\u003e\r\n\u003c/h3\u003e\r\n\u003c/div\u003e\r\n\r\nCanonical Voting is a 3D detection method that disentangles Hough voting targets into Local Canonical Coordinates (LCC), box scales and box orientations. LCC and box scales are regressed for each point while box orientations are generated by a canonical voting scheme. Finally, a LCC-aware back-projection checking algorithm iteratively cuts out bounding boxes from the generated vote maps, with the elimination of false positives. Our model achieves state-of-the-art performance on challenging large-scale datasets of real point cloud scans: ScanNet, SceneNN and SUN RGB-D.\r\n\r\n# News\r\n- **[2022.03]** Our voting-based category-level 9D pose estimation method [CPPF](https://github.com/qq456cvb/CPPF), which achieves decent sim-to-real performance, is accepted to CVPR 2022!\r\n\r\n# Change Logs\r\n- [2022.04.11] Upload `Bathtub` fixed Scan2CAD annotations.\r\n- [2022.04.11] Update install dependencies to more recent versions.\r\n- [2022.04.22] Fix a bug in evaluating joint models.\r\n# Contents\r\n- [Overview](#overview)\r\n- [Installation](#installation)\r\n- [Train and Test on ScanNet](#train-and-test-on-scannet)\r\n- [Test on SceneNN](#test-on-scenenn)\r\n- [Train and Test on SUN RGB-D](#train-and-test-on-sun-rgb-d)\r\n- [Pretrained Models](#pretrained-models)\r\n- [Citation](#citation)\r\n\r\n# Overview\r\nThis is the official Pytorch implementation of our work: Canonical Voting.\r\n# Installation\r\n- [MinkowskiEngine](https://github.com/NVIDIA/MinkowskiEngine) v0.5.3\r\n- Install our custom Hough Voting module under `houghvoting` folder, by running `python setup.py install`\r\n- Tested with PyTorch v1.8.1 + CUDA 10.2\r\n- Other dependecies: \r\n```\r\npip install hydra-core==1.1.1 scipy scikit-learn tqdm shapely numpy-quaternion==2021.8.30.10.33.11 pickle plyfile\r\n```\r\n\r\n# Train and Test on ScanNet\r\n\u003cdetails\u003e\r\n\u003csummary\u003eData Preparation\u003c/summary\u003e\r\n\r\nYou will need to first download the original [ScanNet](https://github.com/ScanNet/ScanNet) dataset. For [Scan2CAD](https://github.com/skanti/Scan2CAD) labels with oriented bounding boxes, we removed some ambiguous Scan2CAD annotations for `Bathtub` (wordnet id: 02808440) category, including washbasins, washstands, etc. You can download our `Bathtub` fixed annotations on [Google Drive](https://drive.google.com/file/d/1-D4gvCcSIXKZGGmi1lHv91fqKq46sYJn/view?usp=sharing).\r\n\r\nDownload our annotated Scan2CAD model segments [here](https://drive.google.com/drive/folders/1yKIcQuJte9vToRLbZYgwdYqUDECBYs1T?usp=sharing) and preprocessed ground-truth boxes [here](https://drive.google.com/drive/folders/1i4ctu3oxwYG19kczqNgryj5uMnZVQZCv?usp=sharing) for evaluation. Adjust their path accordingly in `config/config.yaml`.\r\n\u003c/details\u003e\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003eStart Training\u003c/summary\u003e\r\n\r\nTo train model jointly for all categories, with one unified model:\r\n```\r\npython train_joint.py\r\n```\r\nTo train model separately for each category:\r\n```\r\npython train_separate.py category=03211117,04379243,02808440,02747177,04256520,03001627,02933112,02871439,others -m\r\n```\r\n\u003c/details\u003e\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003eEvaluate mAP\u003c/summary\u003e\r\n\r\nOnce trained, you can evaluate the model's mAP on ScanNet val set.\r\n\r\nTo eval the jointly trained model:\r\n```\r\npython eval_joint.py\r\n```\r\nTo eval the separately trained model:\r\n```\r\npython eval_separate.py\r\n```\r\n\u003c/details\u003e\r\n\r\n# Test on SceneNN\r\n\u003cdetails\u003e\r\n\u003csummary\u003eData Preparation\u003c/summary\u003e\r\n\r\nYou will need to download our processed [SceneNN](https://mega.nz/folder/n7hzDQxb#mV8t4d7psPYN5bSkkxHuYw) data, which contains raw segmentation labels, instance labels and bounding box annotations. Set `scene_nn_root` in `config.yaml` to your downloaded directory.\r\n\u003c/details\u003e\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003eEvaluate mAP\u003c/summary\u003e\r\n\r\nRun `eval_joint.py` or `eval_separate.py` with modified variable `SCENENN=True`.\r\n\u003c/details\u003e\r\n\r\n# Train and Test on SUN RGB-D\r\n\u003cdetails\u003e\r\n\u003csummary\u003eData Preparation\u003c/summary\u003e\r\n\r\nwe follow [BRNet](https://github.com/cheng052/BRNet) to prepare data for training and testing, while separately train a ``learned`` FPS proposal sampler as described in the paper. \r\n\u003c/details\u003e\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003eStart Training\u003c/summary\u003e\r\n\r\nFirst download the pretrained CanonicalVoting model on [Google Drive](https://drive.google.com/file/d/1-ZujySGPiLxzyu8OWsUkzxXsLHPisVMq/view?usp=sharing).\r\n\r\nTo reproduce the result, replace the original BRNet module with out BRNetCanon in `sunrgbd/brnetcanon.py`. Besides, change L88 and L95 of `configs/_base_/models/brnet.py` to `sample_mod='custom'`; and change L11 of `configs/_base_/schedules/schedule_cos.py` to `total_epochs=72` since changing sampling strategy takes more epochs to converge.\r\n\r\n\r\n\u003c/details\u003e\r\n\r\n# Pretrained Models\r\n\u003cdetails\u003e\r\n\u003csummary\u003ePretrained Model on ScanNet\u003c/summary\u003e\r\n\r\nPretrained models for both joint and separate training settings can be found [here](https://drive.google.com/drive/folders/1Af5mRVwwI370txOREXkooea8nK_SwzGk?usp=sharing). You will get about 15.4 mAP and 21.7 mAP for joint and separate training settings, respectively.\r\n\u003c/details\u003e\r\n\r\n\u003cdetails\u003e\r\n\u003csummary\u003ePretrained Model on SUN RGB-D\u003c/summary\u003e\r\n\r\nPretrained CanonicalVoting model can be found [here](https://drive.google.com/file/d/1-ZujySGPiLxzyu8OWsUkzxXsLHPisVMq/view?usp=sharing).\r\n\u003c/details\u003e\r\n\r\n# Citation\r\nIf you find our algorithm useful or use our processed data, please consider citing:\r\n```\r\n@article{you2022canonical,\r\n  title={Canonical Voting: Towards Robust Oriented Bounding Box Detection in 3D Scenes},\r\n  author={You, Yang and Ye, Zelin and Lou, Yujing and Li, Chengkun and Li, Yong-Lu and Ma, Lizhuang and Wang, Weiming and Lu, Cewu},\r\n  journal={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\r\n  year={2022}\r\n}\r\n```\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqq456cvb%2FCanonicalVoting","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fqq456cvb%2FCanonicalVoting","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fqq456cvb%2FCanonicalVoting/lists"}