{"id":13443591,"url":"https://github.com/AutoVision-cloud/Deformable-PV-RCNN","last_synced_at":"2025-03-20T16:32:13.079Z","repository":{"id":119122385,"uuid":"289382259","full_name":"AutoVision-cloud/Deformable-PV-RCNN","owner":"AutoVision-cloud","description":"[ECCVW-2020] Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations","archived":false,"fork":false,"pushed_at":"2022-05-05T22:59:43.000Z","size":524,"stargazers_count":102,"open_issues_count":0,"forks_count":12,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-08-01T03:44:04.402Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AutoVision-cloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-08-21T23:13:37.000Z","updated_at":"2024-06-21T14:53:04.000Z","dependencies_parsed_at":null,"dependency_job_id":"4ee639dd-f538-42b4-a4da-9282acb6a280","html_url":"https://github.com/AutoVision-cloud/Deformable-PV-RCNN","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutoVision-cloud%2FDeformable-PV-RCNN","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutoVision-cloud%2FDeformable-PV-RCNN/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutoVision-cloud%2FDeformable-PV-RCNN/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AutoVision-cloud%2FDeformable-PV-RCNN/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AutoVision-cloud","download_url":"https://codeload.github.com/AutoVision-cloud/Deformable-PV-RCNN/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221780085,"owners_count":16879040,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:02:04.573Z","updated_at":"2024-10-28T04:31:24.604Z","avatar_url":"https://github.com/AutoVision-cloud.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations\n\nBy [Prarthana Bhattacharyya](https://scholar.google.com/citations?user=v6pGkNQAAAAJ\u0026hl=en) and [Krzysztof Czarnecki](https://scholar.google.com/citations?hl=en\u0026user=ZzCpumQAAAAJ).\n\nWe provide code support and configuration files to reproduce the results in the paper for\n[\"Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations\"](https://arxiv.org/abs/2008.08766) on KITTI 3D object detection. Our code is based on [OpenPCDet](https://github.com/open-mmlab/OpenPCDet), which is a clean open-sourced project for benchmarking 3D object detection methods. \n\n\n## Introduction\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/introduction.png\" width=\"450px\" height=\"150px\" /\u003e\n  \u003cp\u003eLearning Deformations in 3D Object Detection.\u003c/p\u003e\n\u003c/div\u003e\n\n**Deformable PV-RCNN** is a high-performing point-cloud based 3D object detector. Currently, the proposal refinement methods used by the state-of-the-art two-stage detectors cannot adequately accommodate differing object scales, varying point-cloud density, part-deformation and clutter. We present a proposal refinement module inspired by 2D deformable convolution networks that can adaptively gather instance-specific features from locations where informative content exists. We also propose a simple context gating mechanism which allows the keypoints to select relevant context information for the refinement stage. We outperform previously published methods on the highly competitive KITTI 3D Object Detection benchmark on cars and cyclists, and are the leading method among point-cloud based approaches on the Orientation Estimation benchmark for pedestrians and cyclists.\n\n\n## Demo\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/demo.png\" width=\"700px\" /\u003e\n  \u003cp\u003eVisualizing PV-RCNN and Deformable PV-RCNN results.\u003c/p\u003e\n\u003c/div\u003e\n\n\n## Usage\na. Clone the repo:\n```\ngit clone --recursive https://github.com/AutoVision-cloud/Deformable-PV-RCNN\n```\nb. Copy Deformable PV-RCNN src into OpenPCDet: \n```\nsh ./init.sh\n```\n\nc. Install OpenPCDet and prepare KITTI data:\n\nPlease refer to [INSTALL.md](docs/INSTALL.md) for installation and dataset preparation.\n\nd. Run experiments with a specific configuration file:\n\nPlease refer to [GETTING_STARTED.md](docs/GETTING_STARTED.md) to learn more about how to train and run inference on this detector.\n\n\n## Results and models\n\nThe 3D AP results on KITTI 3D Object Detection validation split of the moderate category are shown in the table below.\n\n|                                                     | Car | Pedestrian | Cyclist  | download | \n|-----------------------------------------------------|:-------:|:-------:|:-------:|:---------:|\n| [PV-RCNN](https://github.com/open-mmlab/OpenPCDet/blob/b32fbddbe06183507bad433ed99b407cbc2175c2/tools/cfgs/kitti_models/pv_rcnn.yaml) | **83.69** | 54.84 | 69.47 | [model-PV-RCNN](https://drive.google.com/file/d/1CXK7LVGU9jPRcygrDReQWhpwax9BJ9hb/view?usp=sharing) |\n| [Deformable PV-RCNN-v1](config/def_pv_rcnn.yaml) | 83.30 | 58.33 | 73.46 | [model-def-PV-RCNN](https://drive.google.com/file/d/18YpEEViDFjKdxhTFxo7mdGWdopCMZ28j/view?usp=sharing) |\n| [Deformable PV-RCNN-v2](config/def_pv_rcnn.yaml) | 83.31 | **59.31** | **74.18** | [model-def-PV-RCNN-all](https://drive.google.com/file/d/1qLr3IIGftvGCzb_mA283wJAFBfqA4sfI/view?usp=sharing) |\n\n**Notes:**\n- Our models are trained with Pytorch 1.5, 8 GTX 1080Ti GPUs and are available for download.\n- `v1` denotes using Conv3 and Conv4 features for deformations.\n- `v2` denotes using all convolutional features for deformations. The current configurations are set for `v2`. \n- To run inference with `v1`, remove `x_conv1` and `x_conv2` from `MODEL.PFE.DEF_SOURCE` in the config file.\n\n\n\nThe results on KITTI 3D Object Detection [test split](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) are shown in the table below.\n\n|                                                     |  FPS | Car | Pedestrian | Cyclist  | 3D mAP\n|-----------------------------------------------------|:---------:|:-------:|:-------:|:-------:|:-------:|\n| [PV-RCNN](https://github.com/open-mmlab/OpenPCDet/blob/b32fbddbe06183507bad433ed99b407cbc2175c2/tools/cfgs/kitti_models/pv_rcnn.yaml) | 13 |  81.43 | **43.29** | 63.71 | 62.81\n| [Deformable PV-RCNN](config/def_pv_rcnn.yaml) | 12 | **81.46** | 40.89 | **68.54** | **63.63**\n\nThe results on KITTI Orientation Estimation benchmark are shown in the table below.\n\n|                                                     |  Car | Pedestrian | Cyclist  | mAOS\n|-----------------------------------------------------|:-------:|:-------:|:-------:|:-------:|\n| [PV-RCNN](https://github.com/open-mmlab/OpenPCDet/blob/b32fbddbe06183507bad433ed99b407cbc2175c2/tools/cfgs/kitti_models/pv_rcnn.yaml) | **94.57** | 52.42 | 79.70 | 75.56\n| [Deformable PV-RCNN](config/def_pv_rcnn.yaml) | 94.52 | **54.38** | **80.05** | **76.32**\n\n\n\n\n## Citing Deformable PV-RCNN\n\n```\n@misc{bhattacharyya2020deformable,\n    title={Deformable PV-RCNN: Improving 3D Object Detection with Learned Deformations},\n    author={Prarthana Bhattacharyya and Krzysztof Czarnecki},\n    year={2020},\n    eprint={2008.08766},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n}\n```\n\n\n## Acknowledgement\n* [second.pytorch](https://github.com/traveller59/second.pytorch)\n* [PointRCNN](https://github.com/sshaoshuai/PointRCNN)\n* [PV-RCNN](https://github.com/open-mmlab/OpenPCDet)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAutoVision-cloud%2FDeformable-PV-RCNN","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAutoVision-cloud%2FDeformable-PV-RCNN","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAutoVision-cloud%2FDeformable-PV-RCNN/lists"}