{"id":13442647,"url":"https://github.com/Luoyadan/CRB-active-3Ddet","last_synced_at":"2025-03-20T14:31:48.886Z","repository":{"id":161356676,"uuid":"493888546","full_name":"Luoyadan/CRB-active-3Ddet","owner":"Luoyadan","description":"The official PyTorch implementation of \"Exploring Active 3D Object Detection from a Generalization Perspective\" (ICLR Spotlight 2023).","archived":false,"fork":false,"pushed_at":"2023-11-13T11:59:40.000Z","size":8292,"stargazers_count":55,"open_issues_count":10,"forks_count":7,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-10-28T05:59:31.547Z","etag":null,"topics":["3d-detection","active-learning"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Luoyadan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-05-19T02:08:18.000Z","updated_at":"2024-10-25T01:14:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"296b1095-3fe0-410a-9efa-3104cc22963c","html_url":"https://github.com/Luoyadan/CRB-active-3Ddet","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Luoyadan%2FCRB-active-3Ddet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Luoyadan%2FCRB-active-3Ddet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Luoyadan%2FCRB-active-3Ddet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Luoyadan%2FCRB-active-3Ddet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Luoyadan","download_url":"https://codeload.github.com/Luoyadan/CRB-active-3Ddet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244630286,"owners_count":20484348,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-detection","active-learning"],"created_at":"2024-07-31T03:01:48.567Z","updated_at":"2025-03-20T14:31:48.265Z","avatar_url":"https://github.com/Luoyadan.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\n\n\u003cp align=\"center\"\u003e\n  \u003cimg height=\"150\" src=\"./miscellaneous/active-3d-logo.png\" /\u003e\n\u003c/p\u003e\n\n-----------------------\n\n\nThis repository is the official Pytorch implementation of our work:\n\n\\[ICLR 2023\\] **CRB: Exploring Active 3D Object Detection from a Generalization Perspective**. \n\n[[OpenReview]](https://openreview.net/forum?id=2RwXVje1rAh) [[arXiv]](https://arxiv.org/abs/2301.09249) [[Supplementary Material]](https://openreview.net/attachment?id=2RwXVje1rAh\u0026name=supplementary_material)\n\n\\[In Submission\\] **Open-CRB: Towards Open World Active Learning for 3D Object Detection**.\n\n[[Open-CRB Branch]](https://github.com/Luoyadan/CRB-active-3Ddet/tree/Open-CRB) [[arXiv]](https://arxiv.org/abs/2310.10391) \n\n:fire: 11/23 updates: release the code and the preprint of **Open-CRB**\n\n:fire: 02/23 updates: checkpoints available at https://drive.google.com/drive/folders/1PMb6tu84AIw66vCRrMBCHpnBeL5WMkuv?usp=sharing\n\n## Framework\nTo alleviate the high annotation cost in LiDAR-based 3D object detection, active learning is a promising solution that learns to select only a small portion of unlabeled data to annotate, without compromising model performance. Our empirical study, however, suggests that mainstream uncertainty-based and diversity-based active learning policies are not effective when applied in the 3D detection task, as they fail to balance the trade-off between point cloud informativeness and box-level annotation costs. To overcome this limitation, we jointly investigate three novel criteria in our framework **CRB** for point cloud acquisition - label conciseness, feature representativeness and geometric balance, which hierarchically filters out the point clouds of redundant 3D bounding box labels, latent features and geometric characteristics (e.g., point cloud density) from the unlabeled sample pool and greedily selects informative ones with fewer objects to annotate. Our theoretical analysis demonstrates that the proposed criteria aligns the marginal distributions of the selected subset and the prior distributions of the unseen test set, and minimizes the upper bound of the generalization error. To validate the effectiveness and applicability of CRB, we conduct extensive experiments on the two benchmark 3D object detection datasets of KITTI and Waymo and examine both one-stage (i.e., SECOND) and two-stage 3D detectors (i.e., PV-RCNN). Experiments evidence that the proposed approach outperforms existing active learning strategies and achieves fully supervised performance requiring 1\\% and 8\\% annotations of bounding boxes and point clouds, respectively.\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"miscellaneous/flowchart.png\" width=\"70%\"\u003e\n\u003c/p\u003e\n\n\n\n----\n## Contents\n* [Installation](#Installation)\n  * [Requirements](#Requirements)\n  * [pcdet v0.5](#install-pcdet-v05)\n* [Getting Started](#getting-started)\n  * [Requirements](#Requirements)\n  * [KITTI Dataset](#KITTI-Dataset)\n  * [Waymo Open Dataset](#Waymo-Open-Dataset)\n  \u003c!-- * [Lyft Dataset](#Lyft-Dataset) --\u003e\n  * [Training \u0026 Testing](#training--testing)\n\n\n# Installation\n\n### Requirements\nAll the codes are tested in the following environment:\n* Python 3.6+\n* PyTorch 1.10.1\n* CUDA 11.3 \n* wandb 0.12.11\n* [`spconv-cu113 v2.1.21`](https://github.com/traveller59/spconv)\n\n\n### Install `pcdet v0.5`\nOur implementations of 3D detectors are based on the lastest [`OpenPCDet`](https://github.com/open-mmlab/OpenPCDet). To install this `pcdet` library and its dependent libraries, please run the following command:\n\n```shell\npython setup.py develop\n```\n\u003e NOTE: Please re-install even if you have already installed pcdet previoursly.\n\n\n# Getting Started\nThe **active learning configs** are located at [tools/cfgs/active-kitti_models](./tools/cfgs/active-kitti_models) and [/tools/cfgs/active-waymo_models](./tools/cfgs/active-waymo_models) for different AL methods. The dataset configs are located within [tools/cfgs/dataset_configs](./tools/cfgs/dataset_configs), \nand the model configs are located within [tools/cfgs](./tools/cfgs) for different datasets. \n\n\n## Dataset Preparation\nCurrently we provide the dataloader of KITTI dataset and Waymo dataset, and the supporting of more datasets are on the way.  \n\n### KITTI Dataset\n* Please download the official [KITTI 3D object detection](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) dataset and organize the downloaded files as follows (the road planes could be downloaded from [[road plane]](https://drive.google.com/file/d/1d5mq0RXRnvHPVeKx6Q612z0YRO1t2wAp/view?usp=sharing), which are optional for data augmentation in the training):\n* If you would like to train [CaDDN](../tools/cfgs/kitti_models/CaDDN.yaml), download the precomputed [depth maps](https://drive.google.com/file/d/1qFZux7KC_gJ0UHEg-qGJKqteE9Ivojin/view?usp=sharing) for the KITTI training set\n* NOTE: if you already have the data infos from `pcdet v0.1`, you can choose to use the old infos and set the DATABASE_WITH_FAKELIDAR option in tools/cfgs/dataset_configs/kitti_dataset.yaml as True. The second choice is that you can create the infos and gt database again and leave the config unchanged.\n\n```\nCRB-active-3Ddet\n├── data\n│   ├── kitti\n│   │   │── ImageSets\n│   │   │── training\n│   │   │   ├──calib \u0026 velodyne \u0026 label_2 \u0026 image_2 \u0026 (optional: planes) \u0026 (optional: depth_2)\n│   │   │── testing\n│   │   │   ├──calib \u0026 velodyne \u0026 image_2\n├── pcdet\n├── tools\n```\n\n* Generate the data infos by running the following command: \n```python \npython -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml\n```\n\n\u003c!-- ### NuScenes Dataset\n* Please download the official [NuScenes 3D object detection dataset](https://www.nuscenes.org/download) and \norganize the downloaded files as follows: \n```\nCRB-active-3Ddet\n├── data\n│   ├── nuscenes\n│   │   │── v1.0-trainval (or v1.0-mini if you use mini)\n│   │   │   │── samples\n│   │   │   │── sweeps\n│   │   │   │── maps\n│   │   │   │── v1.0-trainval  \n├── pcdet\n├── tools\n```\n\n* Install the `nuscenes-devkit` with version `1.0.5` by running the following command: \n```shell script\npip install nuscenes-devkit==1.0.5\n```\n\n* Generate the data infos by running the following command (it may take several hours): \n```python \npython -m pcdet.datasets.nuscenes.nuscenes_dataset --func create_nuscenes_infos \\\n    --cfg_file tools/cfgs/dataset_configs/nuscenes_dataset.yaml \\\n    --version v1.0-trainval\n``` --\u003e\n\n### Waymo Open Dataset\n* Please download the official [Waymo Open Dataset](https://waymo.com/open/download/), \nincluding the training data `training_0000.tar~training_0031.tar` and the validation \ndata `validation_0000.tar~validation_0007.tar`.\n* Unzip all the above `xxxx.tar` files to the directory of `data/waymo/raw_data` as follows (You could get 798 *train* tfrecord and 202 *val* tfrecord ):  \n```\nCRB-active-3Ddet\n├── data\n│   ├── waymo\n│   │   │── ImageSets\n│   │   │── raw_data\n│   │   │   │── segment-xxxxxxxx.tfrecord\n|   |   |   |── ...\n|   |   |── waymo_processed_data_v0_5_0\n│   │   │   │── segment-xxxxxxxx/\n|   |   |   |── ...\n│   │   │── waymo_processed_data_v0_5_0_gt_database_train_sampled_1/\n│   │   │── waymo_processed_data_v0_5_0_waymo_dbinfos_train_sampled_1.pkl\n│   │   │── waymo_processed_data_v0_5_0_gt_database_train_sampled_1_global.npy (optional)\n│   │   │── waymo_processed_data_v0_5_0_infos_train.pkl (optional)\n│   │   │── waymo_processed_data_v0_5_0_infos_val.pkl (optional)\n├── pcdet\n├── tools\n```\n* Install the official `waymo-open-dataset` by running the following command: \n```shell script\npip3 install --upgrade pip\npip3 install waymo-open-dataset-tf-2-0-0==1.2.0 --user\n```\n\u003e Waymo version in our project is 1.2.0\n\n* Extract point cloud data from tfrecord and generate data infos by running the following command (it takes several hours, \nand you could refer to `data/waymo/waymo_processed_data_v0_5_0` to see how many records that have been processed): \n```python \npython -m pcdet.datasets.waymo.waymo_dataset --func create_waymo_infos \\\n    --cfg_file tools/cfgs/dataset_configs/waymo_dataset.yaml\n```\n\nNote that you do not need to install `waymo-open-dataset` if you have already processed the data before and do not need to evaluate with official Waymo Metrics. \n\n\u003c!-- \n### Lyft Dataset\n* Please download the official [Lyft Level5 perception dataset](https://level-5.global/data/perception) and \norganize the downloaded files as follows: \n```\nCRB-active-3Ddet\n├── data\n│   ├── lyft\n│   │   │── ImageSets\n│   │   │── trainval\n│   │   │   │── data \u0026 maps \u0026 images \u0026 lidar \u0026 train_lidar\n├── pcdet\n├── tools\n```\n\n* Install the `lyft-dataset-sdk` with version `0.0.8` by running the following command: \n```shell script\npip install -U lyft_dataset_sdk==0.0.8\n```\n\n* Generate the data infos by running the following command (it may take several hours): \n```python \npython -m pcdet.datasets.lyft.lyft_dataset --func create_lyft_infos \\\n    --cfg_file tools/cfgs/dataset_configs/lyft_dataset.yaml\n```\n\n* You need to check carefully since we don't provide a benchmark for it. --\u003e\n\n\n\n\n## Training \u0026 Testing\n\n\n### Test and evaluate the pretrained models\nThe weights of our pre-trained model will be released upon acceptance.\n\n* Test with a pretrained model: \n```shell script\npython test.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --ckpt ${CKPT}\n```\n\n* To test all the saved checkpoints of a specific training setting and draw the performance curve on the Tensorboard, add the `--eval_all` argument: \n```shell script\npython test.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --eval_all\n```\n\n* To test with multiple GPUs:\n```shell script\nsh scripts/dist_test.sh ${NUM_GPUS} \\\n    --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE}\n\n# or\n\nsh scripts/slurm_test_mgpu.sh ${PARTITION} ${NUM_GPUS} \\\n    --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE}\n```\n\n### Train a backbone\nIn our active learning setting, the 3D detector will be pre-trained with a small labeled set $\\mathcal{D}_L$ which is randomly sampled from the trainig set. To train such a backbone, please run\n\n```shell script\nsh scripts/${DATASET}/train_${DATASET}_backbone.sh\n```\n\n\n### Train with different active learning strategies\nWe provide several options for active learning algorithms, including\n- random selection [`random`]\n- confidence sample [`confidence`]\n- entropy sampling [`entropy`]\n- MC-Reg sampling [`montecarlo`]\n- greedy coreset [`coreset`]\n- learning loss [`llal`]\n- BADGE sampling [`badge`]\n- CRB sampling [`crb`]\n\n\nYou could optionally add extra command line parameters `--batch_size ${BATCH_SIZE}` and `--epochs ${EPOCHS}` to specify your preferred parameters. \n\n* Train:\n```shell script\npython train.py --cfg_file ${CONFIG_FILE}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLuoyadan%2FCRB-active-3Ddet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FLuoyadan%2FCRB-active-3Ddet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FLuoyadan%2FCRB-active-3Ddet/lists"}