{"id":13442757,"url":"https://github.com/hailanyi/TED","last_synced_at":"2025-03-20T15:30:45.062Z","repository":{"id":119034037,"uuid":"531045126","full_name":"hailanyi/TED","owner":"hailanyi","description":"Transformation-Equivariant 3D Object Detection for Autonomous Driving","archived":false,"fork":false,"pushed_at":"2024-05-03T09:12:30.000Z","size":626,"stargazers_count":138,"open_issues_count":11,"forks_count":32,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-10-28T05:59:46.065Z","etag":null,"topics":["3d-object-detection","autonomous-driving","kitti"],"latest_commit_sha":null,"homepage":"https://arxiv.org/abs/2211.11962","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hailanyi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-31T11:01:52.000Z","updated_at":"2024-10-16T12:00:06.000Z","dependencies_parsed_at":"2023-09-29T03:40:41.048Z","dependency_job_id":"35cca87b-68eb-47e3-aa59-177e868c84c4","html_url":"https://github.com/hailanyi/TED","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailanyi%2FTED","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailanyi%2FTED/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailanyi%2FTED/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hailanyi%2FTED/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hailanyi","download_url":"https://codeload.github.com/hailanyi/TED/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244639889,"owners_count":20485945,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-object-detection","autonomous-driving","kitti"],"created_at":"2024-07-31T03:01:50.219Z","updated_at":"2025-03-20T15:30:44.566Z","avatar_url":"https://github.com/hailanyi.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"\n# Transformation-Equivariant 3D Object Detection for Autonomous Driving\nThis is a improved version of [TED](https://arxiv.org/abs/2211.11962) by a multiple refinement design. \nThis code is mainly based on [OpenPCDet](https://github.com/open-mmlab/OpenPCDet) and [CasA](https://github.com/hailanyi/CasA), some codes are from \n[PENet](https://github.com/JUGGHM/PENet_ICRA2021) and [SFD](https://github.com/LittlePey/SFD).\n\n## Detection Framework\nThe overall detection framework is shown below.\n(1) Transformation-equivariant Sparse Convolution (TeSpConv) backbone; (2) Transformation-equivariant Bird Eye View (TeBEV) pooling; \n(3) Multi-grid pooling and multi-refinement. \nTeSpConv applies shared weights on multiple transformed point clouds to record the transformation-equivariant voxel features. \nTeBEV pooling aligns and aggregates the scene-level equivariant features into lightweight representations for proposal generation.\n Multi-grid pooling and multi-refinement align and aggregate the instance-level invariant features for proposal refinement.\n \n![](./tools/images/framework.png)\n\n## Model Zoo\nWe release two models, which are based on LiDAR-only and multi-modal data respectively. We denoted the two models as TED-S and TED-M respectively.\n\n* All models are trained with 8 V100 GPUs and are available for download. \n\n* The models are trained with train split (3712 samples) of KITTI dataset\n\n* The results are the 3D AP(R40) of Car on the *val* set of KITTI dataset.\n\n* These models are not suitable to directly report results on KITTI test set, please use slightly lower score threshold and train the models on all or 80% training data to achieve a desirable performance on KITTI test set.\n\n|                                             |Modality|GPU memory of training| Easy | Mod. | Hard  | download | \n|---------------------------------------------|----------:|----------:|:-------:|:-------:|:-------:|:---------:|\n| [TED-S](tools/cfgs/models/kitti/TED-S.yaml)|LiDAR only|~12 GB |93.25 |87.99| 86.28| [google](https://drive.google.com/file/d/1hqoj-lV4Cr3m7U3EphdCSjHmhBlekRm8/view?usp=sharing) / [baidu(p91t)](https://pan.baidu.com/s/1ecobwO673ScrGYOHbooGIw) / 36M | \n| [TED-M](tools/cfgs/models/kitti/TED-M.yaml)|LiDAR+RGB |~15 GB| 95.62 |89.24 |86.77 | [google](https://drive.google.com/file/d/1hXe1at-LKogTfWorALmq6djjYqhKX7nD/view?usp=sharing) / [baidu(nkr5)](https://pan.baidu.com/s/1FP80452dfM09YtE8DBaicQ) / 65M|\n\n## Getting Started\n\n```\nconda create -n spconv2 python=3.9\nconda activate spconv2\npip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html\npip install numpy==1.19.5 protobuf==3.19.4 scikit-image==0.19.2 waymo-open-dataset-tf-2-5-0 nuscenes-devkit==1.0.5 spconv-cu111 numba scipy pyyaml easydict fire tqdm shapely matplotlib opencv-python addict pyquaternion awscli open3d pandas future pybind11 tensorboardX tensorboard Cython prefetch-generator\n```\n\n### Dependency\nOur released implementation is tested on.\n+ Ubuntu 18.04\n+ Python 3.6.9 \n+ PyTorch 1.8.1\n+ Spconv 1.2.1\n+ NVIDIA CUDA 11.1\n+ 8x Tesla V100 GPUs\n\nWe also tested on.\n+ Ubuntu 18.04\n+ Python 3.9.13\n+ PyTorch 1.8.1\n+ Spconv 2.1.22 # pip install spconv-cu111\n+ NVIDIA CUDA 11.1\n+ 2x 3090 GPUs\n\n### Prepare dataset\n\nPlease download the official [KITTI 3D object detection](http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d) dataset and organize the downloaded files as follows (the road planes could be downloaded from [[road plane]](https://drive.google.com/file/d/1d5mq0RXRnvHPVeKx6Q612z0YRO1t2wAp/view?usp=sharing), which are optional for data augmentation in the training):\n\n```\nTED\n├── data\n│   ├── kitti\n│   │   │── ImageSets\n│   │   │── training\n│   │   │   ├──calib \u0026 velodyne \u0026 label_2 \u0026 image_2 \u0026 (optional: planes)\n│   │   │── testing\n│   │   │   ├──calib \u0026 velodyne \u0026 image_2\n├── pcdet\n├── tools\n```\n\nYou need creat a 'velodyne_depth' dataset to run our multimodal detector:\nYou can download our preprocessed data from [google (13GB)](https://drive.google.com/file/d/1xki9v_zsQMM8vMVNo0ENi1Mh_GNMjHUg/view?usp=sharing), [baidu (a20o)](https://pan.baidu.com/s/1OH4KIVoSSH7ea3-3CqkZRQ), or generate the data by yourself:\n* [Install this project](#installation).\n* Download the PENet depth completion model [here (500M)](https://drive.google.com/file/d/1RDdKlKJcas-G5OA49x8OoqcUDiYYZgeM/view?usp=sharing) and put it into ```tools/PENet```.\n* Then run the following code to generate RGB pseudo points.\n```\ncd tools/PENet\npython3 main.py --detpath [your path like: ../../data/kitti/training]\n```\n\nAfter 'velodyne_depth' generation, run following command to creat dataset infos:\n```\ncd ../..\npython3 -m pcdet.datasets.kitti.kitti_dataset create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml\npython3 -m pcdet.datasets.kitti.kitti_dataset_mm create_kitti_infos tools/cfgs/dataset_configs/kitti_dataset.yaml\n```\n\nAnyway, the data structure should be: \n```\nTED\n├── data\n│   ├── kitti\n│   │   │── ImageSets\n│   │   │── training\n│   │   │   ├──calib \u0026 velodyne \u0026 label_2 \u0026 image_2 \u0026 (optional: planes) \u0026 velodyne_depth\n│   │   │── testing\n│   │   │   ├──calib \u0026 velodyne \u0026 image_2 \u0026 velodyne_depth\n│   │   │── gt_database\n│   │   │── gt_database_mm\n│   │   │── kitti_dbinfos_train_mm.pkl\n│   │   │── kitti_dbinfos_train.pkl\n│   │   │── kitti_infos_test.pkl\n│   │   │── kitti_infos_train.pkl\n│   │   │── kitti_infos_trainval.pkl\n│   │   │── kitti_infos_val.pkl\n├── pcdet\n├── tools\n```\n\n### Installation\n\n```\ngit clone https://github.com/hailanyi/TED.git\ncd TED\npython3 setup.py develop\n```\n\n### Training\n\nSingle GPU train:\n```\ncd tools\npython3 train.py --cfg_file ${CONFIG_FILE}\n```\nFor example, if you train the TED-S model:\n```\ncd tools\npython3 train.py --cfg_file cfgs/models/kitti/TED-S.yaml\n```\n\nMultiple GPU train: \n\nYou can modify the gpu number in the dist_train.sh and run\n```\ncd tools\nsh dist_train.sh\n```\nThe log infos are saved into log.txt\nYou can run ```cat log.txt``` to view the training process.\n\n### Evaluation\n\n```\ncd tools\npython3 test.py --cfg_file ${CONFIG_FILE} --batch_size ${BATCH_SIZE} --ckpt ${CKPT}\n```\n\nFor example, if you test the TED-S model:\n\n```\ncd tools\npython3 test.py --cfg_file cfgs/models/kitti/TED-S.yaml --ckpt TED-S.pth\n```\n\nMultiple GPU test: you need modify the gpu number in the dist_test.sh and run\n```\nsh dist_test.sh \n```\nThe log infos are saved into log-test.txt\nYou can run ```cat log-test.txt``` to view the test results.\n\n## License\n\nThis code is released under the [Apache 2.0 license](LICENSE).\n\n## Acknowledgement\n\n[CasA](https://github.com/hailanyi/CasA)\n\n[OpenPCDet](https://github.com/open-mmlab/OpenPCDet)\n\n[PENet](https://github.com/JUGGHM/PENet_ICRA2021)\n\n[SFD](https://github.com/LittlePey/SFD)\n\n## Citation\n    @inproceedings{TED,\n        title={Transformation-Equivariant 3D Object Detection for Autonomous Driving},\n        author={Wu, Hai and Wen, Chenglu and Li, Wei and Yang, Ruigang and Wang, Cheng},\n        year={2023},\n        booktitle={AAAI}\n        \n    }\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhailanyi%2FTED","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhailanyi%2FTED","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhailanyi%2FTED/lists"}