{"id":13441688,"url":"https://github.com/leofansq/CBR","last_synced_at":"2025-03-20T12:32:24.649Z","repository":{"id":171260467,"uuid":"626393225","full_name":"leofansq/CBR","owner":"leofansq","description":"[IROS2023] Calibration-free BEV Representation for Infrastructure Perception","archived":false,"fork":false,"pushed_at":"2023-05-31T08:46:45.000Z","size":1824,"stargazers_count":27,"open_issues_count":1,"forks_count":3,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-08-01T03:37:06.327Z","etag":null,"topics":["calibration-free","camera-based-detection","representation-learning","roadside-perception"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leofansq.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2023-04-11T11:33:42.000Z","updated_at":"2024-07-12T06:37:33.000Z","dependencies_parsed_at":"2024-01-16T02:45:49.809Z","dependency_job_id":"68cfd369-0b8f-4c6f-9bdb-92589d685ed7","html_url":"https://github.com/leofansq/CBR","commit_stats":null,"previous_names":["leofansq/cbr"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leofansq%2FCBR","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leofansq%2FCBR/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leofansq%2FCBR/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leofansq%2FCBR/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leofansq","download_url":"https://codeload.github.com/leofansq/CBR/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221760114,"owners_count":16876355,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["calibration-free","camera-based-detection","representation-learning","roadside-perception"],"created_at":"2024-07-31T03:01:37.035Z","updated_at":"2025-03-20T12:32:24.644Z","avatar_url":"https://github.com/leofansq.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Calibration-free BEV Representation for Infrastructure Perception\r\n\r\n![architecture](./img/CBR.png)\r\n\r\n\u003e For technical details, please refer to:\r\n\u003e\r\n\u003e [Calibration-free BEV Representation for Infrastructure Perception](https://arxiv.org/abs/2303.03583)\r\n\r\n### (0) Abstract\r\n\r\nEffective BEV object detection on infrastructure can greatly improve traffic scenes understanding and vehicle-toinfrastructure (V2I) cooperative perception. However, cameras installed on infrastructure have various postures, and previous BEV detection methods rely on accurate calibration, which is difficult for practical applications due to inevitable natural factors (e.g., wind and snow). In this paper, we propose a Calibration-free BEV Representation (CBR) network, which achieves 3D detection based on BEV representation without calibration parameters and additional depth supervision. Specifically, we utilize two multi-layer perceptrons for decoupling the features from perspective view to front view and bird-eye view under boxes-induced foreground supervision. Then, a cross-view feature fusion module matches features from orthogonal views according to similarity and conducts BEV feature enhancement with front view features. Experimental results on DAIR-V2X demonstrate that CBR achieves acceptable performance without any camera parameters and is naturally not affected by calibration noises. We hope CBR can serve as a baseline for future research addressing practical challenges of infrastructure perception.\r\n\r\n\r\n### (1) Setup\r\n\r\nThis code has been tested with Python 3.8.10, Pytorch 1.11.0, CUDA 11.7.\r\n\r\n* Setup environment\r\n    ```bash\r\n    conda create -n CBR python=3.8.10\r\n    source activate CBR\r\n    ```\r\n\r\n* Clone the respository\r\n\r\n* Install the requirements\r\n    ```\r\n    pip install -r requirements.txt\r\n    ```\r\n\r\n* Dataset\r\n    * Download [DAIR-V2X dataset](https://github.com/AIR-THU/DAIR-V2X#dataset-download-)\r\n    * Generate KITTI-format data for DAIR-V2X-C following the [guidelines](https://github.com/AIR-THU/DAIR-V2X/blob/main/configs/vic3d/late-fusion-image/imvoxelnet/README.md#create-kitti-format-data-option-for-model-training). If you want to conduct experiments on DAIR-V2X-I, you should also generate the corresponding KITTI-format data.\r\n    * Create a symlink\r\n        ```bash\r\n        # For DAIR-V2X-C\r\n        cd datasets/dair_c_inf/training\r\n        ln -s ${DAIR-V2X-C_KITTI_ROOT}/cooperative-vehicle-infrastructure/training/image_2 ./image_2\r\n        ln -s ${DAIR-V2X-C_KITTI_ROOT}/cooperative-vehicle-infrastructure/training/label_2 ./label_2\r\n\r\n        # For DAIR-V2X-I\r\n        cd datasets/dair_i/training\r\n        ln -s ${DAIR-V2X-I_KITTI_ROOT}/single-infrastructure-side/training/image_2 ./image_2\r\n        ln -s ${DAIR-V2X-I_KITTI_ROOT}/single-infrastructure-side/training/label_2 ./label_2\r\n        ```\r\n    * Generate foreground segmentation labels\r\n        ```bash\r\n        cd tools\r\n        # For DAIR-V2X-C\r\n        python gen_bevseg_gt.py\r\n        python gen_fvseg_gt.py\r\n\r\n        # For DAIR-V2X-I\r\n        python gen_bevseg_gt.py --dataset dair_i\r\n        python gen_fvseg_gt.py --dataset dair_i\r\n        ```\r\n\r\n### (2) Experiments on DAIR-V2X-C\r\n* Training\r\n    ```bash\r\n    CUDA_VISIBLE_DEVICES=0 python train.py\r\n    ```\r\n* Evaluation\r\n    ```bash\r\n    CUDA_VISIBLE_DEVICES=0 python eval.py\r\n    ```\r\n* Visualization\r\n    ```bash\r\n    CUDA_VISIBLE_DEVICES=0 python eval.py --vis\r\n    ```\r\n    ![demo_dair_v2x_c](./img/demo_dair_c.jpg)\r\n\r\n### (3) Experiments on DAIR-V2X-I\r\n* Training\r\n    ```bash\r\n    CUDA_VISIBLE_DEVICES=0 python train.py --dataset dair_i\r\n    ```\r\n* Evaluation\r\n    ```bash\r\n    CUDA_VISIBLE_DEVICES=0 python eval.py --dataset dair_i --pretrained_path ./pretrained/dair_i/\r\n    ```\r\n    \u003e Download the pretrained model for DAIR-V2X-I from [key:mrmc](https://pan.baidu.com/s/1X6KwJVwGGr7Ac0WBzaCV_w) and put them in './pretrained/dair_i/'.\r\n* Visualization\r\n    ```bash\r\n    CUDA_VISIBLE_DEVICES=0 python eval.py --dataset dair_i --pretrained_path ./pretrained/dair_i/ --vis\r\n    ```\r\n    ![demo_dair_v2x_i](./img/demo_dair_i.jpg)\r\n\r\n\r\n## Citation\r\n\r\nIf you find our work useful in your research, please consider citing:\r\n\r\n```\r\n@article{fan2023calibration,\r\n  title={Calibration-free BEV Representation for Infrastructure Perception},\r\n  author={Fan, Siqi and Wang, Zhe and Huo, Xiaoliang and Wang, Yan and Liu, Jingjing},\r\n  journal={arXiv preprint arXiv:2303.03583},\r\n  year={2023}}\r\n```\r\n\r\n### Acknowledgment\r\n\r\nPart of our code refers to the work [PYVA](https://github.com/JonDoe-297/cross-view)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleofansq%2FCBR","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleofansq%2FCBR","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleofansq%2FCBR/lists"}