{"id":13442257,"url":"https://github.com/ucla-mobility/V2V4Real","last_synced_at":"2025-03-20T13:33:10.870Z","repository":{"id":113541609,"uuid":"609351183","full_name":"ucla-mobility/V2V4Real","owner":"ucla-mobility","description":"[CVPR2023 Highlight] The official codebase for paper \"V2V4Real: A large-scale real-world dataset for Vehicle-to-Vehicle Cooperative Perception\"","archived":false,"fork":false,"pushed_at":"2024-08-14T16:14:27.000Z","size":3104,"stargazers_count":187,"open_issues_count":13,"forks_count":12,"subscribers_count":9,"default_branch":"main","last_synced_at":"2024-08-14T17:55:33.538Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ucla-mobility.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-03T23:37:52.000Z","updated_at":"2024-08-14T16:14:30.000Z","dependencies_parsed_at":"2023-06-29T23:30:16.999Z","dependency_job_id":"d731439e-8c15-454b-b23b-170458fd4b05","html_url":"https://github.com/ucla-mobility/V2V4Real","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucla-mobility%2FV2V4Real","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucla-mobility%2FV2V4Real/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucla-mobility%2FV2V4Real/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ucla-mobility%2FV2V4Real/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ucla-mobility","download_url":"https://codeload.github.com/ucla-mobility/V2V4Real/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221768510,"owners_count":16877651,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:43.611Z","updated_at":"2025-03-20T13:33:10.862Z","avatar_url":"https://github.com/ucla-mobility.png","language":"Python","funding_links":[],"categories":["Python","Vehicle-to-Everything Field Datasets","Anti-UAV Datasets"],"sub_categories":[],"readme":"# V2V4Real: A large-scale real-world dataset for Vehicle-to-Vehicle Cooperative Perception\n[![website](https://img.shields.io/badge/Website-Explore%20Now-blueviolet?style=flat\u0026logo=google-chrome)](https://research.seas.ucla.edu/mobility-lab/v2v4real/)\n[![paper](https://img.shields.io/badge/arXiv-Paper-\u003cCOLOR\u003e.svg)](https://arxiv.org/pdf/2303.07601.pdf)\n[![supplement](https://img.shields.io/badge/Supplementary-Material-red)](https://arxiv.org/pdf/2303.07601.pdf)\n[![video](https://img.shields.io/badge/Video-Presentation-F9D371)]()\n\n\nThis is the official implementation of CVPR2023 **Highlight** paper. \"V2V4Real: A large-scale real-world dataset for Vehicle-to-Vehicle Cooperative Perception\".\n[Runsheng Xu](https://derrickxunu.github.io/),  [Xin Xia](https://scholar.google.com/citations?user=vCYqMTIAAAAJ\u0026hl=en), [Jinlong Li](https://jinlong17.github.io/), [Hanzhao Li](), [Shuo Zhang](),  [Zhengzhong Tu](https://github.com/vztu), [Zonglin Meng](), [Hao Xiang](https://xhwind.github.io/), [Xiaoyu Dong](), [Rui Song](), [Hongkai Yu](https://scholar.google.com/citations?user=JnQts0kAAAAJ\u0026hl=en), [Bolei Zhou](https://boleizhou.github.io/), [Jiaqi Ma](https://mobility-lab.seas.ucla.edu/)\n\nSupported by the [UCLA Mobility Lab](https://mobility-lab.seas.ucla.edu/).\n\n\u003cp align=\"center\"\u003e\n\u003cimg src=\"imgs/scene1.png\" width=\"600\" alt=\"\" class=\"img-responsive\"\u003e\n\u003c/p\u003e\n\n## Overview\n- [Codebase Features](#codebase-features)\n- [Data Download](#data-download)\n- [Changelog](#changelog)\n- [Devkit Setup](#devkit-setup)\n- [Quick Start](#quick-start)\n- [Benchmark](#benchmark)\n- [Citation](#citation)\n- [Acknowledgment](#known-issues)\n\n## CodeBase Features\n- Support both simulation and real-world cooperative perception dataset\n    - [x] V2V4Real\n    - [x] OPV2V\n- Multiple Tasks supported\n    - [x] 3D object detection\n    - [ ] Cooperative tracking\n    - [x] Sim2Real\n- SOTA model supported\n    - [x] [Attentive Fusion [ICRA2022]](https://arxiv.org/abs/2109.07644)\n    - [x] [Cooper [ICDCS]](https://arxiv.org/abs/1905.05265)\n    - [x] [F-Cooper [SEC2019]](https://arxiv.org/abs/1909.06459)\n    - [x] [V2VNet [ECCV2020]](https://arxiv.org/abs/2008.07519)\n    - [x] [V2X-ViT [ECCV2022]](https://github.com/DerrickXuNu/v2x-vit)\n    - [x] [CoBEVT [CoRL2022]](https://arxiv.org/abs/2207.02202)\n\n## Data Download\nPlease check our [website](https://research.seas.ucla.edu/mobility-lab/v2v4real/) to download the data (OPV2V format).\n\nAfter downloading the data, please put the data in the following structure:\n```shell\n├── v2v4real\n│   ├── train\n|      |── testoutput_CAV_data_2022-03-15-09-54-40_1\n│   ├── validate\n│   ├── test\n```\n## Changelog\n- Oct. 22, 2023: Kitti format data is released [google drive](https://drive.google.com/drive/folders/10mZ_Cr3MImBktyFfB0yao-1pgKlXuOco?usp=share_link)\n- Mar. 21, 2023: Sim2Real related codebase and pre-trained models are released\n- Apr. 08, 2023: Dateset and pre-trained models are released\n- Mar. 23, 2023: The codebase for 3D object detection is released\n- Mar. 19, 2023: The website is ready\n- Mar. 14, 2023: Tha paper is released\n\n## Devkit setup\nV2V4Real's codebase is build upon [OpenCOOD](https://github.com/DerrickXuNu/OpenCOOD). Compared to OpenCOOD, this codebase supports both the simulation and real-world data and more perception tasks. Furthermore, this repo provides augmentations that OpenCOOD does not support. We highly recommend you to use this codebase to train your model on V2V4Real dataset\n\nTo set up the codebase environment, do the following steps:\n#### 1. Create conda environment (python \u003e= 3.7)\n```shell\nconda create -n v2v4real python=3.7\nconda activate v2v4real\n```\n#### 2. Pytorch Installation (\u003e= 1.12.0 Required)\nTake pytorch 1.12.0 as an example:\n```shell\nconda install pytorch==1.12.0 torchvision==0.13.0 cudatoolkit=11.3 -c pytorch -c conda-forge\n```\n#### 3. spconv 2.x Installation\n```shell\npip install spconv-cu113\n```\n#### 4. Install other dependencies\n```shell\npip install -r requirements.txt\npython setup.py develop\n```\n#### 5.Install bbx nms calculation cuda version\n```shell\npython opencood/utils/setup.py build_ext --inplace\n```\n\n## Quick Start\n### Data sequence visualization\nTo quickly visualize the LiDAR stream in the OPV2V dataset, first modify the `validate_dir`\nin your `opencood/hypes_yaml/visualization.yaml` to the opv2v data path on your local machine, e.g. `opv2v/validate`,\nand the run the following commond:\n```python\ncd ~/OpenCOOD\npython opencood/visualization/vis_data_sequence.py [--color_mode ${COLOR_RENDERING_MODE} --isSim]\n```\nArguments Explanation:\n- `color_mode` : str type, indicating the lidar color rendering mode. You can choose from 'v2vreal', 'constant', 'intensity' or 'z-value'.\n- `isSim` : bool type, if you are visualizing the simulation data, then claim this argument.\n\n### Train your model\nOpenCOOD uses yaml file to configure all the parameters for training. To train your own model\nfrom scratch or a continued checkpoint, run the following commonds:\n```python\npython opencood/tools/train.py --hypes_yaml ${CONFIG_FILE} [--model_dir  ${CHECKPOINT_FOLDER} --half]\n```\nArguments Explanation:\n- `hypes_yaml`: the path of the training configuration file, e.g. `opencood/hypes_yaml/point_pillar_fax.yaml`, meaning you want to train\nCoBEVT with pointpillar backbone. See [Tutorial 1: Config System](https://opencood.readthedocs.io/en/latest/md_files/config_tutorial.html) to learn more about the rules of the yaml files.\n- `model_dir` (optional) : the path of the checkpoints. This is used to fine-tune the trained models. When the `model_dir` is\ngiven, the trainer will discard the `hypes_yaml` and load the `config.yaml` in the checkpoint folder.\n- `half` (optional): If set, the model will be trained with half precision. It cannot be set with multi-gpu training togetger.\n\nTo train on **multiple gpus**, run the following command:\n```\nCUDA_VISIBLE_DEVICES=0,1,2,3 python -m torch.distributed.launch --nproc_per_node=4  --use_env opencood/tools/train.py --hypes_yaml ${CONFIG_FILE} [--model_dir  ${CHECKPOINT_FOLDER}]\n```\n\n### Train Sim2Real\nWe provide `train_da.py` to train the sim2real models shown in the paper. The models will take the simulation data and \nv2v4real data without gt labels as input, and compute the domain adaptation loss. To train the sim2real model, run the following command:\n```python\npython opencood/tools/train_da.py --hypes_yaml hypes_yaml/domain_adaptions/xxx.yaml [--model_dir  ${CHECKPOINT_FOLDER} --half\n```\n\n### Test the model\nBefore you run the following command, first make sure the `validation_dir` in config.yaml under your checkpoint folder\nrefers to the testing dataset path, e.g. `v2v4real/test`.\n\n```python\npython opencood/tools/inference.py --model_dir ${CHECKPOINT_FOLDER} --fusion_method ${FUSION_STRATEGY} [--show_vis] [--show_sequence]\n```\nArguments Explanation:\n- `model_dir`: the path to your saved model.\n- `fusion_method`: indicate the fusion strategy, currently support 'nofusion', 'early', 'late', and 'intermediate'.\n- `show_vis`: whether to visualize the detection overlay with point cloud.\n- `show_sequence` : the detection results will visualized in a video stream. It can NOT be set with `show_vis` at the same time.\n\nThe evaluation results  will be dumped in the model directory.\n\nImportant notes for testing:\n1. Remember to change the `validation_dir` in config.yaml under your checkpoint folder to the testing dataset path, e.g. `v2v4real/test`.\n2. To test under async mode, you need to set the `async_mode` in config.yaml to `True` and set the `async_overhead` to the desired delay time (default 100ms).\n3. The testing script for cooperative 3D object detection and sim2real is the same\n\n## Benchmark\n### Results of Cooperative 3D object detection\n| Method        | Backbone    | Sync AP@0.5 | Sync AP@0.7 | Async AP@0.5 | Async AP@0.7 | Bandwidth | Download Link                                                            |\n|--------------|-------------|----------------|----------------|--------------|--------------|-----------|--------------------------------------------------------------------------|\n| No Fusion    | PointPillar | 39.8           | 22.0          | 39.8          | 22.0          |      0.0     |    [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619899319493)                                                                |\n| Late Fusion  | PointPillar | 55.0           | 26.7       | 50.2        | 22.4         |      0.003     |        [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619899319493)                                                                  |\n| Early Fusion | PointPillar  | 59.7          | 32.1         | 52.1        | 25.8       |      0.96     |         [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619904736644)              |\n| [F-Cooper](https://arxiv.org/abs/1909.06459) | PointPillar | 60.7          | 31.8          | 53.6        | 26.7       |      0.20     |     [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619904126201)                                                                    |\n| [Attentive Fusion](https://arxiv.org/abs/2109.07644)     | PointPillar | 64.5         | 34.3          | 56.4         | 28.5      |     0.20      |      [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619906517230)                                                       |\n| [V2VNet](https://arxiv.org/abs/2008.07519)         |PointPillar | 64.7         | 33.6           | 57.7        | 27.5     |     0.20      |     [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619896948709)                                                                  |\n| [V2X-ViT](https://arxiv.org/pdf/2203.10638.pdf)    | PointPillar | 64.9          | **36.9**           | 55.9       | 29.3       |   0.20        | [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619911355608)\n| [CoBEVT](https://arxiv.org/abs/2207.02202)      | PointPillar |    **66.5**     |  36.0   | **58.6**  | **29.7**  | 0.20| [url](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/file/1619912313471)|\n\n### Results of Cooperative tracking\n| Method       | AMOTA(↑) | AMOTP(↑) | sAMOTA(↑) | MOTA(↑)  | MT(↑)    | ML(↓)    |\n|--------------|----------|----------|-----------|----------|----------|----------|\n| No Fusion    | 16.08    | 41.60    | 53.84     | 43.46    | 29.41    | 60.18    |\n| Late Fusion  | 29.28    | 51.08    | 71.05     | 59.89    | 45.25    | 31.22    |\n| Early Fusion | 26.19    | 48.15    | 67.34     | 60.87    | 40.95    | 32.13    |\n| F-Cooper     | 23.29    | 43.11    | 65.63     | 58.34    | 35.75    | 38.91    |\n| AttFuse      | 28.64    | 50.48    | 73.21     | 63.03    | 46.38    | 28.05    |\n| V2VNet       | 30.48    | 54.28    | 75.53     | **64.85**    | **48.19**    | 27.83    |\n| V2X-ViT      | 30.85    | 54.32    | 74.01     | 64.82    | 45.93    | **26.47**    |\n| CoBEVT       | **32.12**    | **55.61**    | **77.65**     | 63.75    | 47.29    | 30.32    |\n\n### Results of Domain Adaption\n| Method       | Domain Adaption | AP@0.5 | Download Link |\n|--------------|----------|----------|-----------\n| F-Cooper     | [1]   | 37.3   |   [Download Link](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/folder/279924385524)  | \n| AttFuse      | [1]      | 23.4   |  [Download Link](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/folder/279923915754)   | \n| V2VNet       | [1]      | 26.3   | [Download Link](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/folder/279922383820)   |\n| V2X-ViT      | [1]     | 39.5   |  [Download Link](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/folder/279924578215)  | \n| CoBEVT       | [1]      | **40.2**  | [Download LInk](https://ucla.app.box.com/v/UCLA-MobilityLab-V2V4REAL/folder/279923471991)   |\n\n[1]: Yuhua Chen, Wen Li, Christos Sakaridis, Dengxin Dai, and\nLuc Van Gool. Domain adaptive faster r-cnn for object de-\ntection in the wild. In Proceedings of the IEEE conference on\ncomputer vision and pattern recognition, pages 3339–3348, 2018.\n\n## Citation\n```shell\n@inproceedings{xu2023v2v4real,\n  title={V2V4Real: A Real-world Large-scale Dataset for Vehicle-to-Vehicle Cooperative Perception},\n  author={Xu, Runsheng and Xia, Xin and Li, Jinlong and Li, Hanzhao and Zhang, Shuo and Tu, Zhengzhong and Meng, Zonglin and Xiang, Hao and Dong, Xiaoyu and Song, Rui and Yu, Hongkai and Zhou, Bolei and Ma, Jiaqi},\n  booktitle={The IEEE/CVF Computer Vision and Pattern Recognition Conference (CVPR)},\n  year={2023}\n}\n```\n\n## Acknowledgment\nThis dataset belongs to the [OpenCDA ecosystem](https://arxiv.org/abs/2301.07325) family. The codebase is build upon [OpenCOOD](https://github.com/DerrickXuNu/OpenCOOD), which is the first Open Cooperative Detection framework for autonomous driving.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fucla-mobility%2FV2V4Real","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fucla-mobility%2FV2V4Real","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fucla-mobility%2FV2V4Real/lists"}