{"id":13442567,"url":"https://github.com/BIT-MJY/CVTNet","last_synced_at":"2025-03-20T14:31:34.516Z","repository":{"id":183581301,"uuid":"596992758","full_name":"BIT-MJY/CVTNet","owner":"BIT-MJY","description":"[TII 2023] A Cross-View Transformer Network for LiDAR-Based Place Recognition in Autonomous Driving Environments.","archived":false,"fork":false,"pushed_at":"2024-07-23T09:06:05.000Z","size":17960,"stargazers_count":97,"open_issues_count":3,"forks_count":8,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-08-01T03:41:19.639Z","etag":null,"topics":["global-localization","lidar-place-recognition","loop-closure-detection","multi-view","place-recognition","slam"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BIT-MJY.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-02-03T11:40:45.000Z","updated_at":"2024-07-26T04:25:56.000Z","dependencies_parsed_at":null,"dependency_job_id":"244d2510-acf5-460d-a241-c038b498fdf4","html_url":"https://github.com/BIT-MJY/CVTNet","commit_stats":null,"previous_names":["bit-mjy/cvtnet"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FCVTNet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FCVTNet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FCVTNet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FCVTNet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BIT-MJY","download_url":"https://codeload.github.com/BIT-MJY/CVTNet/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244630180,"owners_count":20484328,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["global-localization","lidar-place-recognition","loop-closure-detection","multi-view","place-recognition","slam"],"created_at":"2024-07-31T03:01:47.370Z","updated_at":"2025-03-20T14:31:34.511Z","avatar_url":"https://github.com/BIT-MJY.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# CVTNet\nThe code for our paper accepted by **IEEE Transactions on Industrial Informatics**:   \n### CVTNet: A Cross-View Transformer Network for LiDAR-Based Place Recognition in Autonomous Driving Environments.\n\n[[IEEE Xplore TII 2023](https://ieeexplore.ieee.org/document/10273716)] [[arXiv](https://arxiv.org/abs/2302.01665)] [[Supplementary Materials](https://ieeexplore.ieee.org/document/10273716/media#media)]\n\n[Junyi Ma](https://github.com/BIT-MJY), Guangming Xiong, [Jingyi Xu](https://github.com/BIT-XJY), [Xieyuanli Chen*](https://github.com/Chen-Xieyuanli)\n\n\u003cimg src=\"https://github.com/BIT-MJY/CVTNet/blob/main/motivation.png\" width=\"70%\"/\u003e\n\nCVTNet fuses the range image views (RIVs) and bird's eye views (BEVs) generated from LiDAR data to recognize previously visited places. RIVs and BEVs have the same shift for each yaw-angle rotation, which can be used to extract aligned features.\n\n\u003cimg src=\"https://github.com/BIT-MJY/CVTNet/blob/main/corresponding_rotation.gif\" width=\"70%\"/\u003e\n\n\n## Table of Contents\n1. [Publication](#Publication)\n2. [Dependencies](#Dependencies)\n3. [How to Use](##How-to-Use)\n4. [TODO](#Miscs)\n5. [Related Work](#Related-Work)\n6. [License](#License)\n\n## Publication\n\nIf you use the code in your work, please cite our [paper](https://arxiv.org/abs/2302.01665):\n\n```\n@ARTICLE{10273716,\n  author={Ma, Junyi and Xiong, Guangming and Xu, Jingyi and Chen, Xieyuanli},\n  journal={IEEE Transactions on Industrial Informatics}, \n  title={CVTNet: A Cross-View Transformer Network for LiDAR-Based Place Recognition in Autonomous Driving Environments}, \n  year={2023},\n  doi={10.1109/TII.2023.3313635}}\n```\n\n## Dependencies\n\nPlease refer to our [SeqOT repo](https://github.com/BIT-MJY/SeqOT).\n\n## How to Use\n\n**[2024-07]** We thank [Xiongwei Zhao](https://github.com/Grandzxw) for helping release the utilization on KITTI dataset!\n\n**[2023-03]** We provide a training and test tutorial for NCLT sequences in this repository. Before any operation, please modify the [config file](https://github.com/BIT-MJY/CVTNet/tree/main/config) according to your setups.\n\n\n### Data Preparation\n\n####  1. data preparation for NCLT dataset:\n\n* laser scans from NCLT dataset: [[2012-01-08](https://s3.us-east-2.amazonaws.com/nclt.perl.engin.umich.edu/velodyne_data/2012-01-08_vel.tar.gz)]       [[2012-02-05](https://s3.us-east-2.amazonaws.com/nclt.perl.engin.umich.edu/velodyne_data/2012-02-05_vel.tar.gz)]\n* [pretrained model](https://drive.google.com/file/d/1iQEY-DMDxchQ2RjG4RPkQcQehb_fujvO/view?usp=share_link)\n* [training indexes](https://drive.google.com/file/d/1jEcnuHjEi0wqe8GAoh6UTa4UXTu0sDPr/view)\n* [ground truth](https://drive.google.com/file/d/13-tpLQiHK4krd-womDV6UPevvHUIFNyF/view?usp=share_link)\n\nYou need to generate RIVs and BEVs from raw LiDAR data by\n\n```\ncd tools\npython gen_ri_bev.py\n```\n#### 2. data preparation for KITTI dataset (provided by [Xiongwei](https://github.com/Grandzxw)):\n\n2.1 You need to generate RIVs and BEVs from raw LiDAR data for train datasets and test datasets by\n```\ncd tools\npython gen_ri_bev.py \n```\n2.2 You need to generate training index for kitti from raw LiDAR data for train datasets by\n```\ncd tools\npython gen_training_index_kitti.py\n```\n2.3 You need to generate ground_truth for kitti by\n```\ncd tools\npython gen_ground_truth_kitti.py\n```\n\n### Training\n\nYou can start the training process with\n\n```\ncd train\npython ./train_cvtnet.py\n```\nNote that we only train our model using the oldest sequence of NCLT dataset (2012-01-08), to prove that our model works well for long time spans even if seeing limited data.  \n\n### Test\n\nYou can test the PR performance of CVTNet by\n\n```\ncd test\npython ./test_cvtnet_prepare.py\npython ./cal_topn_recall.py\n```\n\nYou can also test the yaw-rotation invariance of CVTNet by\n\n```\ncd test\npython ./test_yaw_rotation_invariance.py\n```\n\n\u003cimg src=\"https://github.com/BIT-MJY/CVTNet/blob/main/yaw_rotation_invariance.gif\" width=\"70%\"/\u003e\n\nIt can be seen that the global descriptors generated by CVTNet are not affected by yaw-angle rotation.\n\n### C++ Implementation\n\nWe provide a toy example showing C++ implementation of CVTNet with libtorch. First, you need to generate the model file by\n\n```\ncd CVTNet_libtorch\npython ./gen_libtorch_model.py\n```\n\n* Before building, make sure that [PCL](https://github.com/PointCloudLibrary/pcl) exists in your environment.\n* Here we use [LibTorch for CUDA 11.3 (Pre-cxx11 ABI)](https://download.pytorch.org/libtorch/cu113/libtorch-shared-with-deps-1.11.0%2Bcu113.zip). Please modify the path of **Torch_DIR** in [CMakeLists.txt](https://github.com/BIT-MJY/CVTNet/blob/main/CVTNet_libtorch/ws/CMakeLists.txt). \n* For more details of LibTorch installation, please check this [website](https://pytorch.org/get-started/locally/).  \n\nThen you can generate a descriptor of the provided 1.pcd by\n\n```\ncd ws\nmkdir build\ncd build\ncmake ..\nmake -j6\n./fast_cvtnet\n```\n\n### [2024-10] Test CVTNet on M2DGR\n\nWe thank [Xianyun Jiao and Jingyi Xu](mailto:yage0080@sjtu.edu.cn) for their efforts to test CVTNet on [M2DGR dataset](https://github.com/SJTU-ViSYS/M2DGR), which was collected in our SJTU campus by Yin et al.. Please contact them if there are any problems.\n\nThe related code is available at this [link](https://drive.google.com/file/d/10GCRTLjFptXakrlLv3NHZC0YXXR_1dA-/view?usp=drive_link).\n\n\u003cimg src=\"https://github.com/sjtuyinjie/mypics/blob/main/forgithub/outdoor.png\" width=\"70%\"/\u003e\n\nSequences of [M2DGR dataset](https://github.com/SJTU-ViSYS/M2DGR)\n\n\n## TODO\n- [x] Release the preprocessing code and pretrained model\n- [ ] Release sequence-enhanced CVTNet (SeqCVT)\n\n## Related Work\n\nThanks for your interest in our previous OT series for LiDAR-based place recognition.\n\n* [OverlapNet](https://github.com/PRBonn/OverlapNet): Loop Closing for 3D LiDAR-based SLAM\n\n```\n@inproceedings{chen2020rss, \n  author = {X. Chen and T. L\\\"abe and A. Milioto and T. R\\\"ohling and O. Vysotska and A. Haag and J. Behley and C. Stachniss},\n  title  = {{OverlapNet: Loop Closing for LiDAR-based SLAM}},\n  booktitle = {Proceedings of Robotics: Science and Systems (RSS)},\n  year = {2020}\n}\n```\n* [OverlapTransformer](https://github.com/haomo-ai/OverlapTransformer): An Efficient and Yaw-Angle-Invariant Transformer Network for LiDAR-Based Place Recognition\n\n```\n@ARTICLE{ma2022ral,\n  author={Ma, Junyi and Zhang, Jun and Xu, Jintao and Ai, Rui and Gu, Weihao and Chen, Xieyuanli},\n  journal={IEEE Robotics and Automation Letters}, \n  title={OverlapTransformer: An Efficient and Yaw-Angle-Invariant Transformer Network for LiDAR-Based Place Recognition}, \n  year={2022},\n  volume={7},\n  number={3},\n  pages={6958-6965},\n  doi={10.1109/LRA.2022.3178797}}\n```\n* [SeqOT](https://github.com/BIT-MJY/SeqOT): A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data\n\n```\n@ARTICLE{ma2022tie,\n  author={Ma, Junyi and Chen, Xieyuanli and Xu, Jingyi and Xiong, Guangming},\n  journal={IEEE Transactions on Industrial Electronics}, \n  title={SeqOT: A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data}, \n  year={2022},\n  doi={10.1109/TIE.2022.3229385}}\n```\n\n## License\n\nCopyright 2023, Junyi Ma, Guangming Xiong, Jingyi Xu, Xieyuanli Chen, Beijing Institute of Technology.\n\nThis project is free software made available under the MIT License. For more details see the LICENSE file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBIT-MJY%2FCVTNet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBIT-MJY%2FCVTNet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBIT-MJY%2FCVTNet/lists"}