https://github.com/haomo-ai/ModaLink
[IROS 2024] This repository contains the implementation of our paper: ModaLink: Unifying Modalities for Efficient Image-to-PointCloud Place Recognition
https://github.com/haomo-ai/ModaLink
camera-to-lidar cross-modal-localization cross-modal-retrieval place-recognition
Last synced: 3 months ago
JSON representation
[IROS 2024] This repository contains the implementation of our paper: ModaLink: Unifying Modalities for Efficient Image-to-PointCloud Place Recognition
- Host: GitHub
- URL: https://github.com/haomo-ai/ModaLink
- Owner: haomo-ai
- License: gpl-3.0
- Created: 2024-03-14T15:59:51.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2024-09-07T03:14:32.000Z (8 months ago)
- Last Synced: 2024-10-03T05:03:04.981Z (7 months ago)
- Topics: camera-to-lidar, cross-modal-localization, cross-modal-retrieval, place-recognition
- Language: Python
- Homepage:
- Size: 38.1 KB
- Stars: 13
- Watchers: 6
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# [IROS 2024] ModaLink
This repository contains the implementation of our IROS 2024 paper:
**ModaLink: Unifying Modalities for Efficient Image-to-PointCloud Place Recognition**
[Weidong Xie](https://sites.google.com/view/dong-hao/), [Lun Luo](https://zjuluolun.github.io/), Nanfei Ye, Yi Ren, Shaoyi Du, Minhang Wang, Jintao Xu, Rui Ai, Weihao Gu and [Xieyuanli Chen](https://github.com/Chen-Xieyuanli)
[Link](https://arxiv.org/abs/2403.18762) to the arXiv version of the paper is available.
The main contributions of this work are:
* We propose a lightweight cross-modal place recognition method called ModaLink based on FoV transformation.
* We introduce a Non-Negative Matrix Factorization-based module to extract extra potential semantic features to improve the distinctiveness of descriptors.
* Extensive experimental results on the KITTI and a self-collected dataset show that our proposed method can achieve state-of-the-art performance while running in real-time of about 30Hz.
## Citation
If you use our implementation in your academic work, please cite the corresponding [paper](https://arxiv.org/abs/2403.18762):
```
@inproceedings{xie2024modalink,
author = {Weidong Xie and Lun Luo and Nanfei Ye and Yi Ren and Shaoyi Du and Minhang Wang and Jintao Xu and Rui Ai and Weihao Gu and Xieyuanli Chen},
title = {{ModaLink: Unifying Modalities for Efficient Image-to-PointCloud Place Recognition}},
booktitle = {In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year = {2024},
}
```
## Dependencies
We use pytorch-gpu for neural networks. An Nvidia GPU is needed for faster retrieval.
To use a GPU, first, you need to install the Nvidia driver and CUDA.
- CUDA Installation guide: [link](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html)