{"id":13442280,"url":"https://github.com/BIT-MJY/SeqOT","last_synced_at":"2025-03-20T13:33:19.561Z","repository":{"id":65070085,"uuid":"501610028","full_name":"BIT-MJY/SeqOT","owner":"BIT-MJY","description":"[TIE 2022] SeqOT: A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data.","archived":false,"fork":false,"pushed_at":"2023-06-30T07:48:47.000Z","size":1152,"stargazers_count":96,"open_issues_count":3,"forks_count":7,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-08-01T03:39:27.181Z","etag":null,"topics":["lidar-place-recognition","sequence-embedding"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/BIT-MJY.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2022-06-09T10:40:16.000Z","updated_at":"2024-07-19T12:06:05.000Z","dependencies_parsed_at":"2024-01-16T02:46:27.551Z","dependency_job_id":"89e7f4ec-e4d2-499f-b37b-268e5acda130","html_url":"https://github.com/BIT-MJY/SeqOT","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FSeqOT","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FSeqOT/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FSeqOT/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/BIT-MJY%2FSeqOT/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/BIT-MJY","download_url":"https://codeload.github.com/BIT-MJY/SeqOT/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221768530,"owners_count":16877655,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["lidar-place-recognition","sequence-embedding"],"created_at":"2024-07-31T03:01:43.851Z","updated_at":"2025-03-20T13:33:19.547Z","avatar_url":"https://github.com/BIT-MJY.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# SeqOT\nThe code for our paper accepted by **IEEE Transactions on Industrial Electronics**:   \n### SeqOT: A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data.\n\n[[IEEE Xplore TIE 2022](https://ieeexplore.ieee.org/document/9994714)] [[arXiv](https://arxiv.org/abs/2209.07951)]\n \nSeqOT is a sequence-enhanced LiDAR-based place recognition method based on our previous work [OverlapTransformer (OT)](https://github.com/haomo-ai/OverlapTransformer) for RAL/IROS 2022.  \n\nDeveloped by [Junyi Ma](https://github.com/BIT-MJY) and [Xieyuanli Chen](https://github.com/Chen-Xieyuanli).\n\n---\n\n\u003cimg src=\"https://github.com/BIT-MJY/SeqOT/blob/main/overview.png\" width=\"48%\" \u003e\u003cimg src=\"https://github.com/BIT-MJY/SeqOT/blob/main/visualize/SeqOT.gif\" width=\"48%\" \u003e  \nFig. 1 System overview with Haomo dataset and visualized evaluation on NCLT dataset. It can be seen that SeqOT is robust to driving directions.  \n\n---\n## News!\n**[2022-12]** Our paper is accepted by **IEEE Transactions on Industrial Electronics (TIE)**!\n\n\n\n## Table of Contents\n1. [Publication](#Publication)\n2. [Dependencies](#Dependencies)\n3. [How to Use](#How-to-Use)\n4. [Data Preparation](#Data-Preparation)\n5. [License](#License)\n\n## Publication\n\nIf you use the code in your work, please cite our [paper](https://ieeexplore.ieee.org/document/9994714):\n\n```\n@ARTICLE{ma2022tie,\n  author={Ma, Junyi and Chen, Xieyuanli and Xu, Jingyi and Xiong, Guangming},\n  journal={IEEE Transactions on Industrial Electronics}, \n  title={SeqOT: A Spatial-Temporal Transformer Network for Place Recognition Using Sequential LiDAR Data}, \n  year={2022},\n  doi={10.1109/TIE.2022.3229385}}\n```\n\n\n## Dependencies\n\nWe use pytorch-gpu for neural networks.\n\nAn nvidia GPU is needed for faster retrival.\n\nTo use a GPU, first you need to install the nvidia driver and CUDA.\n\n- CUDA Installation guide: [link](https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html)  \n  We use CUDA 11.3 in our work. Other versions of CUDA are also supported but you should choose the corresponding torch version in the following Torch dependences.  \n\n- System dependencies:\n\n  ```bash\n  sudo apt-get update \n  sudo apt-get install -y python3-pip python3-tk\n  sudo -H pip3 install --upgrade pip\n  ```\n- Torch dependences:  \n  Following this [link](https://pytorch.org/get-started/locally/), you can download Torch dependences by pip:\n  ```bash\n  pip3 install torch==1.10.2+cu113 torchvision==0.11.3+cu113 torchaudio==0.10.2+cu113 -f https://download.pytorch.org/whl/cu113/torch_stable.html\n  ```\n  or by conda:\n  ```bash\n  conda install pytorch torchvision torchaudio cudatoolkit=11.3 -c pytorch\n  ```  \n  \n\n- Other Python dependencies (may also work with different versions than mentioned in the requirements file):\n\n  ```bash\n  sudo -H pip3 install -r requirements.txt\n  ```\n  \n## How to Use\n  \nWe provide a training and test tutorial for NCLT sequences in this repository. \n  \n### Training\n\nYou can start the two-step training with\n\n```\ncd train\npython training_seqot.py \npython gen_sub_descriptors.py\npython training_gem.py \n```\nNote that we only train our model using the oldest sequence of NCLT dataset (2012-01-08), to prove that our model works well for long time spans even if seeing limited data.  \n\n### Test\n\nThe sub-descriptors of the database and query sequence have already been generated by `python gen_sub_descriptors.py` above. Here you can test SeqOT by\n\n```\ncd test\npython test_gem_prepare.py\npython test_seqot.py\n```\n\nBefore training and test, please modify the params in the [config.yml](https://github.com/BIT-MJY/SeqOT/blob/main/config/config.yml)\n\n##### data_root\n* range_image_database_root: path of reference range images in database sequence.\n* range_image_query_root: path of query range images in query sequence.\n##### training_seqot\n* traindata_file: index for training. ([link](https://drive.google.com/file/d/1jEcnuHjEi0wqe8GAoh6UTa4UXTu0sDPr/view?usp=share_link))\n* poses_file: poses of the trajectory of the database. ([link](https://drive.google.com/file/d/1NusFV_xMI1s7i2s3u9vKdHnw0dWI7uii/view?usp=share_link))\n* height: height of range images.\n* width: width of range images.\n* seqlen: sequence length to generate sub-descriptors.\n* lr: learning rate for training feature extracter.\n* resume: whether resuming from the pretrained model below.\n* weights: pretrained feature extracter. ([link](https://drive.google.com/file/d/17EDuLzolOFjE7bq9BbwqDDnnWwiTlqWB/view?usp=share_link))\n##### gen_sub_descriptors\n* seqlen: sequence length to generate sub-descriptors.\n* weights: pretrained feature extracter. ([link](https://drive.google.com/file/d/17EDuLzolOFjE7bq9BbwqDDnnWwiTlqWB/view?usp=share_link))\n##### training_gem\n* traindata_file: index for training (same as the one of training_seqot).\n* poses_file: poses of the trajectory of the database (same as the one of training_seqot).\n* descriptors_file: sub-descriptors of database for training (generated by gen_sub_descriptors.py).\n* seqlen: sequence length to generate final-descriptors.\n* lr: learning rate for training the pooling module.\n* resume: whether resuming from the pretrained model below.\n* weights: pretrained pooling module. ([link](https://drive.google.com/file/d/1MdwavdbegA6P739mKlC2xJAcmZSbVD6N/view?usp=share_link))\n##### test_gem_prepare\n* sub_descriptors_database_file: sub-descriptors of database scans (generated by gen_sub_descriptors.py).\n* sub_descriptors_query_file: sub-descriptors of query scans (generated by gen_sub_descriptors.py).\n* seqlen: sequence length to generate final-descriptors (same as the one of training_gem).\n* weights: pretrained pooling module. ([link](https://drive.google.com/file/d/1MdwavdbegA6P739mKlC2xJAcmZSbVD6N/view?usp=share_link))\n##### test_seqot\ngroud_truth_file: ground truth containing correct loops. ([link](https://drive.google.com/file/d/13-tpLQiHK4krd-womDV6UPevvHUIFNyF/view?usp=share_link))\n##### viz\n* poses_database: poses of the trajectory of the database. ([link](https://drive.google.com/file/d/1NusFV_xMI1s7i2s3u9vKdHnw0dWI7uii/view?usp=share_link))\n* poses_query: poses of the trajectory of the query sequence. ([link](https://drive.google.com/file/d/1NBZeYK5giNr5r5l0TI0Ov9W_M8mvZT4a/view?usp=share_link))\n##### gen_training_index\n* poses_database: poses of the trajectory of the database. ([link](https://drive.google.com/file/d/1NusFV_xMI1s7i2s3u9vKdHnw0dWI7uii/view?usp=share_link))\n* poses_query: poses of the trajectory of the query sequence. ([link](https://drive.google.com/file/d/1NBZeYK5giNr5r5l0TI0Ov9W_M8mvZT4a/view?usp=share_link))\n* scan_database_root: path of reference .bin of the database. ([link](https://s3.us-east-2.amazonaws.com/nclt.perl.engin.umich.edu/velodyne_data/2012-01-08_vel.tar.gz))\n\n\n## Data Preparation\n\nPlease use the following commands to prepare data. We have uploaded all the necessary files mentioned above except the range images which you can generate easily following our instruction.\n\n### Range Image Generation\n\nPlease use [this script](https://github.com/BIT-MJY/SeqOT/blob/main/tools/utils/gen_depth_data.py) to generate range images of NCLT dataset. You need to modify the params including `scan_folder` and `dst_folder`.\n\n### Training Index Generation\n\nPlease use [this script](https://github.com/BIT-MJY/SeqOT/blob/main/data_prepararion/gen_training_index.py) to generate training indices of NCLT dataset. You need to modify the params including `poses_database`, `poses_query`, and `scan_database_root`.\n\n```\ncd data_prepararion\npython ./gen_training_index.py \n```\n### Groud Truth Generation\n\nPlease use [this script](https://github.com/BIT-MJY/SeqOT/blob/main/data_prepararion/gen_ground_truth.py) to generate groud truth loops of NCLT dataset. \n\n```\ncd data_prepararion\npython ./gen_ground_truth.py \n```\n## License\n\nCopyright 2022, Junyi Ma, Xieyuanli Chen, Jingyi Xu, Guangming Xiong, Beijing Institute of Technology.\n\nThis project is free software made available under the MIT License. For more details see the LICENSE file.\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBIT-MJY%2FSeqOT","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FBIT-MJY%2FSeqOT","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FBIT-MJY%2FSeqOT/lists"}