Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/PRBonn/MapMOS

Building Volumetric Beliefs for Dynamic Environments Exploiting Map-Based Moving Object Segmentation (RAL 2023)
https://github.com/PRBonn/MapMOS

cloud deep-learning map minkowski-engine minkowskiengine mos moving object point point-cloud segmentation static

Last synced: 4 months ago
JSON representation

Building Volumetric Beliefs for Dynamic Environments Exploiting Map-Based Moving Object Segmentation (RAL 2023)

Awesome Lists containing this project

README 

        


  
Building Volumetric Beliefs for Dynamic Environments Exploiting Map-Based Moving Object Segmentation


  

    

    

    





  






  Our approach identifies moving objects in the current scan (blue points) and the local map (black points) of the environment and maintains a volumetric belief map representing the dynamic environment.




Click here for qualitative results!



[![MapMOS](https://github.com/PRBonn/MapMOS/assets/38326482/a4238431-bd2d-4b2c-991b-7ff5e9378a8e)](https://github.com/PRBonn/MapMOS/assets/38326482/04c7e5a2-dd44-431a-95b0-c42d5605078a)



 Our predictions for the KITTI Tracking sequence 19 with true positives (green), false positives (red), and false negatives (blue).






## Installation

First, make sure the [MinkowskiEngine](https://github.com/NVIDIA/MinkowskiEngine) is installed on your system, see [here](https://github.com/NVIDIA/MinkowskiEngine#installation) for more details.



Next, clone our repository

```bash

git clone [email protected]:PRBonn/MapMOS && cd MapMOS

```



and install with

```bash

make install

```



**or**

```bash

make install-all

```

if you want to install the project with all optional dependencies (needed for the visualizer). In case you want to edit the Python code, install in editable mode:

```bash

make editable

```



## How to Use It

Just type



```bash

mapmos_pipeline --help

```

to see how to run MapMOS.



This is what you should see



![Screenshot from 2023-08-03 13-07-14](https://github.com/PRBonn/MapMOS/assets/38326482/c769afa6-709d-4648-b42d-11092d5b92ac)



Check the [Download](#downloads) section for a pre-trained model. Like [KISS-ICP](https://github.com/PRBonn/kiss-icp), our pipeline runs on a variety of point cloud data formats like `bin`, `pcd`, `ply`, `xyz`, `rosbags`, and more. To visualize these, just type



```bash

mapmos_pipeline --visualize /path/to/weights.ckpt /path/to/data

```



Want to evaluate with ground truth labels?



Because these labels come in all shapes, you need to specify a dataloader. This is currently available for SemanticKITTI, NuScenes, HeLiMOS, and our labeled KITTI Tracking sequence 19 and Apollo sequences (see [Downloads](#downloads)).



Want to reproduce the results from the paper?

For reproducing the results of the paper, you need to pass the corresponding config file. They will make sure that the de-skewing option and the maximum range are set properly. To compare different map fusion strategies from our paper, just pass the `--paper` flag to the `mapmos_pipeline`.



## Training

To train our approach, you need to first cache your data. To see how to do that, just `cd` into the `MapMOS` repository and type



```bash

python3 scripts/precache.py --help

```



After this, you can run the training script. Again, `--help` shows you how:

```bash

python3 scripts/train.py --help

```



Want to verify the cached data?



You can inspect the cached training samples by using the script `python3 scripts/cache_to_ply.py --help`.



Want to change the logging directory?



The training log and checkpoints will be saved by default to the current working directory. To change that, export the `export LOGS=/your/path/to/logs` environment variable before running the training script.



## HeLiMOS

We provide additional training and evaluation data for different sensor types in our [HeLiMOS paper](https://www.ipb.uni-bonn.de/pdfs/lim2024iros.pdf). To train on the HeLiMOS data, use the following commands:



```shell

python3 scripts/precache.py /path/to/HeLiMOS helimos /path/to/cache --config config/helimos/*_training.yaml

python3 scripts/train.py /path/to/HeLiMOS helimos /path/to/cache --config config/helimos/*_training.yaml

```



by replacing the paths and the config file names. To evaluate for example on the Velodyne test data, run



```shell

mapmos_pipeline /path/to/weights.ckpt /path/to/HeLiMOS --dataloader helimos -s Velodyne/test.txt

```



Note that our sequence `-s` encodes both the sensor type `Velodyne` and split `test.txt`, just replace these with `Ouster`, `Aeva`, or `Avia` and/or `train.txt` or `val.txt` to run MapMOS on different sensors and/or splits.



## Downloads

You can download the post-processed and labeled [Apollo dataset](https://www.ipb.uni-bonn.de/html/projects/apollo_dataset/LiDAR-MOS.zip) and [KITTI Tracking sequence 19](https://www.ipb.uni-bonn.de/html/projects/kitti-tracking/post-processed/kitti-tracking.zip) from our website.



The [weights](https://www.ipb.uni-bonn.de/html/projects/MapMOS/mapmos.ckpt) of our pre-trained model can be downloaded as well.



## Publication

If you use our code in your academic work, please cite the corresponding [paper](https://www.ipb.uni-bonn.de/pdfs/mersch2023ral.pdf):



```bibtex

@article{mersch2023ral,

  author = {B. Mersch and T. Guadagnino and X. Chen and I. Vizzo and J. Behley and C. Stachniss},

  title = {{Building Volumetric Beliefs for Dynamic Environments Exploiting Map-Based Moving Object Segmentation}},

  journal = {IEEE Robotics and Automation Letters (RA-L)},

  volume = {8},

  number = {8},

  pages = {5180--5187},

  year = {2023},

  issn = {2377-3766},

  doi = {10.1109/LRA.2023.3292583},

  codeurl = {https://github.com/PRBonn/MapMOS},

}

```



## Acknowledgments

This implementation is heavily inspired by [KISS-ICP](https://github.com/PRBonn/kiss-icp).



## License

This project is free software made available under the MIT License. For details see the LICENSE file.