Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mit-spark/kimera

Index repo for Kimera code
https://github.com/mit-spark/kimera

3d-reconstruction computer-vision robotics semantics slam visual-inertial-odometry

Last synced: about 2 months ago
JSON representation

Index repo for Kimera code

Awesome Lists containing this project

README 

        


  

    sparklab

  

  

    kimera

  

  

    mit

  




# Kimera



Kimera is a C++ library for real-time metric-semantic simultaneous localization and mapping, which uses camera images and inertial data to build a semantically annotated 3D mesh of the environment. Kimera is modular, ROS-enabled, and runs on a CPU.



Kimera comprises four **modules**:

- A fast and accurate Visual Inertial Odometry (VIO) pipeline ([Kimera-VIO](https://github.com/MIT-SPARK/Kimera-VIO))

- A full SLAM implementation based on Robust Pose Graph Optimization ([Kimera-RPGO](https://github.com/MIT-SPARK/Kimera-RPGO))

- A per-frame and multi-frame 3D mesh generator ([Kimera-Mesher](https://github.com/MIT-SPARK/Kimera-VIO))

- And a generator of semantically annotated 3D meshes ([Kimera-Semantics](https://github.com/MIT-SPARK/Kimera-Semantics))





    

    Kimera

    




Click on the following links to install Kimera's modules and get started! It is very easy to install!



### [Kimera-VIO & Kimera-Mesher](https://github.com/MIT-SPARK/Kimera-VIO)





    

      Kimera-VIO

   




### [Kimera-RPGO](https://github.com/MIT-SPARK/Kimera-RPGO)





  

    Kimera-RPGO

  




### [Kimera-Semantics](https://github.com/MIT-SPARK/Kimera-Semantics)





  

    Kimera-Semantics

  




### Chart



![overall_chart](./docs/media/kimera_chart_23.jpeg)



## Citation



If you found any of the above modules useful, we would really appreciate if you could cite our work:



- [1] A. Rosinol, T. Sattler, M. Pollefeys, L. Carlone. [**Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities**](https://arxiv.org/abs/1903.01067). IEEE Int. Conf. on Robotics and Automation (ICRA), 2019. [arXiv:1903.01067](https://arxiv.org/abs/1903.01067)



```bibtex

@InProceedings{Rosinol19icra-incremental,

  title = {Incremental visual-inertial 3d mesh generation with structural regularities},

  author = {Rosinol, Antoni and Sattler, Torsten and Pollefeys, Marc and Carlone, Luca},

  year = {2019},

  booktitle = {2019 International Conference on Robotics and Automation (ICRA)},

  pdf = {https://arxiv.org/pdf/1903.01067.pdf}

}

```



- [2] A. Rosinol, M. Abate, Y. Chang, L. Carlone, [**Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping**](https://arxiv.org/abs/1910.02490). IEEE Intl. Conf. on Robotics and Automation (ICRA), 2020. [arXiv:1910.02490](https://arxiv.org/abs/1910.02490).

 

 ```bibtex

 @InProceedings{Rosinol20icra-Kimera,

   title = {Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping},

   author = {Rosinol, Antoni and Abate, Marcus and Chang, Yun and Carlone, Luca},

   year = {2020},

   booktitle = {IEEE Intl. Conf. on Robotics and Automation (ICRA)},

   url = {https://github.com/MIT-SPARK/Kimera},

   pdf = {https://arxiv.org/pdf/1910.02490.pdf}

 }

```



- [3] A. Rosinol, A. Gupta, M. Abate, J. Shi, L. Carlone. [**3D Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans**](https://arxiv.org/abs/2002.06289). Robotics: Science and Systems (RSS), 2020. [arXiv:2002.06289](https://arxiv.org/abs/2002.06289).



```bibtex

@InProceedings{Rosinol20rss-dynamicSceneGraphs,

  title = {{3D} Dynamic Scene Graphs: Actionable Spatial Perception with Places, Objects, and Humans},

  author = {A. Rosinol and A. Gupta and M. Abate and J. Shi and L. Carlone},

  year = {2020},

  booktitle = {Robotics: Science and Systems (RSS)},

  pdf = {https://arxiv.org/pdf/2002.06289.pdf}

}

```



- [4] A. Rosinol, A. Gupta, M. Abate, J. Shi, L. Carlone. [**Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs**](https://arxiv.org/abs/2101.06894). [arXiv:2101.06894](https://arxiv.org/abs/2101.06894).



```bibtex

@InProceedings{Rosinol21arxiv-Kimera,

  title = {{K}imera: from {SLAM} to Spatial Perception with {3D} Dynamic Scene Graphs},

  author = {A. Rosinol, A. Violette, M. Abate, N. Hughes, Y. Chang, J. Shi, A. Gupta, L. Carlone},

  year = {2021},

  booktitle = {arxiv},

  pdf = {https://arxiv.org/pdf/2101.06894.pdf}

}

```



## Open-Source Datasets



In addition to the [real-life tests](http://ci-sparklab.mit.edu:8080/job/MIT-SPARK-Kimera/job/master/VIO_20Euroc_20Performance_20Report/) on the [Euroc](https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets) dataset, we use a photo-realistic Unity-based simulator to test Kimera.

The simulator provides:

- RGB Stereo camera

- Depth camera

- Ground-truth 2D Semantic Segmentation

- IMU data

- Ground-Truth Odometry

- 2D Lidar

- TF (ground-truth odometry of robots, and agents)

- Static TF (ground-truth poses of static objects)



Using this simulator, we created several large visual-inertial datasets which feature scenes with and without dynamic agents (humans), as well as a large variety of environments (indoors and outdoors, small and large).

These are ideal to test your Metric-Semantic SLAM and/or other Spatial-AI systems!



- [uHumans](http://web.mit.edu/sparklab/datasets/uHumans/) (released with [3])

- [uHumans2](http://web.mit.edu/sparklab/datasets/uHumans2/) (released with [4])



## Acknowledgments



Kimera is partially funded by ARL [DCIST](https://www.dcist.org/), [ONR RAIDER](https://www.onr.navy.mil/), [MIT Lincoln Laboratory](https://www.ll.mit.edu/), and

[“la Caixa” Foundation](https://becarioslacaixa.net/en/antoni-rosinol-vidal-B004789) (ID 100010434), LCF/BQ/AA18/11680088 (A. Rosinol).



## License



[BSD License](LICENSE.BSD)