{"id":15031155,"url":"https://github.com/mit-spark/kimera-vio","last_synced_at":"2025-05-14T07:08:36.122Z","repository":{"id":37502456,"uuid":"209074567","full_name":"MIT-SPARK/Kimera-VIO","owner":"MIT-SPARK","description":"Visual Inertial Odometry with SLAM capabilities and 3D Mesh generation.","archived":false,"fork":false,"pushed_at":"2025-03-01T16:04:28.000Z","size":188933,"stargazers_count":1660,"open_issues_count":51,"forks_count":435,"subscribers_count":62,"default_branch":"master","last_synced_at":"2025-04-11T20:43:40.609Z","etag":null,"topics":["cpu","euroc-dataset","localization","mapping","real-time","reconstruction","robotics","slam","state-estimation","vio","visual-inertial-odometry"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-2-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MIT-SPARK.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.BSD","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-17T14:20:56.000Z","updated_at":"2025-04-11T06:36:54.000Z","dependencies_parsed_at":"2023-02-08T20:15:16.090Z","dependency_job_id":"163adcfd-3cd6-498b-8436-c2b03d6b84c7","html_url":"https://github.com/MIT-SPARK/Kimera-VIO","commit_stats":{"total_commits":3253,"total_committers":25,"mean_commits":130.12,"dds":0.3301567783584384,"last_synced_commit":"2c7dff1941088e9fe9028f623afb2897451ff2ef"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FKimera-VIO","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FKimera-VIO/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FKimera-VIO/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MIT-SPARK%2FKimera-VIO/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MIT-SPARK","download_url":"https://codeload.github.com/MIT-SPARK/Kimera-VIO/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254092775,"owners_count":22013290,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cpu","euroc-dataset","localization","mapping","real-time","reconstruction","robotics","slam","state-estimation","vio","visual-inertial-odometry"],"created_at":"2024-09-24T20:15:00.539Z","updated_at":"2025-05-14T07:08:31.112Z","avatar_url":"https://github.com/MIT-SPARK.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ca href=\"http://mit.edu/sparklab/\"\u003e\n    \u003cimg align=\"left\" src=\"docs/media/sparklab_logo.png\" width=\"80\" alt=\"sparklab\"\u003e\n  \u003c/a\u003e \n  \u003ca href=\"https://www.mit.edu/~arosinol/\"\u003e\n    \u003cimg align=\"center\" src=\"docs/media/kimeravio_logo.png\" width=\"150\" alt=\"kimera\"\u003e\n  \u003c/a\u003e \n  \u003ca href=\"https://mit.edu\"\u003e \n    \u003cimg align=\"right\" src=\"docs/media/mit.png\" width=\"100\" alt=\"mit\"\u003e\n  \u003c/a\u003e\n\u003c/div\u003e\n\n# Kimera-VIO: Open-Source Visual Inertial Odometry\n\n[![Build Status](http://ci-sparklab.mit.edu:8080/job/MIT-SPARK-Kimera/job/master/badge/icon)](http://ci-sparklab.mit.edu:8080/job/MIT-SPARK-Kimera/job/master/)\nFor evaluation plots, check our [jenkins server](http://ci-sparklab.mit.edu:8080/job/MIT-SPARK-Kimera/job/master/VIO_20Euroc_20Performance_20Report).\n\n**Authors:** [Antoni Rosinol](https://www.mit.edu/~arosinol/), Yun Chang, Marcus Abate, Nathan Hughes, Sandro Berchier, [Luca Carlone](https://lucacarlone.mit.edu/)\n\n## What is Kimera-VIO?\n\nKimera-VIO is a Visual Inertial Odometry pipeline for accurate State Estimation from **Stereo** + **IMU** data. It can optionally use **Mono** + **IMU** data instead of stereo cameras.\n\n## Publications\n\nWe kindly ask to cite our paper if you find this library useful:\n\n- A. Rosinol, M. Abate, Y. Chang, L. Carlone, [**Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping**](https://arxiv.org/abs/1910.02490). IEEE Intl. Conf. on Robotics and Automation (ICRA), 2020. [arXiv:1910.02490](https://arxiv.org/abs/1910.02490).\n \n ```bibtex\n @InProceedings{Rosinol20icra-Kimera,\n   title = {Kimera: an Open-Source Library for Real-Time Metric-Semantic Localization and Mapping},\n   author = {Rosinol, Antoni and Abate, Marcus and Chang, Yun and Carlone, Luca},\n   year = {2020},\n   booktitle = {IEEE Intl. Conf. on Robotics and Automation (ICRA)},\n   url = {https://github.com/MIT-SPARK/Kimera},\n   pdf = {https://arxiv.org/pdf/1910.02490.pdf}\n }\n ```\n \n - A. Rosinol and A. Violette and M. Abate and N. Hughes and Y. Chang and J. Shi and A. Gupta and L. Carlone, [**Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs**](https://arxiv.org/abs/2101.06894). arXiv preprint, 2021. [arXiv:2101.06894](https://arxiv.org/abs/2101.06894).\n\n```bibtex\n@article{Rosinol21arxiv-Kimera,\n   title = {Kimera: from {SLAM} to Spatial Perception with {3D} Dynamic Scene Graphs},\n   author = {Rosinol, Antoni and Violette, Andrew and Abate, Marcus and Hughes, Nathan and Chang, Yun\n   and Shi, Jingnan and Gupta, Arjun and Carlone, Luca},\n   year = {2021},\n   journal = {arXiv preprint arXiv: 2101.06894},\n   pdf = {https://arxiv.org/pdf/2101.06894.pdf} \n}\n```\n\n### Related Publications\n\nBackend optimization is based on:\n\n - C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza. [**On-Manifold Preintegration Theory for Fast and Accurate Visual-Inertial Navigation**](https://arxiv.org/pdf/1512.02363.pdf). IEEE Trans. Robotics, 33(1):1-21, 2016.\n\n - L. Carlone, Z. Kira, C. Beall, V. Indelman, and F. Dellaert. [**Eliminating Conditionally Independent Sets in Factor Graphs: A Unifying Perspective based on Smart Factors.**](https://www.cc.gatech.edu/~dellaert/dhtml/pubs/Carlone14icra.pdf) IEEE Intl. Conf. on Robotics and Automation (ICRA), 2014.\n\nAlternatively, the `Regular VIO` Backend, using structural regularities, is described in this paper:\n\n- A. Rosinol, T. Sattler, M. Pollefeys, and L. Carlone. [**Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities**](https://arxiv.org/pdf/1903.01067.pdf). IEEE Int. Conf. on Robotics and Automation (ICRA), 2019.\n\n## Demo\n\n\u003cdiv align=\"center\"\u003e\n  \u003cimg src=\"docs/media/kimeravio_ROS_mesh.gif\"/\u003e\n\u003c/div\u003e\n\n# 1. Installation\n\nTested on Ubuntu 20.04.\n\n## Prerequisites\n\n- [GTSAM](https://github.com/borglab/gtsam) \u003e= 4.1\n- [OpenCV](https://github.com/opencv/opencv) \u003e= 3.4\n- [OpenGV](https://github.com/laurentkneip/opengv)\n- [Glog](http://rpg.ifi.uzh.ch/docs/glog.html), [Gflags](https://gflags.github.io/gflags/)\n- [Gtest](https://github.com/google/googletest/blob/master/googletest/docs/primer.md) (installed automagically).\n- [DBoW2](https://github.com/dorian3d/DBoW2)\n- [Kimera-RPGO](https://github.com/MIT-SPARK/Kimera-RPGO)\n- [ANMS](https://github.com/BAILOOL/ANMS-Codes) (source files in `src/frontend/feature-detector/anms`, used for adaptive non-max suppression).\n\n\u003e Note: if you want to avoid building all dependencies yourself, we provide a docker image that will install them for you. Check installation instructions in [docs/kimera_vio_install.md](./docs/kimera_vio_install.md).\n\n\u003e Note 2: if you use ROS, then [Kimera-VIO-ROS](https://github.com/MIT-SPARK/Kimera-VIO-ROS) can install all dependencies and Kimera inside a catkin workspace.\n\n## Installation Instructions\n\nFind how to install Kimera-VIO and its dependencies here: **[Installation instructions](./docs/kimera_vio_install.md)**.\n\n# 2. Usage\n\n## General tips\n\nThe LoopClosureDetector (and PGO) module is disabled by default. If you wish to run the pipeline with loop-closure detection enabled, set the `use_lcd` flag to true. For the example script, this is done by passing `-lcd` at commandline like so:\n```bash\n./scripts/stereoVIOEUROC.bash -lcd\n```\n\nTo log output, set the `log_output` flag to true. For the script, this is done with the `-log` commandline argument. By default, log files will be saved in [output_logs](output_logs/).\n\nTo run the pipeline in sequential mode (one thread only), set `parallel_run`to false. This can be done in the example script with the `-s` argument at commandline.\n\n## i. [Euroc](http://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets) Dataset\n\n#### Download Euroc's dataset\n\n- Download one of Euroc's datasets, for example [V1_01_easy.zip](http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/V1_01_easy/V1_01_easy.zip).\n\n\u003e Datasets MH_04 and V2_03 have different number of left/right frames. We suggest using instead [**our version of Euroc here**](https://drive.google.com/open?id=1_kwqHojvBusHxilcclqXh6haxelhJW0O).\n\n- Unzip the dataset to your preferred directory, for example, in `~/Euroc/V1_01_easy`:\n```bash\nmkdir -p ~/Euroc/V1_01_easy\nunzip -o ~/Downloads/V1_01_easy.zip -d ~/Euroc/V1_01_easy\n```\n\n#### Yamelize Euroc's dataset\n\nAdd `%YAML:1.0` at the top of each `.yaml` file inside Euroc.\nYou can do this manually or run the `yamelize.bash` script by indicating where the dataset is (it is assumed below to be in `~/path/to/euroc`):\n\u003e You don't need to yamelize the dataset if you download our version [here](https://drive.google.com/open?id=1_kwqHojvBusHxilcclqXh6haxelhJW0O)\n```bash\ncd Kimera-VIO\nbash ./scripts/euroc/yamelize.bash -p ~/path/to/euroc\n```\n\n### Run Kimera-VIO in Euroc's dataset\n\nUsing a bash script bundling all command-line options and gflags:\n\n```bash\ncd Kimera-VIO\nbash ./scripts/stereoVIOEuroc.bash -p \"PATH_TO_DATASET/V1_01_easy\"\n```\n\n\u003e Alternatively, one may directly use the executable in the build folder:\n`./build/stereoVIOEuroc`. Nevertheless, check the script `./scripts/stereoVIOEuroc.bash` to understand what parameters are expected, or check the [parameters](#Parameters) section below.\n\nKimera can also run in monocular mode. For Euroc, this means only processing the left image. To use this simply use the parameters in `params/EurocMono`. In the bash script there is a `PARAMS_PATH` variable that can be set to point to these parameters instead.\n\n## ii. Using [ROS wrapper](https://github.com/MIT-SPARK/Kimera-VIO-ROS)\n\nWe provide a ROS wrapper of Kimera-VIO that you can find at: https://github.com/MIT-SPARK/Kimera-VIO-ROS.\n\nThis library can be cloned into a catkin workspace and built alongside the ROS wrapper.\n\n## iii. Evaluation and Debugging\n\nFor more information on tools for debugging and evaluating the pipeline, see [our documentation](/docs/kimera_vio_debug_evaluation.md)\n\n## iv. Unit Testing\n\nWe use [gtest](https://github.com/google/googletest) for unit testing.\nTo run the unit tests: build the code, navigate inside the `build` folder and run `testKimeraVIO`:\n```bash\ncd build\n./testKimeraVIO\n```\n\nA useful flag is `./testKimeraVIO --gtest_filter=foo` to only run the test you are interested in (regex is also valid).\n\nAlternatively, you can run `rosrun kimera_vio run_gtest.py` from anywhere on your system if you've built Kimera-VIO through ROS and sourced the workspace containing Kimera-VIO. This script passes all arguments to `testKimeraVIO`, so you should feel free to use whatever flags you would normally use.\n\n# 3. Parameters\nKimera-VIO accepts two independent sources of parameters:\n- YAML files: contains parameters for Backend and Frontend.\n- [gflags](https://gflags.github.io/gflags/) contains parameters for all the rest.\n\nTo get help on what each gflag parameter does, just run the executable with the `--help` flag: `./build/stereoVIOEuroc --help`. You should get a list of gflags similar to the ones [here](./docs/gflags_parameters.md).\n\n  - Optionally, you can try the VIO using structural regularities, as in [our ICRA 2019 paper](https://ieeexplore.ieee.org/abstract/document/8794456), by specifying the option ```-r```: ```./stereoVIOEuroc.bash -p \"PATH_TO_DATASET/V1_01_easy\" -r```\n\nOpenCV's 3D visualization also has some shortcuts for interaction: check [tips for usage](./docs/tips_usage.md)\n\nCamera parameters can be described using the pinhole model or the omni model. The omni model is based on the OCamCalib toolbox described in [this paper](http://rpg.ifi.uzh.ch/docs/CCMVS2007_scaramuzza.pdf). A tutorial for generating the calibration can be found [here](https://sites.google.com/site/scarabotix/ocamcalib-toolbox).\n\nThe Omni camera model requires these additional parameters:\n\n```\nomni_affine\nomni_distortion_center\n```\n\nThe distortion polynomial is stored in the `distortion_coefficients` field. The matlab toolbox gives only 4 coefficients as output, however Kimera supports 5 coefficients. The second one can be set to zero. For example, if your output from OCamCalib is `[1, 2, 3, 4]` then you can set `distortion_coefficients` to `[1, 0, 2, 3, 4]` in the camera parameters file.\n\nInverse polynomial for projection is not required. In the omni camera case, the `intrinsics` field represents the intrinsics of a **ideal pinhole model** of the fisheye camera, which is primarily used when instantiating gtsam calibrations that currently are only implemented for pinhole cameras. Leaving it blank is sufficient as the code will generate a ideal model based on the image size. You may supply your own ideal pinhole intrinsics and they will be used instead. In the pinhole case, these values must be supplied consistently with the camera parameters (focal lengths and image center). \n\n# 4. Contribution guidelines\n\nWe strongly encourage you to submit issues, feedback and potential improvements.\nWe follow the branch, open PR, review, and merge workflow.\n\nTo contribute to this repo, ensure your commits pass the linter pre-commit checks.\nTo enable these checks you will need to [install linter](./docs/linter_install.md).\nWe also provide a `.clang-format` file with the style rules that the repo uses, so that you can use [`clang-format`](https://clang.llvm.org/docs/ClangFormat.html) to reformat your code.\n\nAlso, check [tips for development](./docs/tips_development.md) and our **[developer guide](./docs/developer_guide.md)**.\n\n# 5. FAQ\n\n### Issues\n  If you have problems building or running the pipeline and/or issues with dependencies, you might find useful information in our [FAQ](./docs/faq.md) or in the issue tracker.\n\n### How to interpret console output\n\n```\nI0512 21:05:55.136549 21233 Pipeline.cpp:449] Statistics\n-----------                                  #\tLog Hz\t{avg     +- std    }\t[min,max]\nData Provider [ms]                      \t    0\t\nDisplay [ms]                            \t  146\t36.5421\t{8.28082 +- 2.40370}\t[3,213]\nVioBackend [ms]                         \t   73\t19.4868\t{15.2192 +- 9.75712}\t[0,39]\nVioFrontend Frame Rate [ms]             \t  222\t59.3276\t{5.77027 +- 1.51571}\t[3,12]\nVioFrontend Keyframe Rate [ms]          \t   73\t19.6235\t{31.4110 +- 7.29504}\t[24,62]\nVioFrontend [ms]                        \t  295\t77.9727\t{12.1593 +- 10.7279}\t[3,62]\nVisualizer [ms]                         \t   73\t19.4639\t{3.82192 +- 0.805234}\t[2,7]\nbackend_input_queue Size [#]            \t   73\t18.3878\t{1.00000 +- 0.00000}\t[1,1]\ndata_provider_left_frame_queue Size (#) \t  663\t165.202\t{182.265 +- 14.5110}\t[1,359]\ndata_provider_right_frame_queue Size (#)\t  663\t165.084\t{182.029 +- 14.5150}\t[1,359]\ndisplay_input_queue Size [#]            \t  146\t36.5428\t{1.68493 +- 0.00000}\t[1,12]\nstereo_frontend_input_queue Size [#]    \t  301\t75.3519\t{4.84718 +- 0.219043}\t[1,5]\nvisualizer_backend_queue Size [#]       \t   73\t18.3208\t{1.00000 +- 0.00000}\t[1,1]\nvisualizer_frontend_queue Size [#]      \t  295\t73.9984\t{4.21695 +- 1.24381}\t[1,7]\n```\n\n- `#` number of samples taken.\n- `Log Hz` average number of samples taken per second in Hz.\n- `avg` average of the actual value logged. Same unit as the logged quantity.\n- `std` standard deviation of the value logged.\n- `[min,max]` minimum and maximum values that the logged value took.\n\nThere are two main things logged: the time it takes for the pipeline modules to run (i.e. `VioBackend`, `Visualizer` etc), and the size of the queues between pipeline modules (i.e. `backend_input_queue`).\n\nFor example:\n```\nVioBackend [ms]                         \t   73\t19.4868\t{15.2192 +- 9.75712}\t[0,39]\n```\n\nShows that the Backend runtime got sampled `73` times, at a rate of `19.48`Hz (which accounts for both the time the Backend waits for input to consume and the time it takes to process it). That it takes `15.21`ms to consume its input with a standard deviation of `9.75`ms and that the least it took to run for one input was `0`ms and the most it took so far is `39`ms.\n\nFor the queues, for example:\n```\nstereo_frontend_input_queue Size [#]    \t  301\t75.3519\t{4.84718 +- 0.219043}\t[1,5]\n```\n\nShows that the Frontend input queue got sampled `301` times, at a rate of `75.38`Hz. That it stores an average of `4.84` elements, with a standard deviation of `0.21` elements, and that the min size it had was `1` element, and the max size it stored was of `5` elements.\n\n# 6. Chart\n\n![vio_chart](./docs/media/kimeravio_chart.png)\n\n![overall_chart](./docs/media/kimera_chart_23.jpeg)\n\n# 7. BSD License\n\nKimera-VIO is open source under the BSD license, see the [LICENSE.BSD](LICENSE.BSD) file.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Fkimera-vio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmit-spark%2Fkimera-vio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmit-spark%2Fkimera-vio/lists"}