{"id":13528616,"url":"https://github.com/KumarRobotics/msckf_vio","last_synced_at":"2025-04-01T14:32:38.069Z","repository":{"id":41225865,"uuid":"112651426","full_name":"KumarRobotics/msckf_vio","owner":"KumarRobotics","description":"Robust Stereo Visual Inertial Odometry for Fast Autonomous Flight","archived":false,"fork":false,"pushed_at":"2023-11-22T09:11:28.000Z","size":44,"stargazers_count":1799,"open_issues_count":10,"forks_count":607,"subscribers_count":103,"default_branch":"master","last_synced_at":"2025-03-31T12:01:33.589Z","etag":null,"topics":["ros","stereo-vision","visual-inertial-odometry"],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KumarRobotics.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-11-30T19:28:22.000Z","updated_at":"2025-03-30T14:57:41.000Z","dependencies_parsed_at":"2024-01-12T17:34:31.360Z","dependency_job_id":null,"html_url":"https://github.com/KumarRobotics/msckf_vio","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KumarRobotics%2Fmsckf_vio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KumarRobotics%2Fmsckf_vio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KumarRobotics%2Fmsckf_vio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KumarRobotics%2Fmsckf_vio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KumarRobotics","download_url":"https://codeload.github.com/KumarRobotics/msckf_vio/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246655183,"owners_count":20812594,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ros","stereo-vision","visual-inertial-odometry"],"created_at":"2024-08-01T07:00:21.845Z","updated_at":"2025-04-01T14:32:38.042Z","avatar_url":"https://github.com/KumarRobotics.png","language":"C++","readme":"# MSCKF\\_VIO\n\n\nThe `MSCKF_VIO` package is a stereo version of MSCKF. The software takes in synchronized stereo images and IMU messages and generates real-time 6DOF pose estimation of the IMU frame.\n\nThe software is tested on Ubuntu 16.04 with ROS Kinetic.\n\nVideo: [https://www.youtube.com/watch?v=jxfJFgzmNSw\u0026t](https://www.youtube.com/watch?v=jxfJFgzmNSw\u0026t=3s)\u003cbr/\u003e\nPaper Draft: [https://arxiv.org/abs/1712.00036](https://arxiv.org/abs/1712.00036)\n\n## License\n\nPenn Software License. See LICENSE.txt for further details.\n\n## Dependencies\n\nMost of the dependencies are standard including `Eigen`, `OpenCV`, and `Boost`. The standard shipment from Ubuntu 16.04 and ROS Kinetic works fine. One special requirement is `suitesparse`, which can be installed through,\n\n```\nsudo apt-get install libsuitesparse-dev\n```\n\n## Compling\nThe software is a standard catkin package. Make sure the package is on `ROS_PACKAGE_PATH` after cloning the package to your workspace. And the normal procedure for compiling a catkin package should work.\n\n```\ncd your_work_space\ncatkin_make --pkg msckf_vio --cmake-args -DCMAKE_BUILD_TYPE=Release\n```\n\n## Calibration\n\nAn accurate calibration is crucial for successfully running the software. To get the best performance of the software, the stereo cameras and IMU should be hardware synchronized. Note that for the stereo calibration, which includes the camera intrinsics, distortion, and extrinsics between the two cameras, you have to use a calibration software. **Manually setting these parameters will not be accurate enough.** [Kalibr](https://github.com/ethz-asl/kalibr) can be used for the stereo calibration and also to get the transformation between the stereo cameras and IMU. The yaml file generated by Kalibr can be directly used in this software. See calibration files in the `config` folder for details. The two calibration files in the `config` folder should work directly with the EuRoC and [fast flight](https://github.com/KumarRobotics/msckf_vio/wiki) datasets. The convention of the calibration file is as follows:\n\n`camx/T_cam_imu`: takes a vector from the IMU frame to the camx frame.\n`cam1/T_cn_cnm1`: takes a vector from the cam0 frame to the cam1 frame.\n\nThe filter uses the first 200 IMU messages to initialize the gyro bias, acc bias, and initial orientation. Therefore, the robot is required to start from a stationary state in order to initialize the VIO successfully.\n\n\n## EuRoC and UPenn Fast flight dataset example usage\n\nFirst obtain either the [EuRoC](https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets) or the [UPenn fast flight](https://github.com/KumarRobotics/msckf_vio/wiki/Dataset) dataset.\n\nRecommended EuRoC ROS Bags:\n- [Vicon Room 1 01](http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/V1_01_easy/V1_01_easy.bag)\n- [Vicon Room 1 02](http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/V1_02_easy/V1_02_easy.bag)\n\nOnce the `msckf_vio` is built and sourced (via `source \u003cpath to catkin_ws\u003e/devel/setup.bash`), there are two launch files prepared for the [EuRoC](https://projects.asl.ethz.ch/datasets/doku.php?id=kmavvisualinertialdatasets) and [UPenn fast flight](https://github.com/KumarRobotics/msckf_vio/wiki/Dataset) dataset named `msckf_vio_euroc.launch` and `msckf_vio_fla.launch` respectively. Each launch files instantiates two ROS nodes:\n\n* `image_processor` processes stereo images to detect and track features\n* `vio` obtains feature measurements from the `image_processor` and tightly fuses them with the IMU messages to estimate pose.\n\nThese launch files can be executed via\n\n```\nroslaunch msckf_vio msckf_vio_euroc.launch\n```\nor\n\n```\nroslaunch msckf_vio msckf_vio_fla.launch\n```\n\nOnce the nodes are running you need to run the dataset rosbags (in a different terminal), for example:\n\n```\nrosbag play V1_01_easy.bag\n```\n\nAs mentioned in the previous section, **The robot is required to start from a stationary state in order to initialize the VIO successfully.**\n\nTo visualize the pose and feature estimates you can use the provided rviz configurations found in `msckf_vio/rviz` folder (EuRoC: `rviz_euroc_config.rviz`, Fast dataset: `rviz_fla_config.rviz`).\n\n\n## ROS Nodes\n\n### `image_processor` node\n\n**Subscribed Topics**\n\n`imu` (`sensor_msgs/Imu`)\n\nIMU messages is used for compensating rotation in feature tracking, and 2-point RANSAC.\n\n`cam[x]_image` (`sensor_msgs/Image`)\n\nSynchronized stereo images.\n\n**Published Topics**\n\n`features` (`msckf_vio/CameraMeasurement`)\n\nRecords the feature measurements on the current stereo image pair.\n\n`tracking_info` (`msckf_vio/TrackingInfo`)\n\nRecords the feature tracking status for debugging purpose.\n\n`debug_stereo_img` (`sensor_msgs::Image`)\n\nDraw current features on the stereo images for debugging purpose. Note that this debugging image is only generated upon subscription.\n\n### `vio` node\n\n**Subscribed Topics**\n\n`imu` (`sensor_msgs/Imu`)\n\nIMU measurements.\n\n`features` (`msckf_vio/CameraMeasurement`)\n\nStereo feature measurements from the `image_processor` node.\n\n**Published Topics**\n\n`odom` (`nav_msgs/Odometry`)\n\nOdometry of the IMU frame including a proper covariance.\n\n`feature_point_cloud` (`sensor_msgs/PointCloud2`)\n\nShows current features in the map which is used for estimation.\n","funding_links":[],"categories":["3. Visual Inertial SLAM","Softwares and Libraries"],"sub_categories":["3.3 Stereo"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKumarRobotics%2Fmsckf_vio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FKumarRobotics%2Fmsckf_vio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FKumarRobotics%2Fmsckf_vio/lists"}