{"id":13438326,"url":"https://github.com/hku-mars/r3live","last_synced_at":"2025-05-15T12:03:39.036Z","repository":{"id":37683699,"uuid":"404724835","full_name":"hku-mars/r3live","owner":"hku-mars","description":"A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package","archived":false,"fork":false,"pushed_at":"2024-05-28T10:27:17.000Z","size":23103,"stargazers_count":2148,"open_issues_count":0,"forks_count":453,"subscribers_count":51,"default_branch":"master","last_synced_at":"2025-04-14T19:59:26.383Z","etag":null,"topics":["3d-mapping","3d-reconstruction","lidar-camera-fusion","lidar-inertial-odometry","lidar-odometry","lidar-slam","mesh-reconstruction","sensor-fusion","slam"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hku-mars.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-09-09T13:05:05.000Z","updated_at":"2025-04-13T08:01:54.000Z","dependencies_parsed_at":"2023-02-14T00:30:59.742Z","dependency_job_id":"c8ac4683-537a-41ee-ae8b-77365b0269e4","html_url":"https://github.com/hku-mars/r3live","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hku-mars%2Fr3live","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hku-mars%2Fr3live/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hku-mars%2Fr3live/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hku-mars%2Fr3live/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hku-mars","download_url":"https://codeload.github.com/hku-mars/r3live/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254337612,"owners_count":22054253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-mapping","3d-reconstruction","lidar-camera-fusion","lidar-inertial-odometry","lidar-odometry","lidar-slam","mesh-reconstruction","sensor-fusion","slam"],"created_at":"2024-07-31T03:01:04.569Z","updated_at":"2025-05-15T12:03:33.993Z","avatar_url":"https://github.com/hku-mars.png","language":"C++","funding_links":[],"categories":["C++","五、Code","1. 机器人项目 | Robots","Awesome LiDAR SLAM Algorithms"],"sub_categories":["4. Lidar SLAM"],"readme":"# R3LIVE\n## A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package\n\n\n## 1. Introduction\n**R3LIVE** is a novel LiDAR-Inertial-Visual sensor fusion framework, which takes advantage of measurement of LiDAR, inertial, and visual sensors to achieve robust and accurate state estimation. R3LIVE is built upon our previous work [R2LIVE](https://github.com/hku-mars/r2live), is contained of two subsystems: the LiDAR-inertial odometry (LIO) and the visual-inertial odometry (VIO). The LIO subsystem ([FAST-LIO](https://github.com/hku-mars/FAST_LIO)) takes advantage of the measurement from LiDAR and inertial sensors and builds the geometric structure of (i.e. the position of 3D points) global maps. The VIO subsystem utilizes the data of visual-inertial sensors and renders the map's texture (i.e. the color of 3D points). \u003cbr\u003e\n\nThe source code of this package is released under [**GPLv2**](http://www.gnu.org/licenses/) license. We only allow it free for personal and academic usage. For commercial use, please contact me \u003cziv.lin.ljrATgmail.com\u003e and Dr. Fu Zhang \u003cfuzhangAThku.hk\u003e to negotiate a different license.\n\n### 1.1 Our paper\nOur paper has been accepted to **ICRA2022**, which is\navailable online on this [page](https://ieeexplore.ieee.org/document/9811935) or be downloaded [here](https://github.com/hku-mars/r3live/blob/master/papers/R3LIVE%20--%20A%20Robust%2C%20Real-time%2C%20RGB-colored%2C%20LiDAR-Inertial-Visual%20tightly-coupled%20stateEstimation%20and%20mapping%20package.pdf).\n\n### 1.2 Our accompanying videos\nOur **accompanying videos** are now available on YouTube (click below images to open) and Bilibili\u003csup\u003e[1](https://www.bilibili.com/video/BV1d341117d6?share_source=copy_web), [2](https://www.bilibili.com/video/BV1e3411q7Di?share_source=copy_web)\u003c/sup\u003e.\n\u003cdiv align=\"center\"\u003e\n\u003ca href=\"https://youtu.be/j5fT8NE5fdg\" target=\"_blank\"\u003e\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/R3LIVE_paper.jpg\" alt=\"video\" width=\"48%\" /\u003e\u003c/a\u003e\n\u003ca href=\"https://youtu.be/4rjrrLgL3nk\" target=\"_blank\"\u003e\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/R3LIVE_demos.jpg\" alt=\"video\" width=\"48%\" /\u003e\u003c/a\u003e\n\u003c/div\u003e\n\n### 1.3 Our associate dataset: R3LIVE-dataset\nOur associate dataset [**R3LIVE-dataset**](https://github.com/ziv-lin/r3live_dataset) that use for evaluation is also available online. You can access and download our datasets via this [**Github repository**](https://github.com/ziv-lin/r3live_dataset).\n\n### 1.4 Our open-source hardware design\nAll of the mechanical modules of our handheld device that use for data collection are designed as FDM printable, with the schematics of the design are also open-sourced in this [**Github repository**](https://github.com/ziv-lin/rxlive_handheld).\n\n## 2. What can R3LIVE do?\n\n### 2.1 Strong robustness in various challenging scenarios\nR3LIVE is robust enough to work well in various of LiDAR-degenerated scenarios (see following figures): \n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/degenerate_01_pic.png\" width=\"98%\" /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/degenerate_02_pic.png\" width=\"98%\" /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/gifs/degenerate_01.gif\"  width=\"48%\" /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/gifs/degenerate_02.gif\"  width=\"48%\" /\u003e\n\u003c/div\u003e\n\nAnd even in simultaneously LiDAR degenerated and visual texture-less environments (see Experiment-1 of our [paper](https://github.com/hku-mars/r3live/blob/master/papers/R3LIVE:%20A%20Robust%2C%20Real-time%2C%20RGB-colored%2C%20LiDAR-Inertial-Visual%20tightly-coupled%20stateEstimation%20and%20mapping%20package.pdf)).  \n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/exp_00.png\" alt=\"video\" width=\"48%\" /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/gifs/degenerate_00.gif\" alt=\"video\" width=\"48%\" /\u003e\n\u003c/div\u003e\n\n### 2.2 Real-time RGB maps reconstruction\nR3LIVE is able to reconstruct the precise, dense, 3D, RGB-colored maps of surrounding environment in real-time (watch this [video](https://youtu.be/j5fT8NE5fdg)).\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/cover_half.jpg\" width = 98% /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/hku_campus_seq_01.png\" width = 98% /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/hku_park_01.png\" width = 98% /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/gifs/hku_demo.gif\"  width=\"48%\" /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/gifs/hkust_demo.gif\" alt=\"video\" width=\"48%\" /\u003e\n\u003c/div\u003e\n\n### 2.3 Ready for 3D applications\nTo make R3LIVE more extensible, we also provide a series of offline utilities for reconstructing and texturing meshes, which further reduce the gap between R3LIVE and various 3D applications (watch this [video](https://youtu.be/4rjrrLgL3nk)).\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/pics/mesh.png\" alt=\"video\" width=\"98%\" /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/gifs/ue_game_0.gif\" alt=\"video\" width=\"48%\" /\u003e\n\u003cimg src=\"https://github.com/ziv-lin/r3live_dataset/blob/main/gifs/ue_game_1.gif\"  width=\"48%\" /\u003e\n\u003c/div\u003e\n\n## 3. Prerequisites\n### 3.1 **ROS**\nFollowing this [ROS Installation](http://wiki.ros.org/ROS/Installation) to install ROS and its additional pacakge:\u003cbr\u003e\n```\nsudo apt-get install ros-XXX-cv-bridge ros-XXX-tf ros-XXX-message-filters ros-XXX-image-transport ros-XXX-image-transport*\n```\n**NOTICE:** remember to replace \"XXX\" on above command as your ROS distributions, for example, if your use ROS-kinetic, the command should be:\u003cbr\u003e\n```\nsudo apt-get install ros-kinetic-cv-bridge ros-kinetic-tf ros-kinetic-message-filters ros-kinetic-image-transport*\n```\n### 3.2. **livox_ros_driver**\nFollow this [livox_ros_driver Installation](https://github.com/Livox-SDK/livox_ros_driver).\n\n### 3.3 **CGAL** and **pcl_viewer** (optional)\n```\nsudo apt-get install libcgal-dev pcl-tools\n```\n### 3.4 **OpenCV \u003e= 3.3**\nYou can use the following command to check your OpenCV version, **if your openCV version lower than OpenCV-3.3**, we recommend you to update your you openCV version if you meet errors in complying our codes. Otherwise, skip this step ^_^\n```\npkg-config --modversion opencv\n```\nWe have successfully test our algorithm with version **3.3.1**, **3.4.16**, **4.2.1** and **4.5.3**.\n\n**Notice:** We have noticed that a large number of users meet a crash of problem after launching our package due to the mismatch of openCV version (see  [issue #11](https://github.com/hku-mars/r3live/issues/11), [issue #20](https://github.com/hku-mars/r3live/issues/20) and [issue #23](https://github.com/hku-mars/r3live/issues/23), and etc.). If you meet with similar problems, please make sure that the OpenCV you complied are same as the OpenCV you run with.  This is **very very** important to launch R3LIVE correctly.\n\n## 4. Build R3LIVE on ROS:\nClone this repository and catkin_make:\n```\ncd ~/catkin_ws/src\ngit clone https://github.com/hku-mars/r3live.git\ncd ../\ncatkin_make\nsource ~/catkin_ws/devel/setup.bash\n```\n\n## 5. Run our examples\n### 5.1 Download our rosbag files ([r3live_dataset](https://github.com/ziv-lin/r3live_dataset)) \nOur datasets for evaluation can be download from our [Google drive](https://drive.google.com/drive/folders/15i-TRa0EA8BCbNdARVqPMDsU9JOlagVF?usp=sharing) or [Baidu-NetDisk [百度网盘]](https://pan.baidu.com/s/1zmVxkcwOSul8oTBwaHfuFg) (code提取码: wwxw). We have released totally **9** rosbag files for evaluating r3live, with the introduction of these datasets can be found on this [page](https://github.com/ziv-lin/r3live_dataset).\n\n### 5.2 Run our examples\nAfter you have downloaded our bag files, you can now run our example ^_^\n```\nroslaunch r3live r3live_bag.launch\nrosbag play YOUR_DOWNLOADED.bag\n```\nIf everything is correct, you will get the result that matches our [paper](https://github.com/hku-mars/r3live/blob/master/papers/R3LIVE:%20A%20Robust%2C%20Real-time%2C%20RGB-colored%2C%20LiDAR-Inertial-Visual%20tightly-coupled%20stateEstimation%20and%20mapping%20package.pdf) and the results posted on this [page](https://github.com/ziv-lin/r3live_dataset). \n\n### 5.3 Save the maps to your disk\nR3LIVE allow you to save the maps you build at anytime you wanted. You just need to click on the \"Control panel\" and press 'S' or 's' key.\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"./control_panel.png\" alt=\"video\" width=\"80%\" /\u003e\n\u003c/div\u003e\n\n### 5.3 Reconstruct and texture your mesh\nAfter you have save your offline map on your disk (default save in directory: ${HOME}/r3live_output), you can launch our utility to reconstruct and texture your mesh.\n```\nroslaunch r3live r3live_reconstruct_mesh.launch\n```\n\n### 5.4 Visualize your saved maps.\nAs default, your offline map (and reconstructed mesh) will be saved in the directory **${HOME}/r3live_output**, you can open it with pcl_viewer (and [meshlab](https://www.meshlab.net/)).\n\nInstall pcl_viewer and meshlab:\n```\nsudo apt-get install pcl-tools meshlab\n```\n\nVisualizing your offline point cloud maps (with suffix *.pcd):\n```\ncd ${HOME}/r3live_output\npcl_viewer rgb_pt.pcd\n```\n\nVisualizing your reconstructed mesh (with suffix *.ply):\n```\ncd ${HOME}/r3live_output\nmeshlab textured_mesh.ply\n```\n\n## 6. Sample and run your own data\n### 6.1  Livox-ros-driver for R2/R3LIVE\nSince the LiDAR data and IMU data published by the official Livox-ros-driver is with the timestamp of LiDAR (started from 0 in each recording), and the timestamp of the image is usually recorded with the timestamp of the operation system. To make them working under the same time-based, we modified the source code of Livox-ros-driver, which is available at [here](https://github.com/ziv-lin/livox_ros_driver_for_R2LIVE). We suggest you replace the official driver with it when sampling your own data for R3LIVE.\n### 6.2  Sensor calibration\nIn order to launch R3LIVE on your own hardware setup, you need to have a carefully calibration of the extrinsic among LiDAR, Camera and IMU. We recommend you using the following repo to kindly calibrate your sensors:\n\n[**livox_camera_calib**](https://github.com/hku-mars/livox_camera_calib): A robust, high accuracy extrinsic calibration tool between high resolution LiDAR (e.g. Livox) and camera in targetless environment. \n\n## 7. Support of the spinning LiDAR\nEven though our proposed method is unrelated to what kind of LiDAR you used, it is impossible for us making R3LIVE compatible with all kinds of existing LiDARs. To launch R3LIVE with spinning LIDAR, it requires you to take some effort in modifying the source code of our LiDAR front-end (see [LiDAR_front_end.cpp](https://github.com/hku-mars/r3live/blob/master/r3live/src/loam/LiDAR_front_end.cpp)). Here we give an example to test our LIO-subsystem with an [Ouster-2 64 Line Spinning LIDAR](https://ouster.com/zh-cn/products/scanning-lidar/os2-sensor/).\n\n###  7.1 Example-1: Ouster OS2-64\nDownload our recorded rosbag file from [here](https://drive.google.com/file/d/1mZoDWDOZOcZ0H6MBMEpGMRc1nueeqToc/view?usp=sharing).\n```\nroslaunch r3live r3live_bag_ouster.launch \nrosbag play ouster_example_for_LIO_test.bag \n```\nNotice: We manually disable our VIO-subsystem due the missed of calibration files in this example.\n\nFinally, we are still working on making R3LIVE compatible with more spinning LiDARs. More and more examples will be released in the future.\n\n## 8. Access our open source hardware design\nIn order to facilitate our users in reproducing our work, we also make our hardware design public available, where you can download all of our CAD source files in [rxlive_handheld](https://github.com/ziv-lin/rxlive_handheld).\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"https://github.com/ziv-lin/rxlive_handheld/blob/master/pics/introduction_alpha.png\" width=\"98%\" /\u003e\n\u003c/div\u003e\n\n## 9. Report our problems and bugs\nWe know our packages might not totally stable in this stage, and we are keep working on improving the performance and reliability of our codes. So, if you have met any bug or problem, please feel free to draw an issue and I will respond ASAP.\n\n\u0026emsp; For reporting our problems and bugs, please attach both your hardware and software environment if possible (printed by R3LIVE, see the following figure), which will be a great help for me in locating your problems.\n\n\u003cdiv align=\"center\"\u003e\n\u003cimg src=\"./envs.png\" alt=\"video\" width=\"80%\" /\u003e\n\u003c/div\u003e\n\n## 10. Acknowledgments\nIn the development of R3LIVE, we stand on the shoulders of the following repositories:\n1. [R2LIVE](https://github.com/hku-mars/r2live): A robust, real-time tightly-coupled multi-sensor fusion package.\n2. [FAST-LIO](https://github.com/hku-mars/FAST_LIO): A computationally efficient and robust LiDAR-inertial odometry package.\n3. [ikd-Tree](https://github.com/hku-mars/ikd-Tree): A state-of-art dynamic KD-Tree for 3D kNN search. \n4. [livox_camera_calib](https://github.com/hku-mars/livox_camera_calib): A robust, high accuracy extrinsic calibration tool between high resolution LiDAR (e.g. Livox) and camera in targetless environment.\n5. [LOAM-Livox](https://github.com/hku-mars/loam_livox): A robust LiDAR Odometry and Mapping (LOAM) package for Livox-LiDAR.\n6. [openMVS](https://github.com/cdcseacave/openMVS): A library for computer-vision scientists and especially targeted to the Multi-View Stereo reconstruction community.  \n8. [VCGlib](https://github.com/cnr-isti-vclab/vcglib): An open source, portable, header-only Visualization and Computer Graphics Library.\n9. [CGAL](): A C++ Computational Geometry Algorithms Library.\n\n## License\nThe source code of this package is released under [**GPLv2**](http://www.gnu.org/licenses/) license. We only allow it free for personal and academic usage. For commercial use, please contact me \u003cziv.lin.ljrATgmail.com\u003e and Dr. Fu Zhang \u003cfuzhangAThku.hk\u003e to negotiate a different license.\n\nWe are still working on improving the performance and reliability of our codes. For any technical issues, please contact me via email Jiarong Lin \u003c ziv.lin.ljrATgmail.com \u003e.\n\nIf you use any code of this repo in your academic research, please cite **at least one** of our papers:\n```\n[1] Lin, Jiarong, and Fu Zhang. \"R3LIVE: A Robust, Real-time, RGB-colored, LiDAR-Inertial-Visual tightly-coupled state Estimation and mapping package.\" \n[2] Xu, Wei, et al. \"Fast-lio2: Fast direct lidar-inertial odometry.\"\n[3] Lin, Jiarong, et al. \"R2LIVE: A Robust, Real-time, LiDAR-Inertial-Visual tightly-coupled state Estimator and mapping.\" \n[4] Xu, Wei, and Fu Zhang. \"Fast-lio: A fast, robust lidar-inertial odometry package by tightly-coupled iterated kalman filter.\"\n[5] Cai, Yixi, Wei Xu, and Fu Zhang. \"ikd-Tree: An Incremental KD Tree for Robotic Applications.\"\n[6] Lin, Jiarong, and Fu Zhang. \"Loam-livox: A fast, robust, high-precision LiDAR odometry and mapping package for LiDARs of small FoV.\"\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhku-mars%2Fr3live","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhku-mars%2Fr3live","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhku-mars%2Fr3live/lists"}