{"id":13528521,"url":"https://github.com/ETH3D/badslam","last_synced_at":"2025-04-01T13:33:10.163Z","repository":{"id":36787804,"uuid":"192239549","full_name":"ETH3D/badslam","owner":"ETH3D","description":"Bundle Adjusted Direct RGB-D SLAM","archived":false,"fork":false,"pushed_at":"2024-02-14T01:03:42.000Z","size":7014,"stargazers_count":703,"open_issues_count":52,"forks_count":116,"subscribers_count":26,"default_branch":"master","last_synced_at":"2024-11-02T14:35:47.579Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ETH3D.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-16T21:43:54.000Z","updated_at":"2024-10-18T02:29:05.000Z","dependencies_parsed_at":"2024-11-02T14:32:01.447Z","dependency_job_id":"327d60d0-6ed4-4b3c-ba05-d6494adb44b2","html_url":"https://github.com/ETH3D/badslam","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ETH3D%2Fbadslam","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ETH3D%2Fbadslam/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ETH3D%2Fbadslam/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ETH3D%2Fbadslam/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ETH3D","download_url":"https://codeload.github.com/ETH3D/badslam/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246647805,"owners_count":20811387,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T07:00:20.614Z","updated_at":"2025-04-01T13:33:06.917Z","avatar_url":"https://github.com/ETH3D.png","language":"C++","funding_links":[],"categories":["2. Visual SLAM","RGB-D SLAM"],"sub_categories":["2.4 RGBD","Bundle Adjusted Direct RGB-D SLAM"],"readme":"# BAD SLAM #\n\n## Overview ##\n\nBAD SLAM is a real-time approach for Simultaneous Localization and Mapping (SLAM) for RGB-D cameras.\nSupported platforms are Linux and Windows. The software requires an NVidia graphics card with CUDA compute capability 5.3 or later\n(however, it would be easy to lower this requirement).\n\nThis repository contains the\n[BAD SLAM application](https://github.com/ETH3D/badslam/tree/master/applications/badslam)\nand the library it is based on,\n[libvis](https://github.com/ETH3D/badslam/tree/master/libvis).\nThe library is work-in-progress and it is not recommended to use it for other projects at this point.\n\nThe application and library code is licensed under the BSD license, but please\nalso notice the licenses of the included or externally used third-party components.\n\nIf you use the provided code for research, please cite the paper describing the approach:\n\n[Thomas Schöps, Torsten Sattler, Marc Pollefeys, \"BAD SLAM: Bundle Adjusted Direct RGB-D SLAM\", CVPR 2019.](http://openaccess.thecvf.com/content_CVPR_2019/html/Schops_BAD_SLAM_Bundle_Adjusted_Direct_RGB-D_SLAM_CVPR_2019_paper.html)\n\nThe Windows port and Kinect-for-Azure (K4A) integration has been contributed by Silvano Galliani (Microsoft AI \u0026 Vision Zurich).\n\n\n## Screenshots \u0026 Videos ##\n\n| Main window   | Surfel normals display | Keyframe inspection |\n| ------------- | ---------------------- | ------------------- | \n| ![Main Window](applications/badslam/doc/main_window.png?raw=true) | ![Settings](applications/badslam/doc/surfel_normals.png?raw=true) | ![Keyframe dialog](applications/badslam/doc/keyframe_dialog.png?raw=true) |\n\n* [Oral presentation of the conference paper by Torsten Sattler at CVPR 2019](https://www.youtube.com/watch?v=0lLnHe0xbZE)\n* [Short demonstration of some of the SLAM GUI features](https://youtu.be/g3yD9qmDW4M)\n\n\n## Camera requirements ##\n\nPlease keep in mind that BAD SLAM has been designed for\nhigh-quality RGB-D videos and is likely to perform badly (no pun intended) on\nlower-quality RGB-D videos. For more details, see the [documentation on camera compatibility](https://github.com/ETH3D/badslam/blob/master/applications/badslam/doc/camera.md).\n\n\n## Pre-built binaries ##\n\nBinaries are available for download as [GitHub releases](https://github.com/ETH3D/badslam/releases).\n\n#### Windows ####\n\nFor Windows, an executable compiled with Visual Studio 2019 is provided.\nIt is also required to download the loop closure resource files as described below in this ReadMe, or loop closures will be disabled. In addition,\nperforming CUDA block-size autotuning as also described below is recommended.\n\nIf the executable fails to start due to missing DLLs, try installing the latest [Visual C++ redistributable files for Visual Studio 2019](https://aka.ms/vs/16/release/vc_redist.x64.exe).\n\n#### Linux ####\n\nFor Linux, an [AppImage](https://appimage.org/) is provided. Please note that it is also required to download\nthe loop closure resource files as described below in this ReadMe, or loop closures will be disabled. In addition,\nperforming CUDA block-size autotuning as also described below is recommended.\n\nIn case you encounter an error like\n```\n./badslam: relocation error: [...]/libQt5DBus.so.5: symbol dbus_message_get_allow_interactive_authorization, version LIBDBUS_1_3 not defined in file libdbus-1.so.3 with link time reference\n```\nthen your dbus library is too old. This can be fixed by downloading a recent version and setting `LD_LIBRARY_PATH` to the directory containing these files before starting the AppImage.\n\n\n## Building ##\n\nBuilding has been tested on Ubuntu 14.04 and Ubuntu 18.04 (with gcc), and on Windows (with Visual Studio 2019 and 2017).\n\nThe following external dependencies are required.\n\n| Dependency   | Version(s) known to work |\n| ------------ | ------------------------ |\n| [Boost](https://www.boost.org/) | 1.54.0 |\n| [CUDA](https://developer.nvidia.com/cuda-downloads) | 8, 9.1, 10.1 |\n| [DLib](https://github.com/dorian3d/DLib) | commit b6c28fb |\n| [Eigen](http://eigen.tuxfamily.org/index.php?title=Main_Page) | 3.3.7 |\n| [g2o](https://github.com/RainerKuemmerle/g2o) |  |\n| [GLEW](http://glew.sourceforge.net/build.html) |  |\n| [GTest](https://github.com/google/googletest) |  |\n| [OpenCV](https://opencv.org/) | 3.1.0, 3.2.0, 3.4.5, 3.4.6; 4.x does NOT work without changes |\n| [OpenGV](https://github.com/laurentkneip/opengv) | in Visual Studio 2017 it compiles only in debug mode |\n| [Qt](https://www.qt.io/) | 5.12.0; minimum version: 5.8 |\n| [SuiteSparse](http://faculty.cse.tamu.edu/davis/suitesparse.html) |  |\n| [zlib](https://zlib.net/) |  |\n\nNotice that OpenCV is only required as a dependency for loop detection by DLib.\nAlso notice that there are (at least) two different libraries with the name DLib,\nso be sure to install the correct one.\n\nThe following external dependencies are optional.\n\n| Dependency   | Purpose |\n| ------------ | ------- | \n| [librealsense2](https://github.com/IntelRealSense/librealsense) | Live input from RealSense D400 series depth cameras (tested with the D435 only). |\n| [Structure SDK](https://structure.io/developers) | Live input from Structure Core cameras (tested with the color version only). To use this, set the SCSDK_ROOT CMake variable to the SDK path. |\n| [k4a \u0026 k4arecord](https://github.com/Microsoft/Azure-Kinect-Sensor-SDK) | Live input from Azure Kinect cameras. |\n\n\n#### Build instructions for Linux ####\n\nSince OpenGV (at the time of writing) always uses the `-march=native` flag,\nboth BAD SLAM and g2o must use this as well. (For g2o, check that the\n`BUILD_WITH_MARCH_NATIVE` CMake option is set to ON.) If there are inconsistencies, the program\nwill likely crash when OpenGV or g2o functionality is used (i.e., at loop closures).\n\nAfter obtaining all dependencies, the application can be built with CMake, for example as follows:\n\n```bash\nmkdir build_RelWithDebInfo\ncd build_RelWithDebInfo\ncmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_CUDA_FLAGS=\"-arch=sm_61\" ..\nmake -j badslam  # Reduce the number of threads if running out of memory, e.g., -j3\n```\n\nMake sure to specify suitable CUDA architecture(s) in CMAKE_CUDA_FLAGS.\nCommon settings would either be the CUDA architecture of your graphics card only (in case\nyou only intend to run the compiled application on the system it was compiled on), or a range of virtual\narchitectures (in case the compiled application is intended for distribution).\nSee the [corresponding CUDA documentation](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-gpu-code-generation-gpu-architecture).\n\nOptionally, after building, the unit tests can be run, which test some of the\nbundle adjustment functionality. To do so, build and run the following executable:\n\n```\nmake -j badslam_test\n./build_RelWithDebInfo/applications/badslam/badslam_test\n```\n\nAll tests should pass. Troubleshooting:\n\n* If you get a CUDA error like \"too many resources requested for launch\", probably\n  a default CUDA kernel block size does not work for your GPU. See below for\n  block-size tuning. The application has been tested on GTX 1080 and GTX 1070 GPUs.\n* If the `Optimization.PoseGraphOptimizer` test crashes, for example with an error\n  message like \"Cholesky failure\", then please verify that your build of g2o has\n  the `BUILD_WITH_MARCH_NATIVE` CMake option set to ON, and that BAD SLAM actually\n  uses this install of g2o.\n\n\n#### Build instructions for Windows ####\n\nThe application can be built by creating a Visual Studio 2019 solution for it with CMake,\nthen compiling the \"badslam\" project in this solution.\n\nIt seemed that a workaround was required to prevent some unresolved external symbols in g2o_csparse_extension\n(for example, duplicating the problematic functions into g2o_solver_csparse).\n\nMake sure to specify suitable CUDA architecture(s) in CMAKE_CUDA_FLAGS.\nCommon settings would either be the CUDA architecture of your graphics card only (in case\nyou only intend to run the compiled application on the system it was compiled on), or a range of virtual\narchitectures (in case the compiled application is intended for distribution).\nSee the [corresponding CUDA documentation](https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-gpu-code-generation-gpu-architecture).\n\n\n## Dataset format ##\n\nFor CUDA block-size tuning (see below), at least one dataset should be obtained,\neven if one intends to run the program with live input.\n\nThe program supports datasets in the format of the [ETH3D SLAM Benchmark](https://www.eth3d.net/slam_overview) for RGB-D videos.\nThis is an extension of the format introduced by the\n[TUM RGB-D benchmark](https://vision.in.tum.de/data/datasets/rgbd-dataset),\ncontaining two small additions:\n\n* The original format does not specify the intrinsic camera calibration.\n  BAD SLAM thus additionally expects a file `calibration.txt` in the\n  dataset directory, consisting of a single line of text structured as follows:\n  ```\n  fx fy cx cy\n  ```\n  These values specify the parameters for the pinhole projection (fx * x + cx, fy * y + cy).\n  The coordinate system convention for cx and cy is that the origin (0, 0) of pixel coordinates is at\n  the **center** of the top-left pixel in the image.\n* The [associate.py](https://svncvpr.in.tum.de/cvpr-ros-pkg/trunk/rgbd_benchmark/rgbd_benchmark_tools/src/rgbd_benchmark_tools/associate.py)\n  tool from the benchmark must be run as follows to associate\n  the color and depth images:\n  ```\n  python associate.py rgb.txt depth.txt \u003e associated.txt\n  ```\n\n\n## Initial setup ##\n\nAfter building the executable and obtaining a dataset, there are two more steps to be done before running the program.\n\nFirst, the resource files for loop closure handling should be set up\n(unless the parameter `--no_loop_detection` is used to disable loop detection).\nDownload the [resource files](https://drive.google.com/file/d/1MpZwPjXDAUxKfSTpeCjG0PAUpaeWuo7D) of the [DLoopDetector](https://github.com/dorian3d/DLoopDetector) demo.\nThe two relevant files from this archive, `brief_k10L6.voc` and `brief_pattern.yml`, must be extracted into\na directory named \"resources\" in the application executable's directory (or an\nanalogous symlink must be created), for example:\n\n```\n- build_RelWithDebInfo\n  - applications\n    - badslam\n      - badslam (executable file)\n      - resources\n        - brief_k10L6.voc (notice that this is compressed in the archive and needs to be extracted separately)\n        - brief_pattern.yml\n```\n\nSecond, the CUDA kernel block size auto-tuning should be run. This is not strictly\nrequired in case the default sizes work for your GPU, but strongly recommended.\nThis step serves two purposes:\n\n* Sometimes, CUDA kernels won't launch with a given thread block size since this\n  would require too many resources. Block size auto-tuning determines and avoids\n  those problematic configurations.\n* The best block sizes to call CUDA kernels may vary between different graphics\n  cards, and the best way to figure them out is to benchmark it, which the\n  tuning does.\n\nTo test your GPU, run the badslam executable with the provided tuning\nscript on any dataset in sequential mode:\n\n```\npython scripts/auto_tune_parameters.py \u003cpath_to_badslam_executable\u003e \u003cpath_to_dataset\u003e --sequential_ba --sequential_loop_detection\n```\n\nThe script will run the program multiple times using different parameters and\nmeasure the runtime, i.e., do not run another computing task at the same time to\nnot influence the measurements. It should output a file `auto_tuning_result.txt`\nand intermediate files `auto_tuning_iteration_X.txt`. Move the result file into\nthe `resources` directory used by BAD SLAM (where the loop detector resources\nare also stored in). The file will be loaded automatically if it exists in this\ndirectory. The intermediate files can be deleted.\n\nSince the program runs multiple times, you may want to limit the number of\nframes it runs on to speed it up with `--end_frame`. Also, please notice that\ntuning data will only be gathered for CUDA kernels that run during the tuning.\nIf later other kernels run during the actual program invocation, they will still\nuse the default block size. So, if for example you want to tune the PCG-related\nkernels instead of those for alternating optimization, then you need to pass the\ncorresponding parameter `--use_pcg` in the tuning call. Since the tuning result\nfiles are simple plain text files, the results of multiple tuning runs with\ndifferent parameters could be easily merged to create a tuning file that covers\nall kernels. Doing this automatically would be a possible future addition to the\ntuning script.\n\n\n## Running ##\n\nThe simplest way to start the program is without any command-line arguments:\n\n```\n./build_RelWithDebInfo/applications/badslam/badslam\n```\n\nIt will show a settings window then that allows to select a dataset or live input, and\nallows to adjust a variety of parameters.\n\nAlternatively, the program can be run without visualization by specifying\nall parameters on the command line. If parameters are given on the command line,\nthe visualization can be used with the `--gui` flag (to start showing the settings window)\nor the `--gui_run` flag (to start running immediately).\n\nFor example, to immediately start running SLAM on a dataset in the GUI, use:\n\n```\n./build_RelWithDebInfo/applications/badslam/badslam \u003cdataset_path\u003e --gui_run\n```\n\nSee the [documentation on command line parameters](https://github.com/ETH3D/badslam/blob/master/applications/badslam/doc/command_line.md) for more details.\n\nThe first time the program runs on a dataset, the performance might be\nlimited by the time it takes to read the image files from the hard disk\n(unless the dataset is on an SSD, or is already cached because the files were written recently).\nSubsequent runs should be faster as long as the files remain cached.\n\nPlease also notice that the real-time mode with parallel odometry and bundle adjustment, despite being the default,\nwas added late in the development process and should be considered potentially unstable (in particular\nwhen optimizing the depth camera's deformation, which lacks synchronization for the access to a GPU buffer).\nThus, to possibly increase robustness, use the `--sequential_ba` parameter.\nLive operation may still be simulated by also specifying `--target_frame_rate \u003cdesired_fps\u003e`.\n\n## Docker ##\nTo build the image, do:\n\n```\n$ docker build  --build-arg CUDA_ARCH=\"DESIRED_ARCH\" -t eth3d/badslam .\n```\n\nwhere `DESIRED_ARCH` corresponds to the CUDA architecture you wish to build\nwith (for example: `sm_61`).\n\nTo run the image using an example dataset, download \u0026 unzip the dataset, then invoke:\n\n```\n$ docker run  --gpus all -it -e DISPLAY  -v /tmp/.X11-unix:/tmp/.X11-unix:ro  --mount type=bind,source=ABSOLUTE_PATH_TO_DATASET,target=/datasets eth3d/badslam  /bin/bash\n```\n\nUsing the \"einstein\" dataset as an example, you could run\n\n```\n$ ./applications/badslam/badslam /datasets/einstein_1 --export_reconstruction einstein.ply \u0026\u0026 meshlab einstein.ply\n```\n\nNote: if you observe something like:\n\n```\nqt.qpa.xcb: could not connect to display :0.0  \n```\n\nMake sure to:\n\n```\n$ xhost + \n```\n\nin your terminal before running the container.\n\n## Extending BAD SLAM ##\n\nContributions to this open source project are very welcome. Please try to\nfollow the existing coding style (which is loosely inspired by the Google C++\ncoding style, but somewhat relaxed in some aspects).\n\nIf you are interested in using the direct bundle adjustment component\nwithout SLAM, then the intrinsics optimization unit test might be a good\nstarting point, showing how to set up keyframes and perform optimization.\nIt is at `applications/badslam/src/badslam/test/test_intrinsics_optimization_[photometric/geometric]_residual.cc`.\n\nIf you plan to change the cost function used for bundle adjustment, you may\nwant to have a look at `scripts/jacobians_derivation.py`. This script\nautomatically computes the Jacobians required for optimization from a\nspecification of the residuals in Python. It also outputs somewhat optimized\nC++ functions to compute the residual, the Jacobian, and both the residual and\nJacobian at the same time. The script requires sympy to run. Its main limitation\nis that it operates on a symbolic representation of the residual (instead of on\nthe algorithm for residual computation, as an autodiff tool would do), which means\nthat its internal residual term may become huge. This may cause excessive\nruntimes of the script for more complex residuals. You can try removing the simplify() calls\nin `jacobian_functions.py` to speed it up, while applying less simplification to\nthe resulting expressions.\n\n\n## Differences to the paper ##\n\nThe open source version of the code has undergone strong refactoring compared to\nthe version used to produce the results in the paper, many new features have\nbeen added, and many fixes were done. The photometric residual used for global optimization is slightly\ndifferent: Instead of using the gradient magnitude as the photometric descriptor,\nthe two components of the gradient are used separately. For these reasons,\nit should not be expected that the code reproduces the results in the paper\nexactly, however the results should be similar.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FETH3D%2Fbadslam","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FETH3D%2Fbadslam","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FETH3D%2Fbadslam/lists"}