{"id":13528455,"url":"https://github.com/JakobEngel/dso","last_synced_at":"2025-04-01T13:32:51.707Z","repository":{"id":41838845,"uuid":"73256421","full_name":"JakobEngel/dso","owner":"JakobEngel","description":"Direct Sparse Odometry","archived":false,"fork":false,"pushed_at":"2024-02-23T16:15:26.000Z","size":971,"stargazers_count":2339,"open_issues_count":137,"forks_count":914,"subscribers_count":116,"default_branch":"master","last_synced_at":"2025-03-30T16:11:18.147Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JakobEngel.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2016-11-09T06:07:10.000Z","updated_at":"2025-03-25T19:36:21.000Z","dependencies_parsed_at":"2024-04-18T19:05:57.066Z","dependency_job_id":null,"html_url":"https://github.com/JakobEngel/dso","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JakobEngel%2Fdso","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JakobEngel%2Fdso/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JakobEngel%2Fdso/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JakobEngel%2Fdso/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JakobEngel","download_url":"https://codeload.github.com/JakobEngel/dso/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246647736,"owners_count":20811372,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T07:00:19.425Z","updated_at":"2025-04-01T13:32:51.674Z","avatar_url":"https://github.com/JakobEngel.png","language":"C++","funding_links":[],"categories":["SLAM","Semi-dense map","OpenSource-SLAM","2. Visual SLAM","C++","四 视觉SLAM从入门到进阶","视觉SLAM","OpenSource SLAM (Simultaneous Localization And Mapping)"],"sub_categories":["Monocular","2.2 Monocular","Code","UAV Trajectory Optimization for model completeness"],"readme":"# DSO: Direct Sparse Odometry\n\nFor more information see\n[https://vision.in.tum.de/dso](https://vision.in.tum.de/dso)\n\n### 1. Related Papers\n* **Direct Sparse Odometry**, *J. Engel, V. Koltun, D. Cremers*, In arXiv:1607.02565, 2016\n* **A Photometrically Calibrated Benchmark For Monocular Visual Odometry**, *J. Engel, V. Usenko, D. Cremers*, In arXiv:1607.02555, 2016\n\nGet some datasets from [https://vision.in.tum.de/mono-dataset](https://vision.in.tum.de/mono-dataset) .\n\n### 2. Installation\n\n\tgit clone https://github.com/JakobEngel/dso.git\n\n#### 2.1 Required Dependencies\n\n##### suitesparse and eigen3 (required).\nRequired. Install with\n\n\t\tsudo apt-get install libsuitesparse-dev libeigen3-dev libboost-all-dev\n\n\n\n#### 2.2 Optional Dependencies\n\n##### OpenCV (highly recommended).\nUsed to read / write / display images.\nOpenCV is **only** used in `IOWrapper/OpenCV/*`. Without OpenCV, respective \ndummy functions from `IOWrapper/*_dummy.cpp` will be compiled into the library, which do nothing.\nThe main binary will not be created, since it is useless if it can't read the datasets from disk.\nFeel free to implement your own version of these functions with your prefered library, \nif you want to stay away from OpenCV.\n\nInstall with\n\n\tsudo apt-get install libopencv-dev\n\n\n##### Pangolin (highly recommended).\nUsed for 3D visualization \u0026 the GUI.\nPangolin is **only** used in `IOWrapper/Pangolin/*`. You can compile without Pangolin, \nhowever then there is not going to be any visualization / GUI capability. \nFeel free to implement your own version of `Output3DWrapper` with your preferred library, \nand use it instead of `PangolinDSOViewer`\n\nInstall from [https://github.com/stevenlovegrove/Pangolin](https://github.com/stevenlovegrove/Pangolin)\n\n\n##### ziplib (recommended).\nUsed to read datasets with images as .zip, as e.g. in the TUM monoVO dataset. \nYou can compile without this, however then you can only read images directly (i.e., have \nto unzip the dataset image archives before loading them).\n\n\tsudo apt-get install zlib1g-dev\n\tcd dso/thirdparty\n\ttar -zxvf libzip-1.1.1.tar.gz\n\tcd libzip-1.1.1/\n\t./configure\n\tmake\n\tsudo make install\n\tsudo cp lib/zipconf.h /usr/local/include/zipconf.h   # (no idea why that is needed).\n\n##### sse2neon (required for ARM builds).\nAfter cloning, just run `git submodule update --init` to include this.  It translates Intel-native SSE functions to ARM-native NEON functions during the compilation process.\n\n#### 2.3 Build\n\n\t\tcd dso\n\t\tmkdir build\n\t\tcd build\n\t\tcmake ..\n\t\tmake -j4\n\t\nthis will compile a library `libdso.a`, which can be linked from external projects. \nIt will also build a binary `dso_dataset`, to run DSO on datasets. However, for this\nOpenCV and Pangolin need to be installed.\n\n\n\n\n\n\n### 3 Usage\nRun on a dataset from [https://vision.in.tum.de/mono-dataset](https://vision.in.tum.de/mono-dataset) using\n\n\t\tbin/dso_dataset \\\n\t\t\tfiles=XXXXX/sequence_XX/images.zip \\\n\t\t\tcalib=XXXXX/sequence_XX/camera.txt \\\n\t\t\tgamma=XXXXX/sequence_XX/pcalib.txt \\\n\t\t\tvignette=XXXXX/sequence_XX/vignette.png \\\n\t\t\tpreset=0 \\\n\t\t\tmode=0\n\nSee [https://github.com/JakobEngel/dso_ros](https://github.com/JakobEngel/dso_ros) for a minimal example on\nhow the library can be used from another project. It should be straight forward to implement extentions for \nother camera drivers, to use DSO interactively without ROS.\n\n\n\n#### 3.1 Dataset Format.\nThe format assumed is that of [https://vision.in.tum.de/mono-dataset](https://vision.in.tum.de/mono-dataset).\nHowever, it should be easy to adapt it to your needs, if required. The binary is run with:\n\n- `files=XXX` where XXX is either a folder or .zip archive containing images. They are sorted *alphabetically*. for .zip to work, need to comiple with ziplib support.\n\n- `gamma=XXX` where XXX is a gamma calibration file, containing a single row with 256 values, mapping [0..255] to the respective irradiance value, i.e. containing the *discretized inverse response function*. See TUM monoVO dataset for an example.\n\n- `vignette=XXX` where XXX is a monochrome 16bit or 8bit image containing the vignette as pixelwise attenuation factors. See TUM monoVO dataset for an example.\n\n- `calib=XXX` where XXX is a geometric camera calibration file. See below.\n\n\n\n##### Geometric Calibration File.\n\n\n###### Calibration File for Pre-Rectified Images\n\n    Pinhole fx fy cx cy 0\n    in_width in_height\n    \"crop\" / \"full\" / \"none\" / \"fx fy cx cy 0\"\n    out_width out_height\n\n###### Calibration File for FOV camera model:\n\n    FOV fx fy cx cy omega\n    in_width in_height\n    \"crop\" / \"full\" / \"fx fy cx cy 0\"\n    out_width out_height\n\n\n###### Calibration File for Radio-Tangential camera model\n\n    RadTan fx fy cx cy k1 k2 r1 r2\n    in_width in_height\n    \"crop\" / \"full\" / \"fx fy cx cy 0\"\n    out_width out_height\n\n\n###### Calibration File for Equidistant camera model\n\n    EquiDistant fx fy cx cy k1 k2 k3 k4\n    in_width in_height\n    \"crop\" / \"full\" / \"fx fy cx cy 0\"\n    out_width out_height\n\n\n(note: for backwards-compatibility, \"Pinhole\", \"FOV\" and \"RadTan\" can be omitted). See the respective\n`::distortCoordinates` implementation in  `Undistorter.cpp` for the exact corresponding projection function.\nFurthermore, it should be straight-forward to implement other camera models.\n\n\n**Explanation:**\n Across all models `fx fy cx cy` denotes the focal length / principal point **relative to the image width / height**, \ni.e., DSO computes the camera matrix `K` as\n\n\t\tK(0,0) = width * fx\n\t\tK(1,1) = height * fy\n\t\tK(0,2) = width * cx - 0.5\n\t\tK(1,2) = height * cy - 0.5\nFor backwards-compatibility, if the given `cx` and `cy` are larger than 1, DSO assumes all four parameters to directly be the entries of K, \nand ommits the above computation. \n\n\n**That strange \"0.5\" offset:**\n Internally, DSO uses the convention that the pixel at integer position (1,1) in the image, i.e. the pixel in the second row and second column,\ncontains the integral over the continuous image function from (0.5,0.5) to (1.5,1.5), i.e., approximates a \"point-sample\" of the \ncontinuous image functions at (1.0, 1.0).\nIn turn, there seems to be no unifying convention across calibration toolboxes whether the pixel at integer position (1,1)\ncontains the integral over (0.5,0.5) to (1.5,1.5), or the integral over (1,1) to (2,2). The above conversion assumes that \nthe given calibration in the calibration file uses the latter convention, and thus applies the -0.5 correction.\nNote that this also is taken into account when creating the scale-pyramid (see `globalCalib.cpp`).\n\n\n**Rectification modes:**\n For image rectification, DSO either supports rectification to a user-defined pinhole model (`fx fy cx cy 0`),\nor has an option to automatically crop the image to the maximal rectangular, well-defined region (`crop`).\n`full` will preserve the full original field of view and is mainly meant for debugging - it will create black \nborders in undefined image regions, which DSO does NOT ignore (i.e., this option will generate additional \noutliers along those borders, and corrupt the scale-pyramid).\n\n\n\n\n#### 3.2 Commandline Options\nthere are many command line options available, see `main_dso_pangolin.cpp`. some examples include\n- `mode=X`: \n    -  `mode=0` use iff a photometric calibration exists (e.g. TUM monoVO dataset). \n    -  `mode=1` use iff NO photometric calibration exists (e.g. ETH EuRoC MAV dataset). \n    -  `mode=2` use iff images are not photometrically distorted (e.g. synthetic datasets).\n\n- `preset=X`\n    - `preset=0`: default settings (2k pts etc.), not enforcing real-time execution\n    - `preset=1`: default settings (2k pts etc.), enforcing 1x real-time execution\n    - `preset=2`: fast settings (800 pts etc.), not enforcing real-time execution. WARNING: overwrites image resolution with 424 x 320.\n    - `preset=3`: fast settings (800 pts etc.), enforcing 5x real-time execution. WARNING: overwrites image resolution with 424 x 320.\n\n- `nolog=1`: disable logging of eigenvalues etc. (good for performance)\n- `reverse=1`: play sequence in reverse\n- `nogui=1`: disable gui (good for performance)\n- `nomt=1`: single-threaded execution\n- `prefetch=1`: load into memory \u0026 rectify all images before running DSO.\n- `start=X`: start at frame X\n- `end=X`: end at frame X\n- `speed=X`: force execution at X times real-time speed (0 = not enforcing real-time)\n- `save=1`: save lots of images for video creation\n- `quiet=1`: disable most console output (good for performance)\n- `sampleoutput=1`: register a \"SampleOutputWrapper\", printing some sample output data to the commandline. meant as example.\n\n\n\n#### 3.3 Runtime Options\nSome parameters can be reconfigured from the Pangolin GUI at runtime. Feel free to add more.\n\n\n#### 3.4 Accessing Data.\nThe easiest way to access the Data (poses, pointclouds, etc.) computed by DSO (in real-time)\nis to create your own `Output3DWrapper`, and add it to the system, i.e., to `FullSystem.outputWrapper`.\nThe respective member functions will be called on various occations (e.g., when a new KF is created, \nwhen a new frame is tracked, etc.), exposing the relevant data.\n\nSee `IOWrapper/Output3DWrapper.h` for a description of the different callbacks available,\nand some basic notes on where to find which data in the used classes.\nSee `IOWrapper/OutputWrapper/SampleOutputWrapper.h` for an example implementation, which just prints\nsome example data to the commandline (use the options `sampleoutput=1 quiet=1` to see the result).\n\nNote that these callbacks block the respective DSO thread, thus expensive computations should not\nbe performed in the callbacks, a better practice is to just copy over / publish / output the data you need.\n\nPer default, `dso_dataset` writes all keyframe poses to a file `result.txt` at the end of a sequence,\nusing the TUM RGB-D / TUM monoVO format ([timestamp x y z qx qy qz qw] of the cameraToWorld transformation).\n\n\n\n#### 3.5 Notes\n- the initializer is very slow, and does not work very reliably. Maybe replace by your own way to get an initialization.\n- see [https://github.com/JakobEngel/dso_ros](https://github.com/JakobEngel/dso_ros) for a minimal example project on how to use the library with your own input / output procedures.\n- see `settings.cpp` for a LOT of settings parameters. Most of which you shouldn't touch.\n- `setGlobalCalib(...)` needs to be called once before anything is initialized, and globally sets the camera intrinsics and video resolution for convenience. probably not the most portable way of doing this though.\n\n\n\n\n### 4 General Notes for Good Results\n\n#### Accurate Geometric Calibration\n- Please have a look at Chapter 4.3 from the DSO paper, in particular Figure 20 (Geometric Noise). Direct approaches suffer a LOT from bad geometric calibrations: Geometric distortions of 1.5 pixel already reduce the accuracy by factor 10.\n\n- **Do not use a rolling shutter camera**, the geometric distortions from a rolling shutter camera are huge. Even for high frame-rates (over 60fps).\n\n- Note that the reprojection RMSE reported by most calibration tools is the reprojection RMSE on the \"training data\", i.e., overfitted to the the images you used for calibration. If it is low, that does not imply that your calibration is good, you may just have used insufficient images.\n\n- try different camera / distortion models, not all lenses can be modelled by all models.\n\n\n#### Photometric Calibration\nUse a photometric calibration (e.g. using [https://github.com/tum-vision/mono_dataset_code](https://github.com/tum-vision/mono_dataset_code) ).\n\n#### Translation vs. Rotation\nDSO cannot do magic: if you rotate the camera too much without translation, it will fail. Since it is a pure visual odometry, it cannot recover by re-localizing, or track through strong rotations by using previously triangulated geometry.... everything that leaves the field of view is marginalized immediately.\n\n\n#### Computation Speed\nIf your computer is slow, try to use \"fast\" settings. Or run DSO on a dataset, without enforcing real-time.\n\n\n#### Initialization\nThe current initializer is not very good... it is very slow and occasionally fails. \nMake sure, the initial camera motion is slow and \"nice\" (i.e., a lot of translation and \nlittle rotation) during initialization.\nPossibly replace by your own initializer.\n\n\n### 5 License\nDSO was developed at the Technical University of Munich and Intel.\nThe open-source version is licensed under the GNU General Public License\nVersion 3 (GPLv3).\nFor commercial purposes, we also offer a professional version, see\n[http://vision.in.tum.de/dso](http://vision.in.tum.de/dso) for\ndetails.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJakobEngel%2Fdso","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJakobEngel%2Fdso","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJakobEngel%2Fdso/lists"}