Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/mpizenberg/visual-odometry-rs

Visual Odometry in Rust (vors)
https://github.com/mpizenberg/visual-odometry-rs

Last synced: about 2 months ago
JSON representation

Visual Odometry in Rust (vors)

Awesome Lists containing this project

README 

        
# Visual Odometry in Rust (vors)



This repository provides both a library ("crate" as we say in Rust) named visual-odometry-rs,

(shortened vors) and a binary program named `vors_track`,

for camera tracking ("visual odometry").



The program works on datasets following the [TUM RGB-D dataset format][tum-rgbd].

It is roughly a hundred lines of code (see `src/bin/vors_track.rs`),

built upon the visual-odometry-rs crate also provided here.



Once you have cloned this repository,

you can run the binary program `vors_track` with cargo directly as follows:



```sh

cargo run --release --bin vors_track -- fr1 /path/to/some/freiburg1/dataset/associations.txt

```



Have a look at [mpizenberg/rgbd-tracking-evaluation][rgbd-track-eval]

for more info about the dataset requirements to run the binary program `vors_track`.



The library is organized around four base namespaces:



- `core::` Core modules for computing gradients, candidate points, camera tracking etc.

- `dataset::` Helper modules for handling specific datasets.

  Currently only provides a module for TUM RGB-D compatible datasets.

- `math::` Basic math modules for functionalities not already provided by [nalgebra][nalgebra],

  like Lie algebra for so3, se3, and an iterative optimizer trait.

- `misc::` Helper modules for interoperability, visualization, and other things that did

  not fit elsewhere yet.



[tum-rgbd]: https://vision.in.tum.de/data/datasets/rgbd-dataset

[rgbd-track-eval]: https://github.com/mpizenberg/rgbd-tracking-evaluation

[nalgebra]: https://www.nalgebra.org/



## Library Usage Examples



Self contained examples for usage of the API are available in the `examples/` directory.

A readme is also present there for more detailed explanations on these examples.



## Functionalities and Vision



Currently, vors provides a **visual odometry framework for working on direct RGB-D camera tracking**.

Setting all this from the ground up took a lot of time and effort,

but I think it is mature enough to be shared as is now.

Beware, however, that the API is evolving a lot.

My hope is that in the near future, we can improve the reach of this project

by working both on research extensions, and platform availability.



Example research extensions:



- Using disparity search for depth initialization to be compatible with RGB (no depth) camera.

- Adding a photometric term to the residual to account for automatic exposure variations.

- Adding automatic photometric and/or geometric camera calibration.

- Building a sliding window of keyframes optimization as in [DSO][dso] to reduce drift.

- Intregrating loop closure and pose graph optimization for having a robust vSLAM system.

- Fusion with IMU for improved tracking and reducing scale drift.

- Modelization of rolling shutter (in most cameras) into the optimization problem.

- Extension to stereo cameras.

- Extension to omnidirectional cameras.



Example platform extensions:



- Making a C FFI to be able to run on systems with C drivers (kinect, realsense, ...).

- Porting to the web with WebAssembly.

- Porting to ARM for running in embedded systems and phones.



## Background Story



Initially, this repository served as a personal experimental sandbox for computer vision in Rust.

See for example my original questions on the rust [discourse][discourse] and [reddit channel][reddit].

Turns out I struggled a bit at first but then really liked the Rust way, compared to C++.



As the name suggests, the focus is now on [visual odometry][vo],

specifically on the recent research field of direct visual odometry.

A reasonable introduction is available in those [lecture slides][vo-slides]

by Waterloo Autonomous Vehicles lab.



In particular, this project initially aimed at improving on the work of [DSO][dso]

by J. Engel et. al. but with all the advantages of using the [Rust programming language][rust],

including:



- Performance without sacrificing code readability

- No memory error, and much higher code safety and reliability

- Friendly tooling ecosystem, no dependency issues, basically one-liner compilation and run

- Best tooling for porting to the web with WebAssembly

- Growing and mindful resources for porting to embedded systems

- Wonderful community



[discourse]: https://users.rust-lang.org/t/computer-vision-in-rust/16198

[reddit]: https://www.reddit.com/r/rust/comments/84s5zo/computer_vision_in_rust/

[vo]: https://en.wikipedia.org/wiki/Visual_odometry

[vo-slides]: http://wavelab.uwaterloo.ca/slam/2017-SLAM/Lecture14-Direct_visual_inertial_odometry_and_SLAM/slides.pdf

[dso]: https://github.com/JakobEngel/dso

[rust]: https://www.rust-lang.org/



## License (MPL-2.0)



This Source Code Form is subject to the terms of the Mozilla Public

License, v. 2.0. If a copy of the MPL was not distributed with this

file, You can obtain one at http://mozilla.org/MPL/2.0/.



## Contributions



All forms of contribution are welcomed, **preferably first as github issues**.



- Questions

- Documentation

- Tests

- Benchmarks

- Features



In case of contribution to source code,

it needs to use [rustfmt][rustfmt] and [clippy][clippy].

To run clippy:



```

touch src/lib.rs; cargo clippy --release --all-targets --all-features

```



[rustfmt]: https://github.com/rust-lang/rustfmt

[clippy]: https://github.com/rust-lang/rust-clippy