https://github.com/rmurai0610/mast3r-slam
[CVPR 2025] MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors
https://github.com/rmurai0610/mast3r-slam
computer-vision cvpr2025 robotics slam
Last synced: 6 days ago
JSON representation
[CVPR 2025] MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors
- Host: GitHub
- URL: https://github.com/rmurai0610/mast3r-slam
- Owner: rmurai0610
- License: other
- Created: 2025-02-24T16:18:37.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2025-03-11T12:23:34.000Z (about 1 month ago)
- Last Synced: 2025-04-07T14:00:42.509Z (13 days ago)
- Topics: computer-vision, cvpr2025, robotics, slam
- Language: Python
- Homepage: https://edexheim.github.io/mast3r-slam/
- Size: 46.3 MB
- Stars: 1,780
- Watchers: 24
- Forks: 138
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
[comment]: <> (# MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors)
MASt3R-SLAM: Real-Time Dense SLAM with 3D Reconstruction Priors
Riku Murai*
·
Eric Dexheimer*
·
Andrew J. Davison
(* Equal Contribution)
[comment]: <> (
PAPER
)
Paper | Video | Project Page
# Getting Started
## Installation
```
conda create -n mast3r-slam python=3.11
conda activate mast3r-slam
```
Check the system's CUDA version with nvcc
```
nvcc --version
```
Install pytorch with **matching** CUDA version following:
```
# CUDA 11.8
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=11.8 -c pytorch -c nvidia
# CUDA 12.1
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.1 -c pytorch -c nvidia
# CUDA 12.4
conda install pytorch==2.5.1 torchvision==0.20.1 torchaudio==2.5.1 pytorch-cuda=12.4 -c pytorch -c nvidia
```
Clone the repo and install the dependencies.
```
git clone https://github.com/rmurai0610/MASt3R-SLAM.git --recursive
cd MASt3R-SLAM/
# if you've clone the repo without --recursive run
# git submodule update --init --recursive
pip install -e thirdparty/mast3r
pip install -e thirdparty/in3d
pip install --no-build-isolation -e .
# Optionally install torchcodec for faster mp4 loading
pip install torchcodec==0.1
```
Setup the checkpoints for MASt3R and retrieval. The license for the checkpoints and more information on the datasets used is written [here](https://github.com/naver/mast3r/blob/mast3r_sfm/CHECKPOINTS_NOTICE).
```
mkdir -p checkpoints/
wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric.pth -P checkpoints/
wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_trainingfree.pth -P checkpoints/
wget https://download.europe.naverlabs.com/ComputerVision/MASt3R/MASt3R_ViTLarge_BaseDecoder_512_catmlpdpt_metric_retrieval_codebook.pkl -P checkpoints/
```
## WSL Users
We have primarily tested on Ubuntu. If you are using WSL, please checkout to the windows branch and follow the above installation.
```
git checkout windows
```
This disables multiprocessing which causes an issue with shared memory as discussed [here](https://github.com/rmurai0610/MASt3R-SLAM/issues/21).
## Examples
```
bash ./scripts/download_tum.sh
python main.py --dataset datasets/tum/rgbd_dataset_freiburg1_room/ --config config/calib.yaml
```
## Live Demo
Connect a realsense camera to the PC and run
```
python main.py --dataset realsense --config config/base.yaml
```
## Running on a video
Our system can process either MP4 videos or folders containing RGB images.
```
python main.py --dataset .mp4 --config config/base.yaml
python main.py --dataset --config config/base.yaml
```
If the calibration parameters are known, you can specify them in intrinsics.yaml
```
python main.py --dataset .mp4 --config config/base.yaml --calib config/intrinsics.yaml
python main.py --dataset --config config/base.yaml --calib config/intrinsics.yaml
```
## Downloading Dataset
### TUM-RGBD Dataset
```
bash ./scripts/download_tum.sh
```
### 7-Scenes Dataset
```
bash ./scripts/download_7_scenes.sh
```
### EuRoC Dataset
```
bash ./scripts/download_euroc.sh
```
### ETH3D SLAM Dataset
```
bash ./scripts/download_eth3d.sh
```
## Running Evaluations
All evaluation script will run our system in a single-threaded, headless mode.
We can run evaluations with/without calibration:
### TUM-RGBD Dataset
```
bash ./scripts/eval_tum.sh
bash ./scripts/eval_tum.sh --no-calib
```
### 7-Scenes Dataset
```
bash ./scripts/eval_7_scenes.sh
bash ./scripts/eval_7_scenes.sh --no-calib
```
### EuRoC Dataset
```
bash ./scripts/eval_euroc.sh
bash ./scripts/eval_euroc.sh --no-calib
```
### ETH3D SLAM Dataset
```
bash ./scripts/eval_eth3d.sh
```
## Reproducibility
There might be minor differences between the released version and the results in the paper after developing this multi-processing version.
We run all our experiments on an RTX 4090, and the performance may differ when running with a different GPU.
## Acknowledgement
We sincerely thank the developers and contributors of the many open-source projects that our code is built upon.
- [MASt3R](https://github.com/naver/mast3r)
- [MASt3R-SfM](https://github.com/naver/mast3r/tree/mast3r_sfm)
- [DROID-SLAM](https://github.com/princeton-vl/DROID-SLAM)
- [ModernGL](https://github.com/moderngl/moderngl)
# Citation
If you found this code/work to be useful in your own research, please considering citing the following:
```bibtex
@article{murai2024_mast3rslam,
title={{MASt3R-SLAM}: Real-Time Dense {SLAM} with {3D} Reconstruction Priors},
author={Murai, Riku and Dexheimer, Eric and Davison, Andrew J.},
journal={arXiv preprint},
year={2024},
}
```