https://github.com/andreaconti/ramdepth

Official code for Range-Agnostic Multi-View Depth Estimation with Keyframe Selection (3DV 2024)
https://github.com/andreaconti/ramdepth

Last synced: 18 days ago
JSON representation

Official code for Range-Agnostic Multi-View Depth Estimation with Keyframe Selection (3DV 2024)

• Host: GitHub
• URL: https://github.com/andreaconti/ramdepth
• Owner: andreaconti
• Created: 2023-11-29T13:44:33.000Z (12 months ago)
• Default Branch: main
• Last Pushed: 2024-02-19T14:53:22.000Z (9 months ago)
• Last Synced: 2024-10-24T15:56:43.371Z (27 days ago)
• Language: Python
• Size: 2.18 MB
• Stars: 43
• Watchers: 2
• Forks: 2
• Open Issues: 1
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        


Range-Agnostic Multi-View Depth Estimation With Keyframe Selection

aka



    

    RAMDepth

    









    Andrea Conti

    ·

    Matteo Poggi

    ·

    Valerio Cambareri

    ·

    Stefano Mattoccia





    [Arxiv]

    [Project Page]



![](https://github.com/andreaconti/ramdepth/blob/main/media/teaser.png)

Multi-View 3D reconstruction techniques process a set of source views and a reference view to yield an estimated depth map for the latter. Unluckily, state-of-the-art frameworks

1. require to know _a priori_ the depth range of the scene, in order to sample a set of _depth hypotheses_ and build a meaningful _cost volume_.

2. do not take into account the _keyframes selection_.

In this paper, we propose a novel framework **free from prior knowledge of the scene depth range** and capable of **distinguishing the most meaningful source frames**. The proposed method unlocks the capability to apply multi-view depth estimation to a wider range of scenarios like large-scale outdoor environments, top-view buildings and large-scale outdoor environments.

Our method relies on an **iterative approach**: starting from a zero-initialized depth map we extract geometrical correlation cues and update the prediction. At each iteration we feed also information extracted from the reference view only (the one on which we desire to compute depth). 

Moreover, at each iteration we use a different source view to exploit multi-view information in a round-robin fashion. For more details please refer to the [paper](https://arxiv.org/pdf/2401.14401.pdf).

## Citation

```bibtex

@InProceedings{Conti_2024_3DV,

    author    = {Conti, Andrea and Poggi, Matteo and Cambareri, Valerio and Mattoccia, Stefano},

    title     = {Range-Agnostic Multi-View Depth Estimation With Keyframe Selection},

    booktitle = {International Conference on 3D Vision},

    month     = {March},

    year      = {2024},

}

```

## Evaluation Code

In this repo we provide __evaluation__ code for our paper, it allows to load the pre-trained models on Blended and TartanAir and test them. Please note that we do not provide the source code of our models but only compiled binaries to perform inference.

Dependencies can be installed with `conda` or `mamba` as follows:

```bash

$ # first of all clone the repo and build the conda environment

$ git clone https://github.com/andreaconti/ramdepth.git

$ cd ramdepth

$ conda env create -f environment.yml  # use mamba if conda is too slow

$ conda activate ramdepth

$ # then, download and install the wheel containing the pretrained models, available for linux, windows and macos

$ pip install https://github.com/andreaconti/ramdepth/releases/download/wheels%2Fv0.1.0/ramdepth-0.1.0-cp310-cp310-linux_x86_64.whl --no-deps

```

Then you can run the [evaluate.ipynb](https://github.com/andreaconti/ramdepth/blob/main/evaluate.ipynb) to select the dataset and pre-trained model you want to test. Results may be slightly different with respect to the results in the main paper due to small differences in dataloaders and framework due to the packaging process.