{"id":19529265,"url":"https://github.com/isl-org/vi-depth","last_synced_at":"2026-03-01T18:32:08.260Z","repository":{"id":145896042,"uuid":"610373222","full_name":"isl-org/VI-Depth","owner":"isl-org","description":"Code for Monocular Visual-Inertial Depth Estimation (ICRA 2023)","archived":false,"fork":false,"pushed_at":"2024-08-26T12:33:14.000Z","size":2446,"stargazers_count":176,"open_issues_count":6,"forks_count":15,"subscribers_count":12,"default_branch":"main","last_synced_at":"2025-04-13T08:26:22.857Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/isl-org.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-06T16:35:57.000Z","updated_at":"2025-04-09T19:07:28.000Z","dependencies_parsed_at":"2024-08-26T14:40:21.720Z","dependency_job_id":"898e293a-e6e7-4551-8699-0a79ceb465de","html_url":"https://github.com/isl-org/VI-Depth","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/isl-org/VI-Depth","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isl-org%2FVI-Depth","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isl-org%2FVI-Depth/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isl-org%2FVI-Depth/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isl-org%2FVI-Depth/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/isl-org","download_url":"https://codeload.github.com/isl-org/VI-Depth/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isl-org%2FVI-Depth/sbom","scorecard":{"id":495671,"data":{"date":"2025-08-11","repo":{"name":"github.com/isl-org/VI-Depth","commit":"ce9255ff5dbf16c64d1062d76337a16b75cc8b9a"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":3.8,"checks":[{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Code-Review","score":0,"reason":"Found 0/5 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"SAST","score":0,"reason":"no SAST tool detected","details":["Warn: no pull requests merged into dev branch"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: MIT License: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v1 not signed: https://api.github.com/repos/isl-org/VI-Depth/releases/96497156","Warn: release artifact v1 does not have provenance: https://api.github.com/repos/isl-org/VI-Depth/releases/96497156"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":3,"reason":"branch protection is not maximal on development and all release branches","details":["Info: 'allow deletion' disabled on branch 'main'","Info: 'force pushes' disabled on branch 'main'","Warn: 'branch protection settings apply to administrators' is disabled on branch 'main'","Warn: could not determine whether codeowners review is allowed","Warn: no status checks found to merge onto branch 'main'","Warn: PRs are not required to make changes on branch 'main'; or we don't have data to detect it.If you think it might be the latter, make sure to run Scorecard with a PAT or use Repo Rules (that are always public) instead of Branch Protection settings"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: github.com/isl-org/.github/SECURITY.md:1","Info: Found linked content: github.com/isl-org/.github/SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: github.com/isl-org/.github/SECURITY.md:1","Info: Found text in security policy: github.com/isl-org/.github/SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}}]},"last_synced_at":"2025-08-19T20:22:40.461Z","repository_id":145896042,"created_at":"2025-08-19T20:22:40.461Z","updated_at":"2025-08-19T20:22:40.461Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29978552,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-01T16:35:47.903Z","status":"ssl_error","status_checked_at":"2026-03-01T16:35:44.899Z","response_time":124,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-11T01:23:11.867Z","updated_at":"2026-03-01T18:32:08.251Z","avatar_url":"https://github.com/isl-org.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PROJECT NOT UNDER ACTIVE MANAGEMENT #  \nThis project will no longer be maintained by Intel.  \nIntel will not provide or guarantee development of or support for this project, including but not limited to, maintenance, bug fixes, new releases or updates.  \nPatches to this project are no longer accepted by Intel.  \n If you have an ongoing need to use this project, are interested in independently developing it, or would like to maintain patches for the community, please create your own fork of the project.  \n  \n# Monocular Visual-Inertial Depth Estimation\n\nThis repository contains code and models for our paper:\n\n\u003e [Monocular Visual-Inertial Depth Estimation](https://arxiv.org/abs/2303.12134)  \n\u003e Diana Wofk, René Ranftl, Matthias Müller, Vladlen Koltun\n\nFor a quick overview of the work you can watch the [short talk](https://youtu.be/Ja4Nic3YYCg) and [teaser](https://youtu.be/IMwiKwSpshQ) on YouTube.\n\n## Introduction\n\n![Methodology Diagram](figures/methodology_diagram.png)\n\nWe present a visual-inertial depth estimation pipeline that integrates monocular depth estimation and visual-inertial odometry to produce dense depth estimates with metric scale. Our approach consists of three stages: (1) input processing, where RGB and IMU data feed into monocular depth estimation alongside visual-inertial odometry, (2) global scale and shift alignment, where monocular depth estimates are fitted to sparse depth from VIO in a least-squares manner, and (3) learning-based dense scale alignment, where globally-aligned depth is locally realigned using a dense scale map regressed by the ScaleMapLearner (SML). The images at the bottom in the diagram above illustrate a VOID sample being processed through our pipeline; from left to right: the input RGB, ground truth depth, sparse depth from VIO, globally-aligned depth, scale map scaffolding, dense scale map regressed by SML, final depth output.\n\n![Teaser Figure](figures/teaser_figure.png)\n\n## Setup\n\n1) Setup dependencies:\n\n    ```shell\n    conda env create -f environment.yaml\n    conda activate vi-depth\n    ```\n\n2) Pick one or more ScaleMapLearner (SML) models and download the corresponding weights to the `weights` folder.\n\n    | Depth Predictor   |  SML on VOID 150  |  SML on VOID 500  | SML on VOID 1500 |\n    | :---              |       :----:      |       :----:      |      :----:      |\n    | DPT-BEiT-Large    | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_beit_large_512.nsamples.150.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_beit_large_512.nsamples.500.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_beit_large_512.nsamples.1500.ckpt) |\n    | DPT-SwinV2-Large  | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_swin2_large_384.nsamples.150.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_swin2_large_384.nsamples.500.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_swin2_large_384.nsamples.1500.ckpt) |\n    | DPT-Large         | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_large.nsamples.150.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_large.nsamples.500.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_large.nsamples.1500.ckpt) |\n    | DPT-Hybrid        | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_hybrid.nsamples.150.ckpt)* | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_hybrid.nsamples.500.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_hybrid.nsamples.1500.ckpt) |\n    | DPT-SwinV2-Tiny   | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_swin2_tiny_256.nsamples.150.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_swin2_tiny_256.nsamples.500.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_swin2_tiny_256.nsamples.1500.ckpt) |\n    | DPT-LeViT         | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_levit_224.nsamples.150.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_levit_224.nsamples.500.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_levit_224.nsamples.1500.ckpt) |\n    | MiDaS-small       | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.midas_small.nsamples.150.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.midas_small.nsamples.500.ckpt) | [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.midas_small.nsamples.1500.ckpt) |\n\n    *Also available with pretraining on TartanAir: [model](https://github.com/isl-org/VI-Depth/releases/download/v1/sml_model.dpredictor.dpt_hybrid.nsamples.150.pretrained.ckpt)\n\n## Inference\n\n1) Place inputs into the `input` folder. An input image and corresponding sparse metric depth map are expected:\n\n    ```bash\n    input\n    ├── image                   # RGB image\n    │   ├── \u003ctimestamp\u003e.png\n    │   └── ...\n    └── sparse_depth            # sparse metric depth map\n        ├── \u003ctimestamp\u003e.png     # as 16b PNG\n        └── ...\n    ```\n\n    The `load_sparse_depth` function in `run.py` may need to be modified depending on the format in which sparse depth is stored. By default, the depth storage method [used in the VOID dataset](https://github.com/alexklwong/void-dataset/blob/master/src/data_utils.py) is assumed.\n\n2) Run the `run.py` script as follows:\n\n    ```bash\n    DEPTH_PREDICTOR=\"dpt_beit_large_512\"\n    NSAMPLES=150\n    SML_MODEL_PATH=\"weights/sml_model.dpredictor.${DEPTH_PREDICTOR}.nsamples.${NSAMPLES}.ckpt\"\n\n    python run.py -dp $DEPTH_PREDICTOR -ns $NSAMPLES -sm $SML_MODEL_PATH --save-output\n    ```\n\n3) The `--save-output` flag enables saving outputs to the `output` folder. By default, the following outputs will be saved per sample:\n\n    ```bash\n    output\n    ├── ga_depth                # metric depth map after global alignment\n    │   ├── \u003ctimestamp\u003e.pfm     # as PFM\n    │   ├── \u003ctimestamp\u003e.png     # as 16b PNG\n    │   └── ...\n    └── sml_depth               # metric depth map output by SML\n        ├── \u003ctimestamp\u003e.pfm     # as PFM\n        ├── \u003ctimestamp\u003e.png     # as 16b PNG\n        └── ...\n    ```\n\n## Evaluation\n\nModels provided in this repo were trained on the VOID dataset. \n1) Download the VOID dataset following [the instructions in the VOID dataset repo](https://github.com/alexklwong/void-dataset#downloading-void).\n2) To evaluate on VOID test sets, run the `evaluate.py` script as follows:\n\n    ```bash\n    DATASET_PATH=\"/path/to/void_release/\"\n\n    DEPTH_PREDICTOR=\"dpt_beit_large_512\"\n    NSAMPLES=150\n    SML_MODEL_PATH=\"weights/sml_model.dpredictor.${DEPTH_PREDICTOR}.nsamples.${NSAMPLES}.ckpt\"\n\n    python evaluate.py -ds $DATASET_PATH -dp $DEPTH_PREDICTOR -ns $NSAMPLES -sm $SML_MODEL_PATH\n    ```\n\n    Results for the example shown above:\n\n    ```\n    Averaging metrics for globally-aligned depth over 800 samples\n    Averaging metrics for SML-aligned depth over 800 samples\n    +---------+----------+----------+\n    |  metric | GA Only  |  GA+SML  |\n    +---------+----------+----------+\n    |   RMSE  |  191.36  |  142.85  |\n    |   MAE   |  115.84  |   76.95  |\n    |  AbsRel |    0.069 |    0.046 |\n    |  iRMSE  |   72.70  |   57.13  |\n    |   iMAE  |   49.32  |   34.25  |\n    | iAbsRel |    0.071 |    0.048 |\n    +---------+----------+----------+\n    ```\n    \n    To evaluate on VOID test sets at different densities (void_150, void_500, void_1500), change the `NSAMPLES` argument above accordingly.\n\n## Citation\n\nIf you reference our work, please consider citing the following:\n\n```bib\n@inproceedings{wofk2023videpth,\n    author      = {{Wofk, Diana and Ranftl, Ren\\'{e} and M{\\\"u}ller, Matthias and Koltun, Vladlen}},\n    title       = {{Monocular Visual-Inertial Depth Estimation}},\n    booktitle   = {{IEEE International Conference on Robotics and Automation (ICRA)}},\n    year        = {{2023}}\n}\n```\n\n## Acknowledgements\n\nOur work builds on and uses code from [MiDaS](https://github.com/isl-org/MiDaS), [timm](https://github.com/rwightman/pytorch-image-models), and [PyTorch Lightning](https://lightning.ai/docs/pytorch/stable/). We'd like to thank the authors for making these libraries and frameworks available.\n\n_Last revisited: August 2024_\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fisl-org%2Fvi-depth","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fisl-org%2Fvi-depth","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fisl-org%2Fvi-depth/lists"}