{"id":18009138,"url":"https://github.com/kwea123/mvsnet_pl","last_synced_at":"2025-03-26T13:31:48.715Z","repository":{"id":109613410,"uuid":"237612406","full_name":"kwea123/MVSNet_pl","owner":"kwea123","description":"MVSNet: Depth Inference for Unstructured Multi-view Stereo using pytorch-lightning","archived":false,"fork":false,"pushed_at":"2021-10-12T23:12:08.000Z","size":1187,"stargazers_count":67,"open_issues_count":5,"forks_count":9,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-21T22:22:03.484Z","etag":null,"topics":["3d-reconstruction","depth-prediction","multi-view-stereo","mvsnet","pytorch","pytorch-lightning"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kwea123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-02-01T12:35:06.000Z","updated_at":"2025-03-18T13:00:51.000Z","dependencies_parsed_at":"2023-06-19T20:45:18.929Z","dependency_job_id":null,"html_url":"https://github.com/kwea123/MVSNet_pl","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwea123%2FMVSNet_pl","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwea123%2FMVSNet_pl/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwea123%2FMVSNet_pl/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kwea123%2FMVSNet_pl/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kwea123","download_url":"https://codeload.github.com/kwea123/MVSNet_pl/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245662929,"owners_count":20652100,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","depth-prediction","multi-view-stereo","mvsnet","pytorch","pytorch-lightning"],"created_at":"2024-10-30T02:08:48.976Z","updated_at":"2025-03-26T13:31:48.709Z","avatar_url":"https://github.com/kwea123.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# MVSNet_pl\nUnofficial implementation of [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://arxiv.org/pdf/1804.02505.pdf) using [pytorch-lightning](https://github.com/PyTorchLightning/pytorch-lightning)\n\n**An improved version of MVSNet: [CasMVSNet](https://github.com/kwea123/CasMVSNet_pl) is available!**\n\n# References \u0026 Credits\nOfficial implementation: [MVSNet](https://github.com/YoYo000/MVSNet)\n\nA pytorch implementation: [MVSNet_pytorch](https://github.com/xy-guo/MVSNet_pytorch).\nThis code is heavily borrowed from his implementation. Thank [xy-guo](https://github.com/xy-guo) for the effortful contribution!\nTwo main difference w.r.t. his repo:\n1. `homo_warping` function is rewritten in a more concise and slightly faster way.\n2. Use [Inplace-ABN](https://github.com/mapillary/inplace_abn) in the model to reduce GPU memory consumption (about 10%).\n\n# Installation\n\n## Hardware\n\n* OS: Ubuntu 16.04 or 18.04\n* NVIDIA GPU with **CUDA\u003e=10.0** (tested with 1 RTX 2080Ti)\n\n## Software\n\n* Python\u003e=3.6.1 (installation via [anaconda](https://www.anaconda.com/distribution/) is recommended, use `conda create -n mvsnet_pl python=3.6` to create a conda environment and activate it by `conda activate mvsnet_pl`)\n* Python libraries\n    * Install core requirements by `pip install -r requirements.txt`\n    * Install [Inplace-ABN](https://github.com/mapillary/inplace_abn) by `pip install git+https://github.com/mapillary/inplace_abn.git@v1.0.11`\n\n# Data download\n\nDownload the preprocessed [DTU training data](https://drive.google.com/file/d/1eDjh-_bxKKnEuz5h-HXS7EDJn59clx6V/view) from original [MVSNet repo](https://github.com/YoYo000/MVSNet) and unzip. For the description of how the data is created, please refer to the original paper.\n\n# Training\nRun\n```\npython train.py \\\n  --root_dir $DTU_DIR \\\n  --num_epochs 6 --batch_size 1 \\\n  --n_depths 192 --interval_scale 1.06 \\\n  --optimizer adam --lr 1e-3 --lr_scheduler cosine\n```\nNote that the model consumes huge GPU memory, so the batch size is generally small. For reference, the above command requires 5901MB of GPU memory.\n\n**IMPORTANT** : the combination of `--n_depths` and `--interval_scale` is important: you need to make sure `2.5 x n_depths x interval_scale` is roughly equal to `510`. The reason is that the actual depth ranges from 425 to 935mm, which is 510mm wide. Therefore, you need to make sure all the depth can be covered by the depth planes you set. Some common combinations are: `--n_depths 256 --interval_scale 0.8`, `--n_depths 192 --interval_scale 1.06` and `--n_depths 128 --interval_scale 1.6`.\n\nSee [opt.py](opt.py) for all configurations.\n\n## Example training log\n\n![log1](assets/log1.png)\n![log2](assets/log2.png)\n![log3](assets/log3.png)\n\n## Metrics\nThe metrics are collected on the DTU val set.\n\n|       | abs_err | 1mm acc | 2mm acc | 4mm acc |\n| :---: |   :---: |  :---: | :---: | :---: |\n| Paper | 7.25mm*     | N/A    | N/A | N/A |\n| This repo | 6.374mm | 54.43% | 74.23% | 85.8% |\n\n*From [P-MVSNet](http://openaccess.thecvf.com/content_ICCV_2019/papers/Luo_P-MVSNet_Learning_Patch-Wise_Matching_Confidence_Aggregation_for_Multi-View_Stereo_ICCV_2019_paper.pdf) Table 2.\n\n## Some observations on training\n1. Larger `n_depths` theoretically gives better results, but requires larger GPU memory, so basically the `batch_size` can just be `1` or `2`. However at the meanwhile, larger `batch_size` is also indispensable. To get a good balance between `n_depths` and `batch_size`, I found that `n_depths 128 batch_size 2` performs **better** than `n_depths 192 batch_size 1` given a fixed GPU memory of 11GB. Of course to get even better results, you'll definitely want to scale up the `batch_size` by using more GPUs, and that is easy under pytorch-lightning's framework!\n2. Longer training epochs produces better results. The pretrained model I provide is trained for 16 epochs, and it performs better than the model trained for only 6 epochs as the paper did.\n3. Image color augmentation worsen the result, and normalization seems to have little to no effect. However, [BlendedMVS](https://arxiv.org/pdf/1911.10127.pdf) claims otherwise, they obtain better results using augmentation.\n\n# Testing\n\n1. Download pretrained model from [release](https://github.com/kwea123/MVSNet_pl/releases).\n2. Use [test.ipynb](test.ipynb) for a simple depth inference for an image.\n\nThe repo is only for training purpose for now. Please refer to the other repositories mentioned at the beginning if you want to evaluate the model.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwea123%2Fmvsnet_pl","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkwea123%2Fmvsnet_pl","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkwea123%2Fmvsnet_pl/lists"}