{"id":13473863,"url":"https://autonomousvision.github.io/sdfstudio/","last_synced_at":"2025-03-26T19:35:01.890Z","repository":{"id":64963374,"uuid":"578702515","full_name":"autonomousvision/sdfstudio","owner":"autonomousvision","description":"A Unified Framework for Surface Reconstruction","archived":false,"fork":false,"pushed_at":"2024-07-11T02:15:36.000Z","size":59352,"stargazers_count":2026,"open_issues_count":163,"forks_count":192,"subscribers_count":32,"default_branch":"master","last_synced_at":"2025-03-20T22:15:30.572Z","etag":null,"topics":["3d-reconstruction","implicit-neural-representation","multi-view-reconstruction","nerf","pytorch","sdf","surface-reconstruction"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/autonomousvision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-12-15T17:20:07.000Z","updated_at":"2025-03-20T08:53:04.000Z","dependencies_parsed_at":"2024-12-05T07:12:33.507Z","dependency_job_id":null,"html_url":"https://github.com/autonomousvision/sdfstudio","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fsdfstudio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fsdfstudio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fsdfstudio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/autonomousvision%2Fsdfstudio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/autonomousvision","download_url":"https://codeload.github.com/autonomousvision/sdfstudio/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245723010,"owners_count":20661868,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","implicit-neural-representation","multi-view-reconstruction","nerf","pytorch","sdf","surface-reconstruction"],"created_at":"2024-07-31T16:01:07.549Z","updated_at":"2025-03-26T19:34:56.877Z","avatar_url":"https://github.com/autonomousvision.png","language":"Python","funding_links":[],"categories":["Uncategorized","Projects"],"sub_categories":["Uncategorized"],"readme":"\u003cp align=\"center\"\u003e\n    \u003cimg alt=\"nerfstudio\" src=\"media/sdf_studio_4.png\" width=\"300\"\u003e\n    \u003ch1 align=\"center\"\u003eA Unified Framework for Surface Reconstruction\u003c/h1\u003e\n    \u003ch3 align=\"center\"\u003e\u003ca href=\"https://autonomousvision.github.io/sdfstudio/\"\u003eProject Page\u003c/a\u003e | \u003ca href=\"docs/sdfstudio-methods.md\"\u003eDocumentation\u003c/a\u003e | \u003ca href=\"docs/sdfstudio-data.md\"\u003eDatasets\u003c/a\u003e | \u003ca href=\"docs/sdfstudio-examples.md\"\u003eExamples\u003c/a\u003e \u003c/h3\u003e\n    \u003cimg src=\"media/overview.png\" center width=\"95%\"/\u003e\n\u003c/p\u003e\n\n# About\n\nSDFStudio is a unified and modular framework for neural implicit surface reconstruction, built on top of the awesome nerfstudio project. We provide a unified implementation of three major implicit surface reconstruction methods: UniSurf, VolSDF, and NeuS. SDFStudio also supports various scene representions, such as MLPs, Tri-plane, and Multi-res. feature grids, and multiple point sampling strategies such as surface-guided sampling as in UniSurf, and Voxel-surface guided sampling from NeuralReconW. It further integrates recent advances in the area such as the utillization of monocular cues (MonoSDF), geometry regularization (UniSurf) and multi-view consistency (Geo-NeuS). Thanks to the unified and modular implementation, SDFStudio makes it easy to transfer ideas from one method to another. For example, Mono-NeuS applies the idea from MonoSDF to NeuS, and Geo-VolSDF applies the idea from Geo-NeuS to VolSDF.\n\n# Updates\n\n**2023.06.16**: Add `bakedangelo` which combines `BakedSDF` with numerical gridents and progressive training of `Neuralangelo`.\n\n**2023.06.16**: Add `neus-facto-angelo` which combines `neus-facto` with numerical gridents and progressive training of `Neuralangelo`.\n\n**2023.06.16**: Support [Neuralangelo](https://research.nvidia.com/labs/dir/neuralangelo/).\n\n**2023.03.12**: Support [BakedSDF](https://bakedsdf.github.io/).\n\n**2022.12.28**: Support [Neural RGB-D Surface Reconstruction](https://dazinovic.github.io/neural-rgbd-surface-reconstruction/).\n\n# Quickstart\n\n## 1. Installation: Setup the environment\n\n### Prerequisites\n\nCUDA must be installed on the system. This library has been tested with version 11.3. You can find more information about installing CUDA [here](https://docs.nvidia.com/cuda/cuda-quick-start-guide/index.html).\n\n### Create environment\n\nSDFStudio requires `python \u003e= 3.7`. We recommend using conda to manage dependencies. Make sure to install [Conda](https://docs.conda.io/en/latest/miniconda.html) before proceeding.\n\n```bash\nconda create --name sdfstudio -y python=3.8\nconda activate sdfstudio\npython -m pip install --upgrade pip\n```\n\n### Dependencies\n\nInstall pytorch with CUDA (this repo has been tested with CUDA 11.3) and [tiny-cuda-nn](https://github.com/NVlabs/tiny-cuda-nn)\n\n```bash\npip install torch==1.12.1+cu113 torchvision==0.13.1+cu113 -f https://download.pytorch.org/whl/torch_stable.html\npip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch\n```\n\n### Installing SDFStudio\n\n```bash\ngit clone https://github.com/autonomousvision/sdfstudio.git\ncd sdfstudio\npip install --upgrade pip setuptools\npip install -e .\n# install tab completion\nns-install-cli\n```\n\n## 2. Train your first model\n\nThe following will train a _NeuS-facto_ model,\n\n```bash\n# Download some test data: you might need to install curl if your system don't have that\nns-download-data sdfstudio\n\n# Train model on the dtu dataset scan65\nns-train neus-facto --pipeline.model.sdf-field.inside-outside False --vis viewer --experiment-name neus-facto-dtu65 sdfstudio-data --data data/sdfstudio-demo-data/dtu-scan65\n\n# Or you could also train model on the Replica dataset room0 with monocular priors\nns-train neus-facto --pipeline.model.sdf-field.inside-outside True --pipeline.model.mono-depth-loss-mult 0.1 --pipeline.model.mono-normal-loss-mult 0.05 --vis viewer --experiment-name neus-facto-replica1 sdfstudio-data --data data/sdfstudio-demo-data/replica-room0 --include_mono_prior True\n```\n\nIf everything works, you should see the following training progress:\n\n\u003cp align=\"center\"\u003e\n    \u003cimg width=\"800\" alt=\"image\" src=\"media/training-process.png\"\u003e\n\u003c/p\u003e\n\nNavigating to the link at the end of the terminal will load the webviewer (developled by nerfstudio). If you are running on a remote machine, you will need to port forward the websocket port (defaults to 7007). With an RTX3090 GPU, it takes ~15 mins for 20K iterations but you can already see reasonable reconstruction results after 2K iterations in the webviewer.\n\n\u003cp align=\"center\"\u003e\n    \u003cimg width=\"800\" alt=\"image\" src=\"media/viewer_screenshot.png\"\u003e\n\u003c/p\u003e\n\n### Resume from checkpoint / visualize existing run\n\nIt is also possible to load a pretrained model by running\n\n```bash\nns-train neus-facto --trainer.load-dir {outputs/neus-facto-dtu65/neus-facto/XXX/sdfstudio_models} sdfstudio-data --data data/sdfstudio-demo-data/dtu-scan65 \n```\n\nThis will automatically resume training. If you do not want to resume training, add `--viewer.start-train False` to your training command. **Note that the order of command matters, dataparser subcommand needs to come after the model subcommand.**\n\n## 3. Exporting Results\n\nOnce you have a trained model you can export mesh and render the mesh.\n\n### Extract Mesh\n\n```bash\nns-extract-mesh --load-config outputs/neus-facto-dtu65/neus-facto/XXX/config.yml --output-path meshes/neus-facto-dtu65.ply\n```\n\n### Render Mesh\n\n```\nns-render-mesh --meshfile meshes/neus-facto-dtu65.ply --traj interpolate  --output-path renders/neus-facto-dtu65.mp4 sdfstudio-data --data data/sdfstudio-demo-data/dtu-scan65\n```\n\nYou will get the following video if everything works properly.\n\nhttps://user-images.githubusercontent.com/13434986/207892086-dd6cae89-7271-4904-9163-6a9bfec49a12.mp4\n\n### Render Video\n\nFirst we must create a path for the camera to follow. This can be done in the viewer under the \"RENDER\" tab. Orient your 3D view to the location where you wish the video to start, then press \"ADD CAMERA\". This will set the first camera key frame. Continue to new viewpoints adding additional cameras to create the camera path. We provide other parameters to further refine your camera path. Once satisfied, press \"RENDER\" which will display a modal that contains the command needed to render the video. Kill the training job (or create a new terminal if you have lots of compute) and the command to generate the video.\n\nTo view all video export options run:\n\n```bash\nns-render --help\n```\n\n## 4. Advanced Options\n\n### Training models other than NeuS-facto\n\nWe provide many other models than NeuS-facto, see [the documentation](docs/sdfstudio-methods.md). For example, if you want to train the original NeuS model, use the following command:\n\n```bash\nns-train neus --pipeline.model.sdf-field.inside-outside False sdfstudio-data --data data/sdfstudio-demo-data/dtu-scan65\n```\n\nFor a full list of included models run `ns-train --help`. Please refer to the [documentation](docs/sdfstudio-methods.md) for a more detailed explanation for each method.\n\n### Modify Configuration\n\nEach model contains many parameters that can be changed, too many to list here. Use the `--help` command to see the full list of configuration options.\n\n**Note, that order of parameters matters! For example, you cannot set `--machine.num-gpus` after the `--data` parameter**\n\n```bash\nns-train neus-facto --help\n```\n\n\u003cdetails\u003e\n\u003csummary\u003e[Click to see output]\u003c/summary\u003e\n\n![help-output](media/help-output.png)\n\n\u003c/details\u003e\n\n### Tensorboard / WandB\n\nNerfstudio supports three different methods to track training progress, using the viewer, [tensorboard](https://www.tensorflow.org/tensorboard), and [Weights and Biases](https://wandb.ai/site). These visualization tools can also be used in SDFStudio. You can specify which visualizer to use by appending `--vis {viewer, tensorboard, wandb}` to the training command. Note that only one may be used at a time. Additionally the viewer only works for methods that are fast (ie. `NeuS-facto` and `NeuS-acc`), for slower methods like `NeuS-facto-bigmlp`, use the other loggers.\n\n## 5. Using Custom Data\n\nPlease refer to the [datasets](docs/sdfstudio-data.md) and [data format](https://github.com/autonomousvision/sdfstudio/blob/master/docs/sdfstudio-data.md#Dataset-format) documentation if you like to use custom datasets.\n\n# Built On\n\n\u003ca href=\"https://github.com/nerfstudio-project/nerfstudio\"\u003e\n\u003c!-- pypi-strip --\u003e\n\u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://docs.nerf.studio/en/latest/_images/logo.png\" /\u003e\n\u003c!-- /pypi-strip --\u003e\n    \u003cimg alt=\"tyro logo\" src=\"https://docs.nerf.studio/en/latest/_images/logo.png\" width=\"150px\" /\u003e\n\u003c!-- pypi-strip --\u003e\n\u003c/picture\u003e\n\u003c!-- /pypi-strip --\u003e\n\u003c/a\u003e\n\n- A collaboration friendly studio for NeRFs\n- Developed by [nerfstudio team](https://github.com/nerfstudio-project)\n\n\u003ca href=\"https://github.com/brentyi/tyro\"\u003e\n\u003c!-- pypi-strip --\u003e\n\u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://brentyi.github.io/tyro/_static/logo-dark.svg\" /\u003e\n\u003c!-- /pypi-strip --\u003e\n    \u003cimg alt=\"tyro logo\" src=\"https://brentyi.github.io/tyro/_static/logo-light.svg\" width=\"150px\" /\u003e\n\u003c!-- pypi-strip --\u003e\n\u003c/picture\u003e\n\u003c!-- /pypi-strip --\u003e\n\u003c/a\u003e\n\n- Easy-to-use config system\n- Developed by [Brent Yi](https://brentyi.com/)\n\n\u003ca href=\"https://github.com/KAIR-BAIR/nerfacc\"\u003e\n\u003c!-- pypi-strip --\u003e\n\u003cpicture\u003e\n    \u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"https://user-images.githubusercontent.com/3310961/199083722-881a2372-62c1-4255-8521-31a95a721851.png\" /\u003e\n\u003c!-- /pypi-strip --\u003e\n    \u003cimg alt=\"tyro logo\" src=\"https://user-images.githubusercontent.com/3310961/199084143-0d63eb40-3f35-48d2-a9d5-78d1d60b7d66.png\" width=\"250px\" /\u003e\n\u003c!-- pypi-strip --\u003e\n\u003c/picture\u003e\n\u003c!-- /pypi-strip --\u003e\n\u003c/a\u003e\n\n- Library for accelerating NeRF renders\n- Developed by [Ruilong Li](https://www.liruilong.cn/)\n\n# Citation\n\nIf you use this library or find the documentation useful for your research, please consider citing:\n\n```bibtex\n@misc{Yu2022SDFStudio,\n    author    = {Yu, Zehao and Chen, Anpei and Antic, Bozidar and Peng, Songyou and Bhattacharyya, Apratim \n                 and Niemeyer, Michael and Tang, Siyu and Sattler, Torsten and Geiger, Andreas},\n    title     = {SDFStudio: A Unified Framework for Surface Reconstruction},\n    year      = {2022},\n    url       = {https://github.com/autonomousvision/sdfstudio},\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/autonomousvision.github.io%2Fsdfstudio%2F","html_url":"https://awesome.ecosyste.ms/projects/autonomousvision.github.io%2Fsdfstudio%2F","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/autonomousvision.github.io%2Fsdfstudio%2F/lists"}