{"id":15036575,"url":"https://github.com/nvlabs/neuralangelo","last_synced_at":"2025-04-13T02:00:50.268Z","repository":{"id":187771894,"uuid":"657780946","full_name":"NVlabs/neuralangelo","owner":"NVlabs","description":"Official implementation of \"Neuralangelo: High-Fidelity Neural Surface Reconstruction\" (CVPR 2023)","archived":false,"fork":false,"pushed_at":"2024-04-14T21:29:12.000Z","size":15127,"stargazers_count":4478,"open_issues_count":102,"forks_count":395,"subscribers_count":61,"default_branch":"main","last_synced_at":"2025-04-06T00:05:53.172Z","etag":null,"topics":["3d-reconstruction","computer-graphics","computer-vision","neural-rendering","neural-representation"],"latest_commit_sha":null,"homepage":"https://research.nvidia.com/labs/dir/neuralangelo/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NVlabs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-23T20:52:42.000Z","updated_at":"2025-04-03T14:22:52.000Z","dependencies_parsed_at":null,"dependency_job_id":"3911efa6-4db4-42c3-b0c4-aba85ee7b406","html_url":"https://github.com/NVlabs/neuralangelo","commit_stats":null,"previous_names":["nvlabs/neuralangelo"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVlabs%2Fneuralangelo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVlabs%2Fneuralangelo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVlabs%2Fneuralangelo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NVlabs%2Fneuralangelo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NVlabs","download_url":"https://codeload.github.com/NVlabs/neuralangelo/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247415966,"owners_count":20935388,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","computer-graphics","computer-vision","neural-rendering","neural-representation"],"created_at":"2024-09-24T20:31:36.254Z","updated_at":"2025-04-06T00:06:04.963Z","avatar_url":"https://github.com/NVlabs.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Neuralangelo\nThis is the official implementation of **Neuralangelo: High-Fidelity Neural Surface Reconstruction**.\n\n[Zhaoshuo Li](https://mli0603.github.io/),\n[Thomas Müller](https://tom94.net/),\n[Alex Evans](https://research.nvidia.com/person/alex-evans),\n[Russell H. Taylor](https://www.cs.jhu.edu/~rht/),\n[Mathias Unberath](https://mathiasunberath.github.io/),\n[Ming-Yu Liu](https://mingyuliu.net/),\n[Chen-Hsuan Lin](https://chenhsuanlin.bitbucket.io/)  \nIEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023\n\n### [Project page](https://research.nvidia.com/labs/dir/neuralangelo/) | [Paper](https://arxiv.org/abs/2306.03092/) | [Colab notebook](https://colab.research.google.com/drive/13u8DX9BNzQwiyPPCB7_4DbSxiQ5-_nGF)\n\n\u003cimg src=\"assets/teaser.gif\"\u003e\n\nThe code is built upon the Imaginaire library from the Deep Imagination Research Group at NVIDIA.  \nFor business inquiries, please submit the [NVIDIA research licensing form](https://www.nvidia.com/en-us/research/inquiries/).\n\n--------------------------------------\n\n## Installation\nWe offer two ways to setup the environment:\n1. We provide prebuilt Docker images, where\n    - `docker.io/chenhsuanlin/colmap:3.8` is for running COLMAP and the data preprocessing scripts. This includes the prebuilt COLMAP library (CUDA-supported).\n    - `docker.io/chenhsuanlin/neuralangelo:23.04-py3` is for running the main Neuralangelo pipeline.\n\n    The corresponding Dockerfiles can be found in the `docker` directory.\n2. The conda environment for Neuralangelo. Install the dependencies and activate the environment `neuralangelo` with\n    ```bash\n    conda env create --file neuralangelo.yaml\n    conda activate neuralangelo\n    ```\nFor COLMAP, alternative installation options are also available on the [COLMAP website](https://colmap.github.io/).\n\n--------------------------------------\n\n## Data preparation\nPlease refer to [Data Preparation](DATA_PROCESSING.md) for step-by-step instructions.  \nWe assume known camera poses for each extracted frame from the video.\nThe code uses the same json format as [Instant NGP](https://github.com/NVlabs/instant-ngp).\n\n--------------------------------------\n\n## Run Neuralangelo!\n```bash\nEXPERIMENT=toy_example\nGROUP=example_group\nNAME=example_name\nCONFIG=projects/neuralangelo/configs/custom/${EXPERIMENT}.yaml\nGPUS=1  # use \u003e1 for multi-GPU training!\ntorchrun --nproc_per_node=${GPUS} train.py \\\n    --logdir=logs/${GROUP}/${NAME} \\\n    --config=${CONFIG} \\\n    --show_pbar\n```\nSome useful notes:\n- This codebase supports logging with [Weights \u0026 Biases](https://wandb.ai/site). You should have a W\u0026B account for this.\n    - Add `--wandb` to the command line argument to enable W\u0026B logging.\n    - Add `--wandb_name` to specify the W\u0026B project name.\n    - More detailed control can be found in the `init_wandb()` function in `imaginaire/trainers/base.py`.\n- Configs can be overridden through the command line (e.g. `--optim.params.lr=1e-2`).\n- Set `--checkpoint={CHECKPOINT_PATH}` to initialize with a certain checkpoint; set `--resume` to resume training.\n- If appearance embeddings are enabled, make sure `data.num_images` is set to the number of training images.\n\n--------------------------------------\n\n## Isosurface extraction\nUse the following command to run isosurface mesh extraction:\n```bash\nCHECKPOINT=logs/${GROUP}/${NAME}/xxx.pt\nOUTPUT_MESH=xxx.ply\nCONFIG=logs/${GROUP}/${NAME}/config.yaml\nRESOLUTION=2048\nBLOCK_RES=128\nGPUS=1  # use \u003e1 for multi-GPU mesh extraction\ntorchrun --nproc_per_node=${GPUS} projects/neuralangelo/scripts/extract_mesh.py \\\n    --config=${CONFIG} \\\n    --checkpoint=${CHECKPOINT} \\\n    --output_file=${OUTPUT_MESH} \\\n    --resolution=${RESOLUTION} \\\n    --block_res=${BLOCK_RES}\n```\nSome useful notes:\n- Add `--textured` to extract meshes with textures.\n- Add `--keep_lcc` to remove noises. May also remove thin structures.\n- Lower `BLOCK_RES` to reduce GPU memory usage.\n- Lower `RESOLUTION` to reduce mesh size.\n\n--------------------------------------\n\n## Frequently asked questions (FAQ)\n1. **Q:** CUDA out of memory. How do I decrease the memory footprint?  \n    **A:** Neuralangelo requires at least 24GB GPU memory with our default configuration. If you run out of memory, consider adjusting the following hyperparameters under `model.object.sdf.encoding.hashgrid` (with suggested values):\n\n    | GPU VRAM      | Hyperparameter          |\n    | :-----------: | :---------------------: |\n    | 8GB           | `dict_size=20`, `dim=4` |\n    | 12GB          | `dict_size=21`, `dim=4` |\n    | 16GB          | `dict_size=21`, `dim=8` |\n\n    Please note that the above hyperparameter adjustment may sacrifice the reconstruction quality.\n\n   If Neuralangelo runs fine during training but CUDA out of memory during evaluation, consider adjusting the evaluation parameters under `data.val`, including setting smaller `image_size` (e.g., maximum resolution 200x200), and setting `batch_size=1`, `subset=1`.\n\n2. **Q:** The reconstruction of my custom dataset is bad. What can I do?  \n    **A:** It is worth looking into the following:\n    - The camera poses recovered by COLMAP may be off. We have implemented tools (using [Blender](https://github.com/mli0603/BlenderNeuralangelo) or [Jupyter notebook](projects/neuralangelo/scripts/visualize_colmap.ipynb)) to inspect the COLMAP results.\n    - The computed bounding regions may be off and/or too small/large. Please refer to [data preprocessing](DATA_PROCESSING.md) on how to adjust the bounding regions manually.\n    - The video capture sequence may contain significant motion blur or out-of-focus frames. Higher shutter speed (reducing motion blur) and smaller aperture (increasing focus range) are very helpful.\n\n--------------------------------------\n\n## Citation\nIf you find our code useful for your research, please cite\n```\n@inproceedings{li2023neuralangelo,\n  title={Neuralangelo: High-Fidelity Neural Surface Reconstruction},\n  author={Li, Zhaoshuo and M\\\"uller, Thomas and Evans, Alex and Taylor, Russell H and Unberath, Mathias and Liu, Ming-Yu and Lin, Chen-Hsuan},\n  booktitle={IEEE Conference on Computer Vision and Pattern Recognition ({CVPR})},\n  year={2023}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvlabs%2Fneuralangelo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvlabs%2Fneuralangelo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvlabs%2Fneuralangelo/lists"}