{"id":15116031,"url":"https://github.com/HengyiWang/spann3r","last_synced_at":"2025-09-27T21:31:39.055Z","repository":{"id":256547275,"uuid":"855712430","full_name":"HengyiWang/spann3r","owner":"HengyiWang","description":"3D Reconstruction with Spatial Memory","archived":false,"fork":false,"pushed_at":"2024-09-20T14:41:05.000Z","size":7971,"stargazers_count":434,"open_issues_count":10,"forks_count":16,"subscribers_count":12,"default_branch":"main","last_synced_at":"2024-09-27T01:50:16.966Z","etag":null,"topics":["3d-reconstruction"],"latest_commit_sha":null,"homepage":"https://hengyiwang.github.io/projects/spanner","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/HengyiWang.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-11T10:42:28.000Z","updated_at":"2024-09-26T12:05:56.000Z","dependencies_parsed_at":"2024-09-18T01:33:37.220Z","dependency_job_id":null,"html_url":"https://github.com/HengyiWang/spann3r","commit_stats":null,"previous_names":["hengyiwang/spann3r"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/HengyiWang/spann3r","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HengyiWang%2Fspann3r","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HengyiWang%2Fspann3r/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HengyiWang%2Fspann3r/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HengyiWang%2Fspann3r/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/HengyiWang","download_url":"https://codeload.github.com/HengyiWang/spann3r/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/HengyiWang%2Fspann3r/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":277295918,"owners_count":25794402,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-27T02:00:08.978Z","response_time":73,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction"],"created_at":"2024-09-26T01:44:07.792Z","updated_at":"2025-09-27T21:31:39.050Z","avatar_url":"https://github.com/HengyiWang.png","language":"Python","funding_links":[],"categories":["Python","3D视觉生成重建"],"sub_categories":["资源传输下载"],"readme":"# 3D Reconstruction with Spatial Memory\n\n### [Paper](https://arxiv.org/abs/2408.16061) | [Project Page](https://hengyiwang.github.io/projects/spanner) | [Video](https://hengyiwang.github.io/projects/spanner/videos/spanner_intro.mp4)\n\n\u003e 3D Reconstruction with Spatial Memory \u003cbr /\u003e\n\u003e [Hengyi Wang](https://hengyiwang.github.io/), [Lourdes Agapito](http://www0.cs.ucl.ac.uk/staff/L.Agapito/)\u003cbr /\u003e\n\u003e arXiv 2024\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"\"\u003e\n    \u003cimg src=\"./assets/spann3r_teaser_white.gif\" alt=\"Logo\" width=\"90%\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n## Update\n\n[2025-02-25] Spann3R v1.01 checkpoint [released](https://drive.google.com/drive/folders/1bqtcVf8lK4VC8LgG-SIGRBECcrFqM7Wy?usp=sharing)\n\n[2024-10-25] Add support for [Nerfstudio](assets/spanner-gs.gif)\n\n[2024-10-18] Add camera param estimation\n\n[2024-09-30] [@hugoycj](https://github.com/hugoycj) adds a gradio demo\n\n[2024-09-20] Instructions for datasets [data_preprocess.md](docs/data_preprocess.md)\n\n[2024-09-11] Code for Spann3R\n\n\n\n## Release Notes\n\n**Spann3R v1.01:** We further train Spann3R with 10-frame sequence on a mixture of 15 datasets, including ScanNet, ScanNetpp, WildRGBD, Co3D, Aria, ArkitScene, BlendMVS, Waymo, Tartanair, OminiObject3d, Megadepth, Vkitti2, Unreal, Spring, Pointodyssey. (NOTE: We have removed Habitat from training, as we found that using our rendered Habitat sequences led to [failure on synthetic data](https://github.com/HengyiWang/spann3r/issues/1)) We report the updated chamfer distance (mean) here:\n\n|               | 7 Scenes   | NRGBD      | Replica    | DTU       |\n| ------------- | ---------- | ---------- | ---------- | --------- |\n| Spann3R       | 0.0291     | 0.0491     | N/A        | 3.764     |\n| Spann3R v1.01 | **0.0255** | **0.0437** | **0.0480** | **2.955** |\n\nAlso, since we include some dynamic scenes, Spann3R v1.01 now supports static/dynamic scene reconstruction. Here are some qualitative examples:\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"\"\u003e\n    \u003cimg src=\"./assets/spann3r101.gif\" alt=\"Logo\" width=\"90%\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\nWe acknowledge the support of the [UKRI/EPSRC AI Hub in Generative Models](https://www.genai.ac.uk/) [grant number EP/Y028805/1] for computing resources.\n\n## Installation\n\n1. Clone Spann3R\n\n   ```\n   git clone https://github.com/HengyiWang/spann3r.git\n   cd spann3r\n   ```\n   \n2. Create conda environment\n\n   ```\n   conda create -n spann3r python=3.9 cmake=3.14.0\n   conda install pytorch==2.3.0 torchvision==0.18.0 torchaudio==2.3.0 pytorch-cuda=11.8 -c pytorch -c nvidia  # use the correct version of cuda for your system\n   \n   pip install -r requirements.txt\n   \n   # Open3D has a bug from 0.16.0, please use dev version\n   pip install -U -f https://www.open3d.org/docs/latest/getting_started.html open3d\n   ```\n\n3. Compile cuda kernels for RoPE\n\n   ```\n   cd croco/models/curope/\n   python setup.py build_ext --inplace\n   cd ../../../\n   ```\n\n4. Download the DUSt3R checkpoint\n\n   ```\n   mkdir checkpoints\n   cd checkpoints\n   # Download DUSt3R checkpoints\n   wget https://download.europe.naverlabs.com/ComputerVision/DUSt3R/DUSt3R_ViTLarge_BaseDecoder_512_dpt.pth\n   ```\n\n5. Download our [checkpoint](https://drive.google.com/drive/folders/1bqtcVf8lK4VC8LgG-SIGRBECcrFqM7Wy?usp=sharing) and place it under `./checkpoints`\n\n## Demo\n\n1. Download the [example data](https://drive.google.com/drive/folders/1bqtcVf8lK4VC8LgG-SIGRBECcrFqM7Wy?usp=sharing) (2 scenes from [map-free-reloc](https://github.com/nianticlabs/map-free-reloc)) and unzip it as `./examples`\n\n2. Run demo:\n\n   ```\n   python demo.py --demo_path ./examples/s00567 --kf_every 10 --vis --vis_cam # --dynamic\n   ```\n\n   For visualization `--vis`, it will give you a window to adjust the rendering view. Once you find the view to render, please click `space key` and close the window. The code will then do the rendering of the incremental reconstruction. `--dynamic` will give you a dynamic visualization that is suitable for dynamic scene reconstruction.\n   \n3. Nerfstudio:\n\n   ```\n   # Run demo use --save_ori to save scaled intrinsics for original images\n   python demo.py --demo_path ./examples/s00567 --kf_every 10 --vis --vis_cam --save_ori\n   \n   # Run splatfacto\n   ns-train splatfacto --data ./output/demo/s00567 --pipeline.model.camera-optimizer.mode SO3xR3\n   \n   # Render your results\n   ns-render interpolate --load-config [path-to-your-config]/config.yml\n   ```\n\n   Note that here you can use `--save_ori` to save the scaled intrinsics into `transform.json` to train NeRF/3D Gaussians with original images.'\n\n\n## Gradio interface \n\nWe also provide a Gradio interface for a better experience, just run by:\n\n```bash\n# For Linux and Windows users (and macOS with Intel??)\npython app.py\n```\n\nYou can specify the `--server_port`, `--share`, `--server_name` arguments to satisfy your needs!\n\n\n## Training and Evaluation\n\n### Datasets\n\nWe use Habitat, ScanNet++, ScanNet, ArkitScenes, Co3D, and BlendedMVS to train our model. Please refer to [data_preprocess.md](docs/data_preprocess.md).\n\n### Train\n\nPlease use the following command to train our model:\n\n```\ntorchrun --nproc_per_node 8 train.py --batch_size 4\n```\n\n### Eval\n\nPlease use the following command to evaluate our model:\n\n```\npython eval.py\n```\n\n\n\n\n## Acknowledgement \n\nOur code, data preprocessing pipeline, and evaluation scripts are based on several awesome repositories:\n\n- [DUSt3R](https://github.com/naver/dust3r)\n- [SplaTAM](https://github.com/spla-tam/SplaTAM)\n- [NeRFStudio](https://github.com/nerfstudio-project/nerfstudio)\n- [MVSNet](https://github.com/YoYo000/MVSNet)\n- [NICE-SLAM](https://github.com/cvg/nice-slam)\n- [NeuralRGBD](https://github.com/dazinovic/neural-rgbd-surface-reconstruction)\n- [SimpleRecon](https://github.com/nianticlabs/simplerecon)\n\nWe thank the authors for releasing their code!\n\nThe research presented here has been supported by a sponsored research award from Cisco Research and the UCL Centre for Doctoral Training in Foundational AI under UKRI grant number EP/S021566/1. This project made use of time on Tier 2 HPC facility JADE2, funded by EPSRC (EP/T022205/1).\n\n## Citation\n\nIf you find our code or paper useful for your research, please consider citing:\n\n```\n@article{wang20243d,\n  title={3D Reconstruction with Spatial Memory},\n  author={Wang, Hengyi and Agapito, Lourdes},\n  journal={arXiv preprint arXiv:2408.16061},\n  year={2024}\n}\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHengyiWang%2Fspann3r","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FHengyiWang%2Fspann3r","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FHengyiWang%2Fspann3r/lists"}