{"id":13442774,"url":"https://github.com/zju3dv/Vox-Fusion","last_synced_at":"2025-03-20T15:30:47.498Z","repository":{"id":63365328,"uuid":"526969398","full_name":"zju3dv/Vox-Fusion","owner":"zju3dv","description":"Code for \"Dense Tracking and Mapping with Voxel-based Neural Implicit Representation\", ISMAR 2022","archived":false,"fork":false,"pushed_at":"2025-02-24T09:52:43.000Z","size":5831,"stargazers_count":271,"open_issues_count":10,"forks_count":25,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-03-16T04:13:29.748Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zju3dv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-20T15:44:51.000Z","updated_at":"2025-02-27T22:39:56.000Z","dependencies_parsed_at":"2024-02-27T03:27:55.011Z","dependency_job_id":"8b3442f2-2875-4ac4-891c-b880f7552ece","html_url":"https://github.com/zju3dv/Vox-Fusion","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FVox-Fusion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FVox-Fusion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FVox-Fusion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zju3dv%2FVox-Fusion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zju3dv","download_url":"https://codeload.github.com/zju3dv/Vox-Fusion/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244639896,"owners_count":20485947,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-31T03:01:50.500Z","updated_at":"2025-03-20T15:30:47.493Z","avatar_url":"https://github.com/zju3dv.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Vox-Fusion\n\n### [Project Page](https://zju3dv.github.io/Vox-Fusion/) | [Video](https://youtu.be/Prp28y1b2Qs) | [Paper](https://arxiv.org/abs/2210.15858)\n\n\n\u003e Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation \n\u003e \n\u003e [Xingrui Yang*](https://yangxingrui.com/), [Hai Li*](https://garylidd.github.io/), [Hongjia Zhai](https://zhaihongjia.github.io/), Yuhang Ming, Yuqian Liu, [Guofeng Zhang](http://www.cad.zju.edu.cn/home/gfzhang/). \n\u003e \n\u003e ISMAR 2022\n\n## Correction\n\n\nWe found a bug in the evaluation script which affected the estimated pose accuracy in Tables 1 and 3 in the original paper. We have corrected this problem and re-run the results with updated configurations. The corrected results are comparable (even better for Replica dataset) to the originally reported results in the paper, which do not affect the contribution and conclusion of our work. We have updated the [arxiv version](https://arxiv.org/abs/2210.15858) of our paper and publish all the latest results (including mesh, pose, gt, eval scripts and training configs) on [Google Drive](https://drive.google.com/drive/folders/15bwoX44uh1YCsMT4F3Td9n1h2qiUOuVG?usp=share_link), in case anyone wants to reproduce our results and compare them using different metrics.\n\n\u003cp align=\"center\"\u003e\n\u003cimg width=\"856\" alt=\"table1\" src=\"https://github.com/zju3dv/Vox-Fusion/assets/48756786/d6ad3115-786a-4dc3-b6d7-14b46b5da976\"\u003e\n\u003cimg width=\"856\" alt=\"table2\" src=\"https://github.com/user-attachments/assets/9c886887-d769-40fe-82cf-4d05510d187c\"\u003e\n\u003cimg width=\"445\" alt=\"table3\" src=\"https://github.com/zju3dv/Vox-Fusion/assets/48756786/df18a2cb-b37f-417e-9228-b4e3358034b7\"\u003e\n\u003c/p\u003e\n\nPlease use the config file replica_all.yaml and scannet_all.yaml in the **Google Drive** to replicate the results from the paper !!!\n\n**If you want to compare our method, please use the data or config file we provide to run.**\n\n## Performance on TUM-RGBD\nWe provide the results on the TUM-RGBD dataset.\n\n|     Metrics      | fr1/desk | fr1/desk2 | fr1/room |fr2/xyz | fr3/office | \n|:----------------:|:-------------:|:--------------:|:--------------:|:-----------------------:|:---------------------:|\n|       ATE RMSE [cm]       |     2.54      |      3.95      |      12.51      |          1.40           |        -         |\n|       ATE Mean [cm]       |     2.26      |      3.18      |      11.63      |          1.19           |        -         |\n|      ATE Median [cm]      |     2.11      |      2.64      |      10.54      |          0.95           |        -         |\n\n\n## Installation\n\nIt is recommended to install [Pytorch](https://pytorch.org/get-started/locally/) (\u003e=1.10) manually for your hardware platform first. You can then install all dependancies using `pip` or `conda`:\n\n```\npip install -r requirements.txt\n```\n\nAfter you have installed all third party libraries, run the following script to build extra Pytorch modules used in this project.\n\n```bash\nsh install.sh\n```\n\n\nReplace the filename in mapping.py with the built library\n```python\ntorch.classes.load_library(\"third_party/sparse_octree/build/lib.xxx/svo.xxx.so\")\n```\n\n\n## Demo\n\nIt is simple to run Vox-Fusion on datasets that already have dataloaders. `src/datasets` list all existing dataloaders. You can of course build your own, we will come back to it later. For now, we use the replica dataset as an example. \n\nFirst you have to modify `configs/replica/room_0.yaml` so the `data_path` section points to the real dataset path. Now you are all set to run the code:\n\n```\npython demo/run.py configs/replica/room_0.yaml\n```\n\nYou should now see a progress bar and some output indicating the system is now running. For now you have to rely on the progress bar to estimate the running time as we are still working on a working GUI.\n\n## Custom Datasets\n\nYou can use virtually any RGB-D dataset with Vox-Fusion including self-captured ones. Make sure to adapt the config files and dataloaders and put them in the correct folder. Make sure to implement a `get_init_pose` function for your dataloader, please refer to `src/datasets/tum.py` for an example.\n\n## Acknowledgement\n\nSome of our codes are adapted from [Nerual RGB-D Surface Reconstruction](https://dazinovic.github.io/neural-rgbd-surface-reconstruction/) and [BARF: Bundle Adjusted NeRF](https://github.com/chenhsuanlin/bundle-adjusting-NeRF/blob/main/camera.py).\n\n## Citation\n\nIf you find our code or paper useful, please cite\n\n```bibtex\n@inproceedings{yang2022vox,\n  title={Vox-Fusion: Dense Tracking and Mapping with Voxel-based Neural Implicit Representation},\n  author={Yang, Xingrui and Li, Hai and Zhai, Hongjia and Ming, Yuhang and Liu, Yuqian and Zhang, Guofeng},\n  booktitle={2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)},\n  pages={499--507},\n  year={2022},\n}\n```\n\n## Contact\nContact [Xingrui Yang](mailto:xingruiy@gmail.com) and [Hai Li](mailto:gary_li@zju.edu.cn) for questions, comments and reporting bugs.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju3dv%2FVox-Fusion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzju3dv%2FVox-Fusion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzju3dv%2FVox-Fusion/lists"}