{"id":13504332,"url":"https://github.com/JingwenWang95/KinectFusion","last_synced_at":"2025-03-29T21:30:31.165Z","repository":{"id":41192523,"uuid":"453537621","full_name":"JingwenWang95/KinectFusion","owner":"JingwenWang95","description":"KinectFusion implemented in Python with PyTorch","archived":false,"fork":false,"pushed_at":"2023-04-11T02:56:38.000Z","size":8847,"stargazers_count":183,"open_issues_count":0,"forks_count":23,"subscribers_count":6,"default_branch":"master","last_synced_at":"2024-11-01T01:33:31.234Z","etag":null,"topics":["3d-reconstruction","dense-slam","iterative-closest-point","kinect-fusion","python","pytorch-implementation","rgbd-slam","slam"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JingwenWang95.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-01-29T22:51:59.000Z","updated_at":"2024-10-31T07:49:31.000Z","dependencies_parsed_at":"2024-11-01T01:31:37.048Z","dependency_job_id":"b0be4265-8797-45ac-96bc-f4f6920344a0","html_url":"https://github.com/JingwenWang95/KinectFusion","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingwenWang95%2FKinectFusion","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingwenWang95%2FKinectFusion/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingwenWang95%2FKinectFusion/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JingwenWang95%2FKinectFusion/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JingwenWang95","download_url":"https://codeload.github.com/JingwenWang95/KinectFusion/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246249088,"owners_count":20747164,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","dense-slam","iterative-closest-point","kinect-fusion","python","pytorch-implementation","rgbd-slam","slam"],"created_at":"2024-08-01T00:00:30.236Z","updated_at":"2025-03-29T21:30:31.159Z","avatar_url":"https://github.com/JingwenWang95.png","language":"Python","funding_links":[],"categories":["Implementations"],"sub_categories":[],"readme":"# KinectFusion implemented in Python with PyTorch\n\n\u003cimg src=\"images/kinfu.gif\" height=240px align=\"right\"/\u003e\n\nThis is a lightweight Python implementation of [KinectFusion](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ismar2011.pdf). All the core functions (TSDF volume, frame-to-model tracking, point-to-plane ICP, raycasting, TSDF fusion, etc.) are implemented using pure PyTorch, i.e. no custom CUDA kernels. \n\nAlthough without any custom CUDA functions, the system could still run at a fairly fast speed: The demo reconstructs the [TUM fr1_desk sequence](https://vision.in.tum.de/data/datasets/rgbd-dataset/download#) into a 225 x 171 x 111 TSDF volume with 2cm resolution at round 17 FPS with a single RTX-2080 GPU (~1.5 FPS in CPU mode) \n\nNote that this project is mainly for study purpose, and is not fully optimized for accurate camera tracking.\n\n## Requirements\nThe core functionalities were implemented in PyTorch (1.10). Open3D (0.14.0) is used for visualisation. Other important dependancies include:\n\n* numpy==1.21.2\n* opencv-python==4.5.5\n* imageio==2.14.1\n* scikit-image==0.19.1\n* trimesh==3.9.43\n\nYou can create an anaconda environment called `kinfu` with the required dependencies by running:\n```\nconda env create -f environment.yml\nconda activate kinfu\n```\n\n## Data Preparation\nThe code was tested on [TUM dataset](https://vision.in.tum.de/data/datasets/rgbd-dataset/download). After downloading the raw sequences, you will need to run the pre-processing script under `dataset/`. For example:\n\n```\npython dataset/preprocess.py --config configs/fr1_desk.yaml\n```\n\nThere are some example config files under [`configs/`](https://github.com/JingwenWang95/KinectFusion/tree/master/configs) which correspond to different sequences. You need to replace [`data_root`](https://github.com/JingwenWang95/KinectFusion/blob/master/configs/fr1_desk.yaml#L1) to your own sequence directory before running the script. \nAfter running the script a new directory `processed/` will appear under your sequence directory. \n\n## Run\nAfter obtaining the processed sequence, you can simply run `kinfu.py`. For example:\n\n```\npython kinfu.py --config configs/fr1_desk.yaml --save_dir reconstruct/fr1_desk\n```\n\nwhich will perform the tracking and mapping headlessly and save the results. Or you could run:\n\n```\npython kinfu_gui.py --config configs/fr1_desk.yaml\n```\n\nIf you want to visualize the tracking and reconstruction process on-the-fly.\n\n## Acknowledgement\nThe code of ICP tracker was heavily borrowed from [Binbin Xu](https://github.com/binbin-xu). Also thank Binbin for implementing part of the TSDF volume code which is inspired by [Andy Zeng's tsdf-fusion-python](https://github.com/andyzeng/tsdf-fusion-python).\n\n## References\n * [KinectFusion: Real-Time Dense Surface Mapping and Tracking (ISMAR 2011)](https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/ismar2011.pdf)\n * [Deep Probabilistic Feature-metric Tracking (RA-L and ICRA 2021 presentation)](https://github.com/smartroboticslab/deep_prob_feature_track)\n * [Taking a Deeper Look at the Inverse Compositional Algorithm (CVPR 2019)](https://arxiv.org/pdf/1812.06861.pdf)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJingwenWang95%2FKinectFusion","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJingwenWang95%2FKinectFusion","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJingwenWang95%2FKinectFusion/lists"}