{"id":21441434,"url":"https://github.com/tum-vision/tandem","last_synced_at":"2025-04-12T22:19:56.965Z","repository":{"id":37394613,"uuid":"422586571","full_name":"tum-vision/tandem","owner":"tum-vision","description":"[CoRL 21'] TANDEM: Tracking and Dense Mapping in Real-time using Deep Multi-view Stereo","archived":false,"fork":false,"pushed_at":"2022-12-29T02:10:16.000Z","size":65095,"stargazers_count":935,"open_issues_count":36,"forks_count":153,"subscribers_count":39,"default_branch":"master","last_synced_at":"2025-04-04T01:11:07.183Z","etag":null,"topics":["3d-reconstruction","multiview-stereo","slam","visual-odometry"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tum-vision.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-10-29T13:31:00.000Z","updated_at":"2025-04-02T09:33:44.000Z","dependencies_parsed_at":"2023-01-31T07:45:54.967Z","dependency_job_id":null,"html_url":"https://github.com/tum-vision/tandem","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tum-vision%2Ftandem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tum-vision%2Ftandem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tum-vision%2Ftandem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tum-vision%2Ftandem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tum-vision","download_url":"https://codeload.github.com/tum-vision/tandem/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248638303,"owners_count":21137649,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d-reconstruction","multiview-stereo","slam","visual-odometry"],"created_at":"2024-11-23T01:26:01.344Z","updated_at":"2025-04-12T22:19:56.942Z","avatar_url":"https://github.com/tum-vision.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003eTANDEM: Tracking and Dense Mapping\u003cbr\u003ein Real-time using Deep Multi-view Stereo\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://lukaskoestler.com\"\u003eLukas Koestler\u003c/a\u003e\u003csup\u003e1*\u003c/sup\u003e \u0026emsp;\u0026emsp;\n    \u003ca href=\"https://vision.in.tum.de/members/yangn\"\u003eNan Yang\u003c/a\u003e\u003csup\u003e1,2*,\u0026dagger;\u003c/sup\u003e \u0026emsp;\u0026emsp;\n    \u003ca href=\"https://www.niclas-zeller.de\"\u003eNiclas Zeller\u003c/a\u003e\u003csup\u003e2,3\u003c/sup\u003e \u0026emsp;\u0026emsp;\n    \u003ca href=\"https://vision.in.tum.de/members/cremers\"\u003eDaniel Cremers\u003c/a\u003e\u003csup\u003e1,2\u003c/sup\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003csup\u003e*\u003c/sup\u003eequal contribution\u0026emsp;\u0026emsp;\u0026emsp;\n    \u003csup\u003e\u0026dagger;\u003c/sup\u003ecorresponding author\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    \u003csup\u003e1\u003c/sup\u003eTechnical University of Munich\u0026emsp;\u0026emsp;\u0026emsp;\n    \u003csup\u003e2\u003c/sup\u003eArtisense\u003cbr\u003e\n    \u003csup\u003e3\u003c/sup\u003eKarlsruhe University of Applied Sciences\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n    Conference on Robot Learning (CoRL) 2021, London, UK\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://3dv2021.surrey.ac.uk/prizes\"\u003e3DV 2021 Best Demo Award\u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n    \u003ca href=\"https://arxiv.org/abs/2111.07418\"\u003earXiv\u003c/a\u003e |\n    \u003ca href=\"https://youtu.be/L4C8Q6Gvl1w\"\u003eVideo\u003c/a\u003e |\n    \u003ca href=\"https://openreview.net/forum?id=FzMHiDesj0I\"\u003eOpenReview\u003c/a\u003e |\n    \u003ca href=\"https://go.vision.in.tum.de/tandem\"\u003eProject Page\u003c/a\u003e\n\u003c/p\u003e\n\n## Code and Data\n- [x] 📣 C++ code released before Christmas! Please check [tandem/](tandem/).\n- [x] 📣 CVA-MVSNet released! Please check [cva_mvsnet/](cva_mvsnet/).\n- [x] 📣 Replica training data released! Please check [replica/](replica/).\n- [x] Minor improvements throughout January. **Contributions are highly welcomed!**\n- [x] Release of the ScanNet-trained model \n- [ ] Docker image for TANDEM. **Contributions are highly welcomed!**\n\n### Abstract\n\u003cp align=\"justify\"\u003eIn this paper, we present TANDEM a real-time monocular tracking and dense mapping framework. For pose estimation, TANDEM performs photometric bundle adjustment based on a sliding window of keyframes. To increase the robustness, we propose a novel tracking front-end that performs dense direct image alignment using depth maps rendered from a global model that is built incrementally from dense depth predictions. To predict the dense depth maps, we propose Cascade View-Aggregation MVSNet (CVA-MVSNet) that utilizes the entire active keyframe window by hierarchically constructing 3D cost volumes with adaptive view aggregation to balance the different stereo baselines between the keyframes. Finally, the predicted depth maps are fused into a consistent global map represented as a truncated signed distance function (TSDF) voxel grid. Our experimental results show that TANDEM outperforms other state-of-the-art traditional and learning-based monocular visual odometry (VO) methods in terms of camera tracking. Moreover, TANDEM shows state-of-the-art real-time 3D reconstruction performance.\u003c/p\u003e\n\n\n### Poster\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/tandem_poster.jpg\"\u003e\n\u003c/p\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftum-vision%2Ftandem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftum-vision%2Ftandem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftum-vision%2Ftandem/lists"}