{"id":13441215,"url":"https://github.com/muskie82/MonoGS","last_synced_at":"2025-03-20T11:37:53.108Z","repository":{"id":225853837,"uuid":"766597813","full_name":"muskie82/MonoGS","owner":"muskie82","description":"[CVPR'24 Highlight \u0026 Best Demo Award] Gaussian Splatting SLAM","archived":false,"fork":false,"pushed_at":"2024-08-07T14:15:29.000Z","size":25814,"stargazers_count":1320,"open_issues_count":63,"forks_count":118,"subscribers_count":17,"default_branch":"main","last_synced_at":"2024-10-15T03:42:29.051Z","etag":null,"topics":["computer-vision","cvpr2024","gaussian-splatting","robotics","slam"],"latest_commit_sha":null,"homepage":"https://rmurai.co.uk/projects/GaussianSplattingSLAM/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/muskie82.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-03T17:46:19.000Z","updated_at":"2024-10-15T01:58:05.000Z","dependencies_parsed_at":"2024-10-15T03:42:36.858Z","dependency_job_id":"cb013f83-a1dd-423a-964f-6adb107cd043","html_url":"https://github.com/muskie82/MonoGS","commit_stats":null,"previous_names":["muskie82/monogs"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muskie82%2FMonoGS","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muskie82%2FMonoGS/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muskie82%2FMonoGS/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/muskie82%2FMonoGS/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/muskie82","download_url":"https://codeload.github.com/muskie82/MonoGS/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221759944,"owners_count":16876323,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["computer-vision","cvpr2024","gaussian-splatting","robotics","slam"],"created_at":"2024-07-31T03:01:31.219Z","updated_at":"2025-03-20T11:37:53.072Z","avatar_url":"https://github.com/muskie82.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"[comment]: \u003c\u003e (# Gaussian Splatting SLAM)\n\n\u003c!-- PROJECT LOGO --\u003e\n\n\u003cp align=\"center\"\u003e\n\n  \u003ch1 align=\"center\"\u003e Gaussian Splatting SLAM\n  \u003c/h1\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://muskie82.github.io/\"\u003e\u003cstrong\u003e*Hidenobu Matsuki\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://rmurai.co.uk/\"\u003e\u003cstrong\u003e*Riku Murai\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.imperial.ac.uk/people/p.kelly/\"\u003e\u003cstrong\u003ePaul H.J. Kelly\u003c/strong\u003e\u003c/a\u003e\n    ·\n    \u003ca href=\"https://www.doc.ic.ac.uk/~ajd/\"\u003e\u003cstrong\u003eAndrew J. Davison\u003c/strong\u003e\u003c/a\u003e\n  \u003c/p\u003e\n  \u003cp align=\"center\"\u003e(* Equal Contribution)\u003c/p\u003e\n\n  \u003ch3 align=\"center\"\u003e CVPR 2024 (Highlight)\u003c/h3\u003e\n\n\n\n[comment]: \u003c\u003e (  \u003ch2 align=\"center\"\u003ePAPER\u003c/h2\u003e)\n  \u003ch3 align=\"center\"\u003e\u003ca href=\"https://arxiv.org/abs/2312.06741\"\u003ePaper\u003c/a\u003e | \u003ca href=\"https://youtu.be/x604ghp9R_Q?si=nYoWr8h2Xh-6L_KN\"\u003eVideo\u003c/a\u003e | \u003ca href=\"https://rmurai.co.uk/projects/GaussianSplattingSLAM/\"\u003eProject Page\u003c/a\u003e\u003c/h3\u003e\n  \u003cdiv align=\"center\"\u003e\u003c/div\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"\"\u003e\n    \u003cimg src=\"./media/teaser.gif\" alt=\"teaser\" width=\"100%\"\u003e\n  \u003c/a\u003e\n  \u003ca href=\"\"\u003e\n    \u003cimg src=\"./media/gui.jpg\" alt=\"gui\" width=\"100%\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\nThis software implements dense SLAM system presented in our paper \u003ca href=\"https://arxiv.org/abs/2312.06741\"\u003eGaussian Splatting SLAM\u003c/a\u003e in CVPR'24.\nThe method demonstrates the first monocular SLAM solely based on 3D Gaussian Splatting (left), which also supports Stereo/RGB-D inputs (middle/right).\n\u003c/p\u003e\n\u003cbr\u003e\n\n# Note\n- In an academic paper, please refer to our work as **Gaussian Splatting SLAM** or **MonoGS** for short (this repo's name) to avoid confusion with other works.\n- Differential Gaussian Rasteriser with camera pose gradient computation is available [here](https://github.com/rmurai0610/diff-gaussian-rasterization-w-pose.git).\n- **[New]** Speed-up version of our code is available in `dev.speedup` branch, It achieves up to 10fps on monocular fr3/office sequence while keeping consistent performance (tested on RTX4090/i9-12900K). The code will be merged into the main branch after further refactoring and testing.\n\n# Getting Started\n## Installation\n```\ngit clone https://github.com/muskie82/MonoGS.git --recursive\ncd MonoGS\n```\nSetup the environment.\n\n```\nconda env create -f environment.yml\nconda activate MonoGS\n```\nDepending on your setup, please change the dependency version of pytorch/cudatoolkit in `environment.yml` by following [this document](https://pytorch.org/get-started/previous-versions/).\n\nOur test setup were:\n- Ubuntu 20.04: `pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.6`\n- Ubuntu 18.04: `pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3`\n\n## Quick Demo\n```\nbash scripts/download_tum.sh\npython slam.py --config configs/mono/tum/fr3_office.yaml\n```\nYou will see a GUI window pops up.\n\n## Downloading Datasets\nRunning the following scripts will automatically download datasets to the `./datasets` folder.\n### TUM-RGBD dataset\n```bash\nbash scripts/download_tum.sh\n```\n\n### Replica dataset\n```bash\nbash scripts/download_replica.sh\n```\n\n### EuRoC MAV dataset\n```bash\nbash scripts/download_euroc.sh\n```\n\n\n\n## Run\n### Monocular\n```bash\npython slam.py --config configs/mono/tum/fr3_office.yaml\n```\n\n### RGB-D\n```bash\npython slam.py --config configs/rgbd/tum/fr3_office.yaml\n```\n\n```bash\npython slam.py --config configs/rgbd/replica/office0.yaml\n```\nOr the single process version as\n```bash\npython slam.py --config configs/rgbd/replica/office0_sp.yaml\n```\n\n\n### Stereo (experimental)\n```bash\npython slam.py --config configs/stereo/euroc/mh02.yaml\n```\n\n## Live demo with Realsense\nFirst, you'll need to install `pyrealsense2`.\nInside the conda environment, run:\n```bash\npip install pyrealsense2\n```\nConnect the realsense camera to the PC on a **USB-3** port and then run:\n```bash\npython slam.py --config configs/live/realsense.yaml\n```\nWe tested the method with [Intel Realsense d455](https://www.mouser.co.uk/new/intel/intel-realsense-depth-camera-d455/). We recommend using a similar global shutter camera for robust camera tracking. Please avoid aggressive camera motion, especially before the initial BA is performed. Check out [the first 15 seconds of our YouTube video](https://youtu.be/x604ghp9R_Q?si=S21HgeVTVfNe0BVL) to see how you should move the camera for initialisation. We recommend to use the code in `dev.speed-up` branch for live demo.\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"\"\u003e\n    \u003cimg src=\"./media/realsense.png\" alt=\"teaser\" width=\"50%\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\n# Evaluation\n\u003c!-- To evaluate the method, please run the SLAM system with `save_results=True` in the base config file. This setting automatically outputs evaluation metrics in wandb and exports log files locally in save_dir. For benchmarking purposes, it is recommended to disable the GUI by setting `use_gui=False` in order to maximise GPU utilisation. For evaluating rendering quality, please set the `eval_rendering=True` flag in the configuration file. --\u003e\nTo evaluate our method, please add `--eval` to the command line argument:\n```bash\npython slam.py --config configs/mono/tum/fr3_office.yaml --eval\n```\nThis flag will automatically run our system in a headless mode, and log the results including the rendering metrics.\n\n# Reproducibility\nThere might be minor differences between the released version and the results in the paper. Please bear in mind that multi-process performance has some randomness due to GPU utilisation.\nWe run all our experiments on an RTX 4090, and the performance may differ when running with a different GPU.\n\n# Acknowledgement\nThis work incorporates many open-source codes. We extend our gratitude to the authors of the software.\n- [3D Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting)\n- [Differential Gaussian Rasterization\n](https://github.com/graphdeco-inria/diff-gaussian-rasterization)\n- [SIBR_viewers](https://gitlab.inria.fr/sibr/sibr_core)\n- [Tiny Gaussian Splatting Viewer](https://github.com/limacv/GaussianSplattingViewer)\n- [Open3D](https://github.com/isl-org/Open3D)\n- [Point-SLAM](https://github.com/eriksandstroem/Point-SLAM)\n\n# License\nMonoGS is released under a **LICENSE.md**. For a list of code dependencies which are not property of the authors of MonoGS, please check **Dependencies.md**.\n\n# Citation\nIf you found this code/work to be useful in your own research, please considering citing the following:\n\n```bibtex\n@inproceedings{Matsuki:Murai:etal:CVPR2024,\n  title={{G}aussian {S}platting {SLAM}},\n  author={Hidenobu Matsuki and Riku Murai and Paul H. J. Kelly and Andrew J. Davison},\n  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},\n  year={2024}\n}\n\n```\n\n\n\n\n\n\n\n\n\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuskie82%2FMonoGS","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmuskie82%2FMonoGS","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmuskie82%2FMonoGS/lists"}