{"id":18914161,"url":"https://github.com/buaacyw/gaussianeditor","last_synced_at":"2025-05-16T08:03:33.103Z","repository":{"id":209330823,"uuid":"723758860","full_name":"buaacyw/GaussianEditor","owner":"buaacyw","description":"[CVPR 2024] GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting","archived":false,"fork":false,"pushed_at":"2024-02-27T11:35:48.000Z","size":7010,"stargazers_count":1250,"open_issues_count":44,"forks_count":62,"subscribers_count":27,"default_branch":"master","last_synced_at":"2025-04-08T19:22:57.148Z","etag":null,"topics":["3d","cvpr2024","generative-ai"],"latest_commit_sha":null,"homepage":"https://buaacyw.github.io/gaussian-editor/","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/buaacyw.png","metadata":{"files":{"readme":"readme.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-26T17:16:48.000Z","updated_at":"2025-04-08T16:11:48.000Z","dependencies_parsed_at":"2024-02-27T12:50:29.880Z","dependency_job_id":null,"html_url":"https://github.com/buaacyw/GaussianEditor","commit_stats":null,"previous_names":["buaacyw/gaussianeditor"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buaacyw%2FGaussianEditor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buaacyw%2FGaussianEditor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buaacyw%2FGaussianEditor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/buaacyw%2FGaussianEditor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/buaacyw","download_url":"https://codeload.github.com/buaacyw/GaussianEditor/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254493381,"owners_count":22080126,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","cvpr2024","generative-ai"],"created_at":"2024-11-08T10:10:25.675Z","updated_at":"2025-05-16T08:03:33.083Z","avatar_url":"https://github.com/buaacyw.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/figs/logo.png\" align=\"center\" width=\"50%\"\u003e\n  \n  \u003ch3 align=\"center\"\u003e\u003cstrong\u003e[CVPR 2024] GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting\u003c/strong\u003e\u003c/h3\u003e\n\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://buaacyw.github.io/\"\u003eYiwen Chen\u003c/a\u003e\u003csup\u003e*1,2\u003c/sup\u003e,\u003c/span\u003e\n    \u003ca href=\"https://scholar.google.com/citations?user=2pbka1gAAAAJ\u0026hl=en\"\u003eZilong Chen\u003c/a\u003e\u003csup\u003e*3\u003c/sup\u003e,\u003c/span\u003e\n    \u003ca href=\"https://icoz69.github.io/\"\u003eChi Zhang\u003c/a\u003e\u003csup\u003e2\u003c/sup\u003e,\n    \u003ca href=\"https://scholar.google.com/citations?user=bKG4Un8AAAAJ\u0026hl=en\"\u003eFeng Wang\u003c/a\u003e\u003csup\u003e3\u003c/sup\u003e,\n    \u003ca href=\"https://www.researchgate.net/scientific-contributions/Xiaofeng-Yang-2185243877\"\u003eXiaofeng Yang\u003c/a\u003e\u003csup\u003e2\u003c/sup\u003e,    \u003cbr\u003e\n    \u003ca href=\"https://yikaiw.github.io/\"\u003eYikai Wang\u003c/a\u003e\u003csup\u003e3\u003c/sup\u003e,\n    \u003ca href=\"https://caizhongang.github.io/\"\u003eZhongang Cai\u003c/a\u003e\u003csup\u003e4\u003c/sup\u003e\n    \u003ca href=\"https://scholar.google.com.hk/citations?user=jZH2IPYAAAAJ\u0026hl=en\"\u003eLei Yang\u003c/a\u003e\u003csup\u003e4\u003c/sup\u003e\n    \u003ca href=\"https://sites.google.com/site/thuliuhuaping\"\u003eHuaping Liu\u003c/a\u003e\u003csup\u003e3\u003c/sup\u003e\n    \u003ca href=\"https://guosheng.github.io/\"\u003eGuosheng Lin\u003c/a\u003e\u003csup\u003e**1,2\u003c/sup\u003e\n    \u003cbr\u003e\n    \u003csup\u003e*\u003c/sup\u003eEqual contribution.\n    \u003csup\u003e**\u003c/sup\u003eCorresponding author.\n    \u003cbr\u003e\n    \u003csup\u003e1\u003c/sup\u003eS-Lab, Nanyang Technological University,\n    \u003cbr\u003e\n\u003csup\u003e2\u003c/sup\u003eSchool of Computer Science and Engineering, Nanyang Technological University,\n    \u003cbr\u003e\n    \u003csup\u003e3\u003c/sup\u003eDepartment of Computer Science and Technology, Tsinghua University,\n    \u003csup\u003e4\u003c/sup\u003eSenseTime Research,\n\n\u003c/p\u003e\n\n\u003cdiv align=\"center\"\u003e\n\n\u003ca href='https://arxiv.org/abs/2311.14521'\u003e\u003cimg src='https://img.shields.io/badge/arXiv-2311.14521-b31b1b.svg'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n \u003ca href='https://buaacyw.github.io/gaussian-editor/'\u003e\u003cimg src='https://img.shields.io/badge/Project-Page-Green'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n \u003ca href='https://www.youtube.com/watch?v=TdZIICSFqsU\u0026ab_channel=YiwenChen'\u003e\u003cimg src='https://img.shields.io/badge/Youtube-Video-red'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n \u003ca href='https://github.com/buaacyw/GaussianEditor/blob/master/LICENSE.txt'\u003e\u003cimg src='https://img.shields.io/badge/License-SLab-blue'\u003e\u003c/a\u003e \u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\u0026nbsp;\n\n\u003c/div\u003e\n\n\n## Demo Videos\n\u003cdetails open\u003e\n  \u003csummary\u003eSwift and controllable 3D editing with only 2-7 minutes.\u003c/summary\u003e\n\nhttps://github.com/buaacyw/GaussianEditor/assets/52091468/10740174-3208-4408-b519-23f58604339e\n\nhttps://github.com/buaacyw/GaussianEditor/assets/52091468/44797174-0242-4c82-a383-2d7b3d4fd693\n\n\nhttps://github.com/buaacyw/GaussianEditor/assets/52091468/18dd3ef2-4066-428a-918d-c4fe673d0af8\n\u003c/details\u003e\n\n## Release\n- [12/5] Docker support. Great thanks to [jhuangBU](https://github.com/jhuangBU). For windows, you can try [this guide](https://github.com/buaacyw/GaussianEditor/issues/9) and [this guide](https://github.com/buaacyw/GaussianEditor/issues/14).\n- [11/29] Release segmentation confidence score scaler. You can now scale the threshold of semantic tracing masks. \n- [11/27] 🔥 We released **GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting** and beta version of GaussianEditing WebUI.\n\n## Contents\n- [Demo Videos](#demo-videos)\n- [Release](#release)\n- [Contents](#contents)\n- [Installation](#installation)\n- [WebUI Guide](#webui-guide)\n- [How to achieve better result](#how-to-achieve-better-result)\n- [Command Line](#command-line)\n- [TODO](#todo)\n- [FAQ](#faq)\n\n## Installation\nOur environment has been tested on Ubuntu 22, CUDA 11.8 with 3090, A5000 and A6000.\n1. Clone our repo and create conda environment\n```\ngit clone https://github.com/buaacyw/GaussianEditor.git \u0026\u0026 cd GaussianEditor\n\n# (Option one) Install by conda\nconda env create -f environment.yaml\n\n# (Option two) You can also install by pip\n# CUDA version 11.7\npip install torch==2.0.1+cu117 torchvision==0.15.2+cu117 --extra-index-url https://download.pytorch.org/whl/cu117\n# CUDA version 11.8\npip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --extra-index-url https://download.pytorch.org/whl/cu118\npip install -r requirements.txt\n\n# (Option three) If the below two options fail, please try this:\n# For CUDA 11.8\nbash install.sh\n```\n\n2. (Optional) Install our forked viser [Required by WebUI)\n```\nmkdir extern \u0026\u0026 cd extern\ngit clone https://github.com/heheyas/viser \npip install -e viser\ncd ..\n```\n\n3. (Optional) Download Wonder3D checkpoints [Required by \u003cb\u003eAdd\u003c/b\u003e]\n```bash\nsh download_wonder3d.sh\n```\n\n## WebUI Guide\nPlease be aware that our WebUI is currently in a beta version. Powered by [Viser](https://github.com/nerfstudio-project/viser/tree/main), you can use our WebUI even if you are limited to remote server. For details, please follow [WebUI Guide](https://github.com/buaacyw/GaussianEditor/blob/master/docs/webui.md).\n\n## How to achieve better result\n\nThe demand for 3D editing is very diverse. For instance, if you only want to change textures and materials or significantly modify geometry, it's clear that a one-size-fits-all hyperparameter won't work. Therefore, we cannot provide a default hyperparameter setting that works effectively in all scenarios. Therefore, if your results do not meet expectations, please refer to our [hyperparameter tuning](https://github.com/buaacyw/GaussianEditor/blob/master/docs/hyperparameter.md) document. In it, we detail the function of each hyperparameter and advise on which parameters to adjust when you encounter specific issues. \n\n## Command Line\nWe also provide a command line version of GaussianEditor. Like WebUI, you need to specify your path to the pretrained Gaussians and COLMAP outputs as mentioned in [here](https://github.com/buaacyw/GaussianEditor/blob/1fa96851c132258e0547ba73372f37cff83c92c3/docs/webui.md?plain=1#L20).\nPlease check scripts in `sciprt` folder. Simply change `data.source` to your COLMAP output directory and \n`system.gs_source` to your pretrained Gaussians and run our demo scripts.\n\n\n## TODO\n\nThe repo is still being under construction, thanks for your patience. \n- [x] Tutorial for hyperparameter tuning.\n- [x] Step-by-step tutorial for WebUI .\n- [x] Realised WebUI beta version and GaussianEditor.\n\n## FAQ\n\n- Bad result for \u003cb\u003eEdit\u003c/b\u003e. We are using [InstructPix2Pix](https://github.com/timothybrooks/instruct-pix2pix) to generate edited 2D images as editing guidance.\nUnfortunately, InstructPix2Pix only works on limited prompts, please first try [here](https://huggingface.co/spaces/timbrooks/instruct-pix2pix) if you are not sure whether your text prompts work.\n- Bad result for \u003cb\u003eAdd\u003c/b\u003e. We use [ControlNet-Inpainting](https://github.com/lllyasviel/ControlNet) to first generate 2D inpainting and then transfer it into 3D. Also it doesn't work for bad prompts. Please try to enlarge your inpainting mask and try more seeds.\n- Bad result for \u003cb\u003eSegmentation\u003c/b\u003e. Try scale the segmentation threshold, which changes the confidence score for segmentation.\n- Missing weights for DPT. Please read this [issue](https://github.com/buaacyw/GaussianEditor/issues/10)\n\n## Acknowledgement\n\nOur code is based on these wonderful repos:\n\n* [Gaussian Splatting](https://github.com/graphdeco-inria/gaussian-splatting)\n* [Wonder3D](https://github.com/xxlong0/Wonder3D)\n* [Threestudio](https://github.com/threestudio-project/threestudio)\n* [Viser](https://github.com/nerfstudio-project/viser)\n* [InstructNerf2Nerf](https://github.com/ayaanzhaque/instruct-nerf2nerf)\n* [InstructPix2Pix](https://github.com/timothybrooks/instruct-pix2pix)\n* [Controlnet](https://github.com/lllyasviel/ControlNet)\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuaacyw%2Fgaussianeditor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbuaacyw%2Fgaussianeditor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbuaacyw%2Fgaussianeditor/lists"}