{"id":27864491,"url":"https://github.com/lakonik/mvedit","last_synced_at":"2025-05-04T21:06:13.892Z","repository":{"id":228499371,"uuid":"772231767","full_name":"Lakonik/MVEdit","owner":"Lakonik","description":"3D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generation","archived":false,"fork":false,"pushed_at":"2024-12-25T09:49:46.000Z","size":77050,"stargazers_count":326,"open_issues_count":11,"forks_count":17,"subscribers_count":11,"default_branch":"main","last_synced_at":"2025-05-04T21:06:02.499Z","etag":null,"topics":["3d","diffusion-models","generative-ai","stable-diffusion","texture"],"latest_commit_sha":null,"homepage":"https://lakonik.github.io/3d-adapter","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Lakonik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-03-14T19:36:56.000Z","updated_at":"2025-03-24T02:28:20.000Z","dependencies_parsed_at":"2024-12-25T10:35:49.451Z","dependency_job_id":null,"html_url":"https://github.com/Lakonik/MVEdit","commit_stats":null,"previous_names":["lakonik/mvedit"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakonik%2FMVEdit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakonik%2FMVEdit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakonik%2FMVEdit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Lakonik%2FMVEdit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Lakonik","download_url":"https://codeload.github.com/Lakonik/MVEdit/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252399540,"owners_count":21741672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["3d","diffusion-models","generative-ai","stable-diffusion","texture"],"created_at":"2025-05-04T21:06:13.251Z","updated_at":"2025-05-04T21:06:13.886Z","avatar_url":"https://github.com/Lakonik.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 3D-Adapter\n\nOfficial PyTorch implementation of the papers:\n\n**3D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generation**\n\u003cbr\u003e\n[Hansheng Chen](https://lakonik.github.io/)\u003csup\u003e1\u003c/sup\u003e, \n[Bokui Shen](https://cs.stanford.edu/people/bshen88/)\u003csup\u003e2\u003c/sup\u003e,\n[Yulin Liu](https://liuyulinn.github.io/)\u003csup\u003e3,4\u003c/sup\u003e,\n[Ruoxi Shi](https://rshi.top/)\u003csup\u003e3\u003c/sup\u003e, \n[Linqi Zhou](https://alexzhou907.github.io/)\u003csup\u003e2\u003c/sup\u003e,\n[Connor Z. Lin](https://connorzlin.com/)\u003csup\u003e2\u003c/sup\u003e,\n[Jiayuan Gu](https://pages.ucsd.edu/~ztu/)\u003csup\u003e3\u003c/sup\u003e, \n[Hao Su](https://cseweb.ucsd.edu/~haosu/)\u003csup\u003e3,4\u003c/sup\u003e,\n[Gordon Wetzstein](http://web.stanford.edu/~gordonwz/)\u003csup\u003e1\u003c/sup\u003e, \n[Leonidas Guibas](https://geometry.stanford.edu/member/guibas/)\u003csup\u003e1\u003c/sup\u003e\u003cbr\u003e\n\u003csup\u003e1\u003c/sup\u003eStanford University, \u003csup\u003e2\u003c/sup\u003eApparate Labs, \u003csup\u003e3\u003c/sup\u003eUCSD, \u003csup\u003e4\u003c/sup\u003eHillbot\n\u003cbr\u003e\n[[Project page](https://lakonik.github.io/3d-adapter)] [[🤗Demo](https://huggingface.co/spaces/Lakonik/3D-Adapter)] [[Paper](https://arxiv.org/abs/2410.18974)]\n\n**Generic 3D Diffusion Adapter Using Controlled Multi-View Editing**\n\u003cbr\u003e\n[Hansheng Chen](https://lakonik.github.io/)\u003csup\u003e1\u003c/sup\u003e, \n[Ruoxi Shi](https://rshi.top/)\u003csup\u003e2\u003c/sup\u003e, \n[Yulin Liu](https://liuyulinn.github.io/)\u003csup\u003e2\u003c/sup\u003e, \n[Bokui Shen](https://cs.stanford.edu/people/bshen88/)\u003csup\u003e3\u003c/sup\u003e,\n[Jiayuan Gu](https://pages.ucsd.edu/~ztu/)\u003csup\u003e2\u003c/sup\u003e, \n[Gordon Wetzstein](http://web.stanford.edu/~gordonwz/)\u003csup\u003e1\u003c/sup\u003e, \n[Hao Su](https://cseweb.ucsd.edu/~haosu/)\u003csup\u003e2\u003c/sup\u003e, \n[Leonidas Guibas](https://geometry.stanford.edu/member/guibas/)\u003csup\u003e1\u003c/sup\u003e\u003cbr\u003e\n\u003csup\u003e1\u003c/sup\u003eStanford University, \u003csup\u003e2\u003c/sup\u003eUCSD, \u003csup\u003e3\u003c/sup\u003eApparate Labs\n\u003cbr\u003e\n[[Project page](https://lakonik.github.io/mvedit)] [[🤗Demo](https://huggingface.co/spaces/Lakonik/MVEdit)] [[Paper](https://arxiv.org/abs/2403.12032)]\n\n\nhttps://github.com/user-attachments/assets/6cba3a92-04fe-46ee-88ca-e6dfe5443c36\n\n\n## Todos\n\n- [ ] Release GRM-based 3D-Adapters (unfortunately, we cannot release these models before the official release of [GRM](https://github.com/justimyhxu/GRM))\n\n## Installation\n\nThe code has been tested in the environment described as follows:\n\n- Linux (tested on Ubuntu 20 and above)\n- [CUDA Toolkit](https://developer.nvidia.com/cuda-toolkit-archive) 11.8 and above\n- [PyTorch](https://pytorch.org/get-started/previous-versions/) 2.1 and above\n- FFmpeg, x264 (optional, for exporting videos)\n\nOther dependencies can be installed via `pip install -r requirements.txt`. \n\nAn example of installation commands is shown below (you may change the CUDA version yourself):\n\n```bash\n# Export the PATH of CUDA toolkit\nexport PATH=/usr/local/cuda-12.1/bin:$PATH\nexport LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64:$LD_LIBRARY_PATH\n\n# Create conda environment\nconda create -y -n mvedit python=3.10 numpy=1.26 ninja\nconda activate mvedit\n\n# Install FFmpeg (optional)\nconda install -c conda-forge ffmpeg x264\n\n# Install PyTorch\nconda install pytorch==2.1.2 torchvision==0.16.2 pytorch-cuda=12.1 -c pytorch -c nvidia\n\n# Clone this repo and install other dependencies\ngit clone https://github.com/Lakonik/MVEdit \u0026\u0026 cd MVEdit\npip install -r requirements.txt\n```\n\nThis codebase also works on Windows systems if the environment is configured correctly. Certain packages (e.g., tiny-cuda-nn) may require adjustments for installation on Windows.\n\n## Inference\n\nWe recommend using the Gradio Web UI and its APIs for inference. A GPU with at least 24GB of VRAM is required to run the Web UI.\n\n### Web UI\n\nRun the following command to start the Web UI:\n\n```bash\npython app.py --unload-models\n```\n\nThe Web UI will be available at [http://localhost:7860](http://localhost:7860). If you add the `--share` flag, a temporary public URL will be generated for you to share the Web UI with others.\n\nAll models will be automatically loaded on demand. The first run will take a very long time to download the models. Check your network connection to GitHub, Google Drive and Hugging Face if the download fails.\n\nTo view other options, run:\n\n```bash\npython app.py -h\n```\n\n### API\n\nAfter starting the Web UI, the API docs will be available at [http://localhost:7860/?view=api](http://localhost:7860/?view=api).\n\nAlternatively, you can access the APIs provided by the official demo ([https://mvedit.hanshengchen.com/?view=api](https://mvedit.hanshengchen.com/?view=api)), including APIs for GRM Adapters.\n\nThe API docs are automatically generated by Gradio, and the data types and default values may be incorrect. Please use the default values in the Web UI as a reference.\n\n**Please refer to our [examples](scripts/) for API usage with python.**\n\n## Training\n\nOptimization-based 3D-Adapters (a.k.a. MVEdit adapters) adopt only off-the-shelf models and require no further training.\n\nThe training code for GRM-based 3D-Adapters will be released after the official release of [GRM](https://github.com/justimyhxu/GRM). \n\n## Acknowledgements\n\nThis codebase is built upon the following repositories:\n- Base library modified from [SSDNeRF](https://github.com/Lakonik/SSDNeRF)\n- NeRF renderer and DMTet modified from [Stable-DreamFusion](https://github.com/ashawkey/stable-dreamfusion)\n- Gaussian Splatting renderer modified from [3DGS](https://github.com/graphdeco-inria/gaussian-splatting) and [Differential Gaussian Rasterization](https://github.com/ashawkey/diff-gaussian-rasterization)\n- Mesh I/O modified from [DreamGaussian](https://github.com/dreamgaussian/dreamgaussian)\n- [GRM](https://github.com/justimyhxu/GRM) for Gaussian reconstruction\n- [Zero123++](https://github.com/SUDO-AI-3D/zero123plus) for image-to-3D initialization\n- [IP-Adapter](https://github.com/tencent-ailab/IP-Adapter) for extra conditioning\n- [TRACER](https://github.com/Karel911/TRACER) for background removal\n- [LoFTR](https://github.com/zju3dv/LoFTR) for pose estimation in image-to-3D\n- [Omnidata](https://github.com/EPFL-VILAB/omnidata) for normal prediction in image-to-3D\n- [Image Packer](https://github.com/theFroh/imagepacker) for mesh preprocessing\n\n## Citation\n\n```bibtex\n@misc{3dadapter2024,\n    title={3D-Adapter: Geometry-Consistent Multi-View Diffusion for High-Quality 3D Generation},\n    author={Hansheng Chen and Bokui Shen and Yulin Liu and Ruoxi Shi and Linqi Zhou and Connor Z. Lin and Jiayuan Gu and Hao Su and Gordon Wetzstein and Leonidas Guibas},\n    year={2024},\n    eprint={2410.18974},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV},\n    url={https://arxiv.org/abs/2410.18974},\n}\n\n@misc{mvedit2024,\n    title={Generic 3D Diffusion Adapter Using Controlled Multi-View Editing},\n    author={Hansheng Chen and Ruoxi Shi and Yulin Liu and Bokui Shen and Jiayuan Gu and Gordon Wetzstein and Hao Su and Leonidas Guibas},\n    year={2024},\n    eprint={2403.12032},\n    archivePrefix={arXiv},\n    primaryClass={cs.CV}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flakonik%2Fmvedit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flakonik%2Fmvedit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flakonik%2Fmvedit/lists"}