{"id":13958438,"url":"https://github.com/TencentARC/InstantMesh","last_synced_at":"2025-07-21T00:30:47.277Z","repository":{"id":232581508,"uuid":"784696015","full_name":"TencentARC/InstantMesh","owner":"TencentARC","description":"InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models","archived":false,"fork":false,"pushed_at":"2025-01-03T09:09:13.000Z","size":34356,"stargazers_count":3863,"open_issues_count":112,"forks_count":423,"subscribers_count":49,"default_branch":"main","last_synced_at":"2025-05-23T21:00:47.612Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TencentARC.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-04-10T11:22:38.000Z","updated_at":"2025-05-23T13:51:34.000Z","dependencies_parsed_at":"2024-04-21T21:51:20.733Z","dependency_job_id":"2617abbb-dda0-40c3-b240-30262ab62dd3","html_url":"https://github.com/TencentARC/InstantMesh","commit_stats":null,"previous_names":["tencentarc/instantmesh"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/TencentARC/InstantMesh","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FInstantMesh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FInstantMesh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FInstantMesh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FInstantMesh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TencentARC","download_url":"https://codeload.github.com/TencentARC/InstantMesh/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TencentARC%2FInstantMesh/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266221246,"owners_count":23894964,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-08T13:01:35.387Z","updated_at":"2025-07-21T00:30:47.270Z","avatar_url":"https://github.com/TencentARC.png","language":"Python","funding_links":[],"categories":["其他_机器视觉","Python","排行榜 [2025-03-18]","AI \u0026 Machine Learning for CG"],"sub_categories":["网络服务_其他","3D Generation"],"readme":"\u003cdiv align=\"center\"\u003e\n  \n# InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models\n\n\u003ca href=\"https://arxiv.org/abs/2404.07191\"\u003e\u003cimg src=\"https://img.shields.io/badge/ArXiv-2404.07191-brightgreen\"\u003e\u003c/a\u003e \n\u003ca href=\"https://huggingface.co/TencentARC/InstantMesh\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%A4%97%20Model_Card-Huggingface-orange\"\u003e\u003c/a\u003e \n\u003ca href=\"https://huggingface.co/spaces/TencentARC/InstantMesh\"\u003e\u003cimg src=\"https://img.shields.io/badge/%F0%9F%A4%97%20Gradio%20Demo-Huggingface-orange\"\u003e\u003c/a\u003e \u003cbr\u003e\n\u003ca href=\"https://replicate.com/camenduru/instantmesh\"\u003e\u003cimg src=\"https://img.shields.io/badge/Demo-Replicate-blue\"\u003e\u003c/a\u003e\n\u003ca href=\"https://colab.research.google.com/github/camenduru/InstantMesh-jupyter/blob/main/InstantMesh_jupyter.ipynb\"\u003e\u003cimg src=\"https://colab.research.google.com/assets/colab-badge.svg\"\u003e\u003c/a\u003e\n\u003ca href=\"https://github.com/jtydhr88/ComfyUI-InstantMesh\"\u003e\u003cimg src=\"https://img.shields.io/badge/Demo-ComfyUI-8A2BE2\"\u003e\u003c/a\u003e\n\n\u003c/div\u003e\n\n---\n\nThis repo is the official implementation of InstantMesh, a feed-forward framework for efficient 3D mesh generation from a single image based on the LRM/Instant3D architecture.\n\nhttps://github.com/TencentARC/InstantMesh/assets/20635237/dab3511e-e7c6-4c0b-bab7-15772045c47d\n\n# 🚩 Features and Todo List\n- [x] 🔥🔥 Release Zero123++ fine-tuning code. \n- [x] 🔥🔥 Support for running gradio demo on two GPUs to save memory.\n- [x] 🔥🔥 Support for running demo with docker. Please refer to the [docker](docker/) directory.\n- [x] Release inference and training code.\n- [x] Release model weights.\n- [x] Release huggingface gradio demo. Please try it at [demo](https://huggingface.co/spaces/TencentARC/InstantMesh) link.\n- [ ] Add support for more multi-view diffusion models.\n\n# ⚙️ Dependencies and Installation\n\nWe recommend using `Python\u003e=3.10`, `PyTorch\u003e=2.1.0`, and `CUDA\u003e=12.1`.\n```bash\nconda create --name instantmesh python=3.10\nconda activate instantmesh\npip install -U pip\n\n# Ensure Ninja is installed\nconda install Ninja\n\n# Install the correct version of CUDA\nconda install cuda -c nvidia/label/cuda-12.1.0\n\n# Install PyTorch and xformers\n# You may need to install another xformers version if you use a different PyTorch version\npip install torch==2.1.0 torchvision==0.16.0 torchaudio==2.1.0 --index-url https://download.pytorch.org/whl/cu121\npip install xformers==0.0.22.post7\n\n# Install other requirements\npip install -r requirements.txt\n```\n\n# 💫 How to Use\n\n## Download the models\n\nWe provide 4 sparse-view reconstruction model variants and a customized Zero123++ UNet for white-background image generation in the [model card](https://huggingface.co/TencentARC/InstantMesh).\n\nOur inference script will download the models automatically. Alternatively, you can manually download the models and put them under the `ckpts/` directory.\n\nBy default, we use the `instant-mesh-large` reconstruction model variant.\n\n## Start a local gradio demo\n\nTo start a gradio demo in your local machine, simply run:\n```bash\npython app.py\n```\n\nIf you have multiple GPUs in your machine, the demo app will run on two GPUs automatically to save memory. You can also force it to run on a single GPU:\n```bash\nCUDA_VISIBLE_DEVICES=0 python app.py\n```\n\nAlternatively, you can run the demo with docker. Please follow the instructions in the [docker](docker/) directory.\n\n## Running with command line\n\nTo generate 3D meshes from images via command line, simply run:\n```bash\npython run.py configs/instant-mesh-large.yaml examples/hatsune_miku.png --save_video\n```\n\nWe use [rembg](https://github.com/danielgatis/rembg) to segment the foreground object. If the input image already has an alpha mask, please specify the `no_rembg` flag:\n```bash\npython run.py configs/instant-mesh-large.yaml examples/hatsune_miku.png --save_video --no_rembg\n```\n\nBy default, our script exports a `.obj` mesh with vertex colors, please specify the `--export_texmap` flag if you hope to export a mesh with a texture map instead (this will cost longer time):\n```bash\npython run.py configs/instant-mesh-large.yaml examples/hatsune_miku.png --save_video --export_texmap\n```\n\nPlease use a different `.yaml` config file in the [configs](./configs) directory if you hope to use other reconstruction model variants. For example, using the `instant-nerf-large` model for generation:\n```bash\npython run.py configs/instant-nerf-large.yaml examples/hatsune_miku.png --save_video\n```\n**Note:** When using the `NeRF` model variants for image-to-3D generation, exporting a mesh with texture map by specifying `--export_texmap` may cost long time in the UV unwarping step since the default iso-surface extraction resolution is `256`. You can set a lower iso-surface extraction resolution in the config file.\n\n# 💻 Training\n\nWe provide our training code to facilitate future research. But we cannot provide the training dataset due to its size. Please refer to our [dataloader](src/data/objaverse.py) for more details.\n\nTo train the sparse-view reconstruction models, please run:\n```bash\n# Training on NeRF representation\npython train.py --base configs/instant-nerf-large-train.yaml --gpus 0,1,2,3,4,5,6,7 --num_nodes 1\n\n# Training on Mesh representation\npython train.py --base configs/instant-mesh-large-train.yaml --gpus 0,1,2,3,4,5,6,7 --num_nodes 1\n```\n\nWe also provide our Zero123++ fine-tuning code since it is frequently requested. The running command is:\n```bash\npython train.py --base configs/zero123plus-finetune.yaml --gpus 0,1,2,3,4,5,6,7 --num_nodes 1\n```\n\n# :books: Citation\n\nIf you find our work useful for your research or applications, please cite using this BibTeX:\n\n```BibTeX\n@article{xu2024instantmesh,\n  title={InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models},\n  author={Xu, Jiale and Cheng, Weihao and Gao, Yiming and Wang, Xintao and Gao, Shenghua and Shan, Ying},\n  journal={arXiv preprint arXiv:2404.07191},\n  year={2024}\n}\n```\n\n# 🤗 Acknowledgements\n\nWe thank the authors of the following projects for their excellent contributions to 3D generative AI!\n\n- [Zero123++](https://github.com/SUDO-AI-3D/zero123plus)\n- [OpenLRM](https://github.com/3DTopia/OpenLRM)\n- [FlexiCubes](https://github.com/nv-tlabs/FlexiCubes)\n- [Instant3D](https://instant-3d.github.io/)\n\nThank [@camenduru](https://github.com/camenduru) for implementing [Replicate Demo](https://replicate.com/camenduru/instantmesh) and [Colab Demo](https://colab.research.google.com/github/camenduru/InstantMesh-jupyter/blob/main/InstantMesh_jupyter.ipynb)!  \nThank [@jtydhr88](https://github.com/jtydhr88) for implementing [ComfyUI support](https://github.com/jtydhr88/ComfyUI-InstantMesh)!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTencentARC%2FInstantMesh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FTencentARC%2FInstantMesh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FTencentARC%2FInstantMesh/lists"}