Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/AiuniAI/Unique3D
Official implementation of Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image
https://github.com/AiuniAI/Unique3D
3d-aigc aigc image-to-3d
Last synced: 6 days ago
JSON representation
Official implementation of Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image
- Host: GitHub
- URL: https://github.com/AiuniAI/Unique3D
- Owner: AiuniAI
- License: mit
- Created: 2024-05-30T18:37:51.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2024-09-18T08:04:01.000Z (2 months ago)
- Last Synced: 2024-09-18T10:38:24.591Z (2 months ago)
- Topics: 3d-aigc, aigc, image-to-3d
- Language: Python
- Homepage: https://wukailu.github.io/Unique3D/
- Size: 9.61 MB
- Stars: 2,855
- Watchers: 36
- Forks: 221
- Open Issues: 52
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- ai-game-devtools - Unique3D - Quality and Efficient 3D Mesh Generation from a Single Image. |[arXiv](https://arxiv.org/abs/2405.20343) | | 3D | (<span id="model">3D Model</span> / <span id="tool">Tool (AI LLM)</span>)
- StarryDivineSky - AiuniAI/Unique3D
- awesome-llm-projects - Unique3D - Quality and Efficient 3D Mesh Generation from a Single Image. (Projects / π§Έ 3D Model)
README
**[δΈζηζ¬](README_zh.md)**
**[ζ₯ζ¬θͺη](README_jp.md)**
# Unique3D
Official implementation of Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image.
[Kailu Wu](https://scholar.google.com/citations?user=VTU0gysAAAAJ&hl=zh-CN&oi=ao), [Fangfu Liu](https://liuff19.github.io/), Zhihan Cai, Runjie Yan, Hanyang Wang, Yating Hu, [Yueqi Duan](https://duanyueqi.github.io/), [Kaisheng Ma](https://group.iiis.tsinghua.edu.cn/~maks/)
## [Paper](https://arxiv.org/abs/2405.20343) | [Project page](https://wukailu.github.io/Unique3D/) | [Huggingface Demo](https://huggingface.co/spaces/Wuvin/Unique3D) | [Gradio Demo](http://unique3d.demo.avar.cn/) | [Online Demo](https://www.aiuni.ai/)
* Demo inference speed: Gradio Demo > Huggingface Demo > Huggingface Demo2 > Online Demo
**If the Gradio Demo is overcrowded or fails to produce stable results, you can use the Online Demo [aiuni.ai](https://www.aiuni.ai/), which is free to try (get the registration invitation code Join Discord: https://discord.gg/aiuni). However, the Online Demo is slightly different from the Gradio Demo, in that the inference speed is slower, but the generation is much more stable.**
High-fidelity and diverse textured meshes generated by Unique3D from single-view wild images in 30 seconds.
## More features
The repo is still being under construction, thanks for your patience.
- [x] Upload weights.
- [x] Local gradio demo.
- [x] Detailed tutorial.
- [x] Huggingface demo.
- [ ] Detailed local demo.
- [x] Comfyui support.
- [x] Windows support.
- [x] Docker support.
- [ ] More stable reconstruction with normal.
- [ ] Training code release.
## Preparation for inference
* [Detailed linux installation guide](Installation.md).
### Linux System Setup.
Adapted for Ubuntu 22.04.4 LTS and CUDA 12.1.
```angular2html
conda create -n unique3d python=3.11
conda activate unique3d
pip install ninja
pip install diffusers==0.27.2
pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.3.1/index.html
pip install -r requirements.txt
```
[oak-barry](https://github.com/oak-barry) provide another setup script for torch210+cu121 at [here](https://github.com/oak-barry/Unique3D).
### Windows Setup.
* Thank you very much `jtydhr88` for the windows installation method! See [issues/15](https://github.com/AiuniAI/Unique3D/issues/15).
According to [issues/15](https://github.com/AiuniAI/Unique3D/issues/15), implemented a bat script to run the commands, so you can:
1. Might still require Visual Studio Build Tools, you can find it from [Visual Studio Build Tools](https://visualstudio.microsoft.com/downloads/?q=build+tools).
2. Create conda env and activate it
1. `conda create -n unique3d-py311 python=3.11`
2. `conda activate unique3d-py311`
3. download [triton whl](https://huggingface.co/madbuda/triton-windows-builds/resolve/main/triton-2.1.0-cp311-cp311-win_amd64.whl) for py311, and put it into this project.
4. run **install_windows_win_py311_cu121.bat**
5. answer y while asking you uninstall onnxruntime and onnxruntime-gpu
6. create the output folder **tmp\gradio** under the driver root, such as F:\tmp\gradio for me.
7. python app/gradio_local.py --port 7860
More details prefer to [issues/15](https://github.com/AiuniAI/Unique3D/issues/15).
### Interactive inference: run your local gradio demo.
1. Download the weights from [huggingface spaces](https://huggingface.co/spaces/Wuvin/Unique3D/tree/main/ckpt) or [Tsinghua Cloud Drive](https://cloud.tsinghua.edu.cn/d/319762ec478d46c8bdf7/), and extract it to `ckpt/*`.
```
Unique3D
βββckpt
βββ controlnet-tile/
βββ image2normal/
βββ img2mvimg/
βββ realesrgan-x4.onnx
βββ v1-inference.yaml
```
2. Run the interactive inference locally.
```bash
python app/gradio_local.py --port 7860
```
## ComfyUI Support
Thanks for the [ComfyUI-Unique3D](https://github.com/jtydhr88/ComfyUI-Unique3D) implementation from [jtydhr88](https://github.com/jtydhr88)!
## Tips to get better results
**Important: Because the mesh is normalized by the longest edge of xyz during training, it is desirable that the input image needs to contain the longest edge of the object during inference, or else you may get erroneously squashed results.**
1. Unique3D is sensitive to the facing direction of input images. Due to the distribution of the training data, orthographic front-facing images with a rest pose always lead to good reconstructions.
2. Images with occlusions will cause worse reconstructions, since four views cannot cover the complete object. Images with fewer occlusions lead to better results.
3. Pass an image with as high a resolution as possible to the input when resolution is a factor.
## Acknowledgement
We have intensively borrowed code from the following repositories. Many thanks to the authors for sharing their code.
- [Stable Diffusion](https://github.com/CompVis/stable-diffusion)
- [Wonder3d](https://github.com/xxlong0/Wonder3D)
- [Zero123Plus](https://github.com/SUDO-AI-3D/zero123plus)
- [Continues Remeshing](https://github.com/Profactor/continuous-remeshing)
- [Depth from Normals](https://github.com/YertleTurtleGit/depth-from-normals)
## Collaborations
Our mission is to create a 4D generative model with 3D concepts. This is just our first step, and the road ahead is still long, but we are confident. We warmly invite you to join the discussion and explore potential collaborations in any capacity. **If you're interested in connecting or partnering with us, please don't hesitate to reach out via email ([email protected])**.
- Follow us on twitter for the latest updates: https://x.com/aiuni_ai
- Join AIGC 3D/4D generation community on discord: https://discord.gg/aiuni
- Research collaboration, please contact: [email protected]
## Citation
If you found Unique3D helpful, please cite our report:
```bibtex
@misc{wu2024unique3d,
title={Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image},
author={Kailu Wu and Fangfu Liu and Zhihan Cai and Runjie Yan and Hanyang Wang and Yating Hu and Yueqi Duan and Kaisheng Ma},
year={2024},
eprint={2405.20343},
archivePrefix={arXiv},
primaryClass={cs.CV}
}
```