https://github.com/cvlab-kaist/3DFuse

Official implementation of "Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation"
https://github.com/cvlab-kaist/3DFuse

Last synced: 2 months ago
JSON representation

Official implementation of "Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation"

• Host: GitHub
• URL: https://github.com/cvlab-kaist/3DFuse
• Owner: cvlab-kaist
• Created: 2023-03-13T14:35:37.000Z (almost 2 years ago)
• Default Branch: main
• Last Pushed: 2024-02-09T07:44:07.000Z (12 months ago)
• Last Synced: 2024-11-07T04:40:58.976Z (3 months ago)
• Language: Python
• Homepage:
• Size: 177 MB
• Stars: 716
• Watchers: 36
• Forks: 42
• Open Issues: 12
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation




















This is official implementation of the paper "Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation". The last column of each example is our result.


## ⚡️News

**❗️2023.04.10:**  We've opened the [**HuggingFace Demo**](https://huggingface.co/spaces/jyseo/3DFuse)! Also, we fixed minor issues, including the seed not being fixed.

**❗️2023.03.31:**  We found that we typed an incorrect version of the model for point cloud inference. **The fixed commit produces much better results.**

## Introduction

 

We introduce 3DFuse, a novel framework that incorporates 3D awareness into pretrained 2D diffusion models, enhancing the robustness and 3D consistency of score distillation-based methods. For more details, please visit our [project page](https://ku-cvlab.github.io/3DFuse/)!

## 🔥TODO

- [x] 3D Generation/Gradio Demo Code

- [x] HuggingFace🤗 Demo Release

- [ ] Colab Demo Release

- [ ] Mesh Converting Code

## Installation

Please follow [installation](INSTALL.md).

## Interactive Gradio App

### for Text-to-3D / Image-to-3D

Enter your own prompt and enjoy! With this gradio app, you can preview the point cloud before 3D generation and determine the desired shape.

```

python gradio_app.py

# or python gradio_app.py --share

```





## Text-to-3D Generation

After modifying the `run.sh` file with the desired prompt and hyperparameters, please execute the following command:

```

sh run.sh

```

## Acknowledgement

We would like to acknowledge the contributions of public projects, including [SJC](https://github.com/pals-ttic/sjc/) and [ControlNet](https://github.com/lllyasviel/ControlNet) whose code has been utilized in this repository.

## Citation

```

@article{seo2023let,

  title={Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation},

  author={Seo, Junyoung and Jang, Wooseok and Kwak, Min-Seop and Ko, Jaehoon and Kim, Hyeonsu and Kim, Junho and Kim, Jin-Hwa and Lee, Jiyoung and Kim, Seungryong},

  journal={arXiv preprint arXiv:2303.07937},

  year={2023}

}

```