https://aashishrai3799.github.io/Towards-Realistic-Generative-3D-Face-Models/

3D face model that can generate high-quality mesh and texture
https://aashishrai3799.github.io/Towards-Realistic-Generative-3D-Face-Models/

Last synced: 3 months ago
JSON representation

3D face model that can generate high-quality mesh and texture

• Host: GitHub
• URL: https://aashishrai3799.github.io/Towards-Realistic-Generative-3D-Face-Models/
• Owner: aashishrai3799
• License: other
• Created: 2022-11-14T05:59:50.000Z (about 2 years ago)
• Default Branch: main
• Last Pushed: 2024-07-30T18:22:27.000Z (6 months ago)
• Last Synced: 2024-07-30T22:42:55.548Z (6 months ago)
• Language: Python
• Size: 50.1 MB
• Stars: 244
• Watchers: 39
• Forks: 19
• Open Issues: 14
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• awesome-scene-representation - Towards Realistic Generative 3D Face Models - Realistic-Generative-3D-Face-Models/) | [bibtex](./citations/rai2023towards.txt) (Uncategorized / Uncategorized)

README

        
# Towards Realistic Generative 3D Face Models (AlbedoGAN)

Aashish Rai, Hiresh Gupta*, Ayush Pandey*,  Francisco Vicente Carrasco, Shingo Jason Takagi, Amaury Aubel, Daeil Kim, Aayush Prakash, Fernando de la Torre

### Carnegie Mellon University, Meta Reality Labs

### WACV 2024 

[[Project Page](https://aashishrai3799.github.io/Towards-Realistic-Generative-3D-Face-Models)] [[Arxiv](https://arxiv.org/pdf/2304.12483.pdf)] [[Demo Results](https://drive.google.com/file/d/1FCEUL1Xaaz75vYZEomSOYjS91tlOcW8Q/view?usp=sharing)]

We propose a 3D face generative model that generates high-quality albedo and precise 3D shape by leveraging StyleGAN2, resulting in a photo-realistic rendered image.

![](figure_1.png)

![](supp_image.png)

## Inference

Conda environment: Refer environment.yml

Download pre-trained models and put in the respective folders. 

Follow [[MICA](https://github.com/Zielon/MICA)] to download insightface and MICA pre-trained models. Put the weights in 'insightface' and 'data/mica_pretrained' folders, respectively.

Follow [[DECA](https://github.com/yfeng95/DECA)] to download DECA pre-trained weights. Put them in the 'data' folder.

Download AlbedoGAN modified weights from the following [[LINK](https://drive.google.com/drive/folders/1nJw8rUBTLcyhvCMTDohE_KcKKtFI6Orm?usp=sharing)]. Put these modified ArcFace backbone and DECA weights to generate better reconstruction results.

- Generate Random 3D Faces (mesh and texture)

    ```

    python demos/demo_generate.py

    ```

    

- Reconstruct 3D Faces from 2D Images

    ```

    python demos/demo_reconstruct.py

    ```

- Generate multi-pose videos

    ```

    python video.py

    ```

## Training code will be released at the earliest convenience. 

## Acknowledgements

For functions or scripts that are based on external sources, we acknowledge the origin individually in each file.

Our code is inspired by the following code repositories. Please make sure to refer to their and our license terms before downloading the pre-trained weights.

1. [[DECA](https://github.com/yfeng95/DECA)]

2. [[MICA](https://github.com/Zielon/MICA)]

3. [[FLAME](https://github.com/soubhiksanyal/FLAME_PyTorch)]

    

## License Terms

The code is available under X11 License. Please read the license terms available at [[Link](https://github.com/aashishrai3799/Towards-Realistic-Generative-3D-Face-Models/blob/main/LICENSE)]. Quick summary available at [[Link](https://www.tldrlegal.com/l/x11)].

## Citation

If you find this paper/code useful, please consider citing:

```bibtex

@article{rai2023towards,

  		title={Towards Realistic Generative 3D Face Models},

  		author={Rai, Aashish and Gupta, Hiresh and Pandey, Ayush and Carrasco, Francisco Vicente and Takagi, Shingo Jason and Aubel, Amaury and Kim, Daeil and Prakash, Aayush and De la Torre, Fernando},

  		journal={arXiv preprint arXiv:2304.12483},

  		year={2023}

 		}

```