Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://huangyangyi.github.io/TeCH/
[3DV 2024] Official repo of "TeCH: Text-guided Reconstruction of Lifelike Clothed Humans"
https://huangyangyi.github.io/TeCH/
Last synced: 3 months ago
JSON representation
[3DV 2024] Official repo of "TeCH: Text-guided Reconstruction of Lifelike Clothed Humans"
- Host: GitHub
- URL: https://huangyangyi.github.io/TeCH/
- Owner: huangyangyi
- License: mit
- Created: 2023-08-17T03:12:09.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-03-07T07:42:20.000Z (10 months ago)
- Last Synced: 2024-06-02T01:32:34.518Z (8 months ago)
- Language: Python
- Homepage: https://huangyangyi.github.io/TeCH/
- Size: 46.9 MB
- Stars: 348
- Watchers: 28
- Forks: 22
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-scene-representation - TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
README
TeCH: Text-guided Reconstruction of Lifelike Clothed Humans
Yangyi Huang*
·
Hongwei Yi*
·
Yuliang Xiu*
·
Tingting Liao
·
Jiaxiang Tang
·
Deng Cai
·
Justus Thies
* Equal contribution
3DV 2024
TeCH considers image-based reconstruction as a conditional generation task, taking conditions from both the input image and the derived descriptions. It is capable of reconstructing "lifelike" 3D clothed humans. “Lifelike” refers to 1) a detailed full-body geometry, including facial features and clothing wrinkles, in both frontal and unseen regions, and 2) a high-quality texture with consistent color and intricate patterns.
## Installation
Please follow the [Installation Instruction](docs/install.md) to setup all the required packages.
## Getting Started
We provide a running script at `scripts/run.sh`. Before getting started, you need to set your own environment variables of `CUDA_HOME` and `REPLICATE_API_TOKEN`([get your token here](https://replicate.com/signin?next=/account/api-tokens)) in the script.
After that, you can use TeCH to create a highly detailed clothed human textured mesh from a single image, for example:
```shell
sh scripts/run.sh input/examples/name.img exp/examples/name
```
The results will be saved in the experiment folder `exp/examples/name`, and the textured mesh will be saved as `exp/examples/name/obj/name_texture.obj`
It is noted that in "Step 3", the current version of Dreambooth implementation requires 2\*32G GPU memory. And 1\*32G GPU memory is efficient for other steps. The entire training process for a subject takes ~3 hours on our V100 GPUs.
## TODOs
- [ ] Release of evaluation protocols and results data for comparison (on CAPE & THUman 2.0 datasets).
- [ ] Switch to the diffuser version of DreamBooth to save training memory.
- [ ] Further improvement of efficiency and robustness.
## Citation
```bibtex
@inproceedings{huang2024tech,
title={{TeCH: Text-guided Reconstruction of Lifelike Clothed Humans}},
author={Huang, Yangyi and Yi, Hongwei and Xiu, Yuliang and Liao, Tingting and Tang, Jiaxiang and Cai, Deng and Thies, Justus},
booktitle={International Conference on 3D Vision (3DV)},
year={2024}
}
```
## Contributors
Kudos to all of our amazing contributors! TeCH thrives through open-source. In that spirit, we welcome all kinds of contributions from the community.
_Contributor avatars are randomly shuffled._
## License
This code and model are available only for **non-commercial** research purposes as defined in the LICENSE (i.e., MIT LICENSE).
Note that, using TeCH, you have to register SMPL-X and agree with the LICENSE of it, and it's not MIT LICENSE, you can check the LICENSE of SMPL-X from https://github.com/vchoutas/smplx/blob/main/LICENSE.
## Acknowledgment
This implementation is mainly built based on [Stable Dreamfusion](https://github.com/ashawkey/stable-dreamfusion), [ECON](https://github.com/YuliangXiu/ECON), [DreamBooth-Stable-Diffusion](https://github.com/XavierXiao/Dreambooth-Stable-Diffusion), and the BLIP API from Salesforce on [Replicate](https://replicate.com/salesforce/blip)