https://github.com/harlanhong/face-depth-network
The component of DaGAN (CVPR 2022)
https://github.com/harlanhong/face-depth-network
cvpr depth face gan
Last synced: about 1 year ago
JSON representation
The component of DaGAN (CVPR 2022)
- Host: GitHub
- URL: https://github.com/harlanhong/face-depth-network
- Owner: harlanhong
- License: other
- Created: 2022-06-26T04:46:22.000Z (almost 4 years ago)
- Default Branch: master
- Last Pushed: 2022-06-27T07:46:10.000Z (almost 4 years ago)
- Last Synced: 2025-03-23T07:12:20.399Z (about 1 year ago)
- Topics: cvpr, depth, face, gan
- Language: Jupyter Notebook
- Homepage:
- Size: 540 KB
- Stars: 22
- Watchers: 2
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
## :book: The Face Depth Network of ``Depth-Aware Generative Adversarial Network for Talking Head Video Generation'' (CVPR 2022)
:fire: If DaGAN is helpful in your photos/projects, please help to :star: it or recommend it to your friends. Thanks:fire:
> [[Paper](https://arxiv.org/abs/2203.06605)] [[Project Page](https://harlanhong.github.io/publications/dagan.html)] [[Demo](https://huggingface.co/spaces/HarlanHong/DaGAN)] [[Poster Video](https://www.youtube.com/watch?v=nahsJNjWzGo&t=1s)]
> [Fa-Ting Hong](https://harlanhong.github.io), [Longhao Zhang](), [Li Shen](), [Dan Xu](https://www.danxurgb.net)
> The Hong Kong University of Science and Technology
### Cartoon Sample
https://user-images.githubusercontent.com/19970321/162151632-0195292f-30b8-4122-8afd-9b1698f1e4fe.mp4
### Human Sample
https://user-images.githubusercontent.com/19970321/162151327-f2930231-42e3-40f2-bfca-a88529599f0f.mp4
### Image Dataset
## :wrench: Dependencies and Installation
- Python >= 3.7 (Recommend to use [Anaconda](https://www.anaconda.com/download/#linux) or [Miniconda](https://docs.conda.io/en/latest/miniconda.html))
- [PyTorch >= 1.7](https://pytorch.org/)
- Option: NVIDIA GPU + [CUDA](https://developer.nvidia.com/cuda-downloads)
- Option: Linux
### ⚙️ Setup
1. Clone repo
```bash
git clone https://github.com/harlanhong/DaGAN-Head.git
cd CVPR2022-Head
```
2. Install dependent packages
```bash
conda install pytorch=0.4.1 torchvision=0.2.1 -c pytorch
pip install tensorboardX==1.4
conda install opencv=3.3.1 # just needed for evaluation
```
Or you can use the environment of [DaGAN](https://github.com/harlanhong/CVPR2022-DaGAN) directly
## :zap: Quick Inference
### Pre-trained checkpoint
The pre-trained checkpoint of face depth network and our DaGAN checkpoints can be found under following link: [OneDrive](https://hkustconnect-my.sharepoint.com/:f:/g/personal/fhongac_connect_ust_hk/EjfeXuzwo3JMn7s0oOPN_q0B81P5Wgu_kbYJAh7uSAKS2w?e=KaQcPk).
**Inference!**
To run a demo, download checkpoint and run the following command to predict scaled disparity for a single image with:
```shell
python test_simple.py --image_path assets/test_image.jpg --model_name tmp/You_Model/models/weights_19
```
## ⏳ Training
### Datasets
1) **Splits**. The train/test/validation splits are upload on the [One drive](https://hkustconnect-my.sharepoint.com/:f:/g/personal/fhongac_connect_ust_hk/Eq3AeGskIzlBnb6qiAsrGPoBM0Euv5yX9k3dtanIRgatcg?e=8gsxdr)
### Train on VoxCeleb
To train a model on specific dataset run:
```
CUDA_VISIBLE_DEVICES=0 python train.py --batch_size 32 --heigh 256 --width 256 --dataset vox --sample_num 100000 --model_name taking_head_10w --data_path vox2
```
### Training on your own dataset
You can train on a custom monocular or stereo dataset by writing a new dataloader class which inherits from `MonoDataset` – see the `CELEBDataset` class in `datasets/celeb_dataset.py` for an example.
## :scroll: Acknowledgement
Our Face-Depth-Network implementation is borrowed from [Monodepth2](https://github.com/nianticlabs/monodepth2). We appreciate the authors of Monodepth2 for making their codes available to public.
## :scroll: BibTeX
```
@inproceedings{hong2022depth,
title={Depth-Aware Generative Adversarial Network for Talking Head Video Generation},
author={Hong, Fa-Ting and Zhang, Longhao and Shen, Li and Xu, Dan},
journal={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
year={2022}
}
```
### :e-mail: Contact
If you have any question, please email `fhongac@cse.ust.hk`.