https://github.com/wangsuzhen/Audio2Head

code for paper "Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion" in the conference of IJCAI 2021
https://github.com/wangsuzhen/Audio2Head

codes ijcai2021 multi-modal paper talking-face talking-head

Last synced: about 1 month ago
JSON representation

code for paper "Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion" in the conference of IJCAI 2021

• Host: GitHub
• URL: https://github.com/wangsuzhen/Audio2Head
• Owner: wangsuzhen
• Created: 2021-10-21T08:31:16.000Z (almost 3 years ago)
• Default Branch: main
• Last Pushed: 2024-02-15T07:19:47.000Z (7 months ago)
• Last Synced: 2024-04-24T17:24:39.707Z (5 months ago)
• Topics: codes, ijcai2021, multi-modal, paper, talking-face, talking-head
• Language: Python
• Homepage:
• Size: 1.02 MB
• Stars: 291
• Watchers: 7
• Forks: 54
• Open Issues: 14
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

• awesome-ai-talking-heads - Audio2Head - driven One-shot Talking-head Generation with Natural Head Motion" in the conference of IJCAI 2021. 🗣️👤 (GitHub projects)

README

        
# Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion (IJCAI 2021)

#### [Paper](https://www.ijcai.org/proceedings/2021/0152.pdf) | [Demo](https://www.youtube.com/watch?v=xvcBJ29l8rA)

#### Requirements

- Python 3.6 , Pytorch >= 1.6 and ffmpeg

- Other requirements are listed in the 'requirements.txt'

  

#### Pretrained Checkpoint

Please download the pretrained checkpoint from [google-drive](https://drive.google.com/file/d/1tvI43ZIrnx9Ti2TpFiEO4dK5DOwcECD7/view?usp=sharing) and put it within the folder (`/checkpoints`).

#### Generate Demo Results

```

python inference.py --audio_path xxx.wav --img_path xxx.jpg

```

Note that the input images must keep the same height and width and the face should be appropriately cropped as in `/demo/img`.

#### License and Citation

```

@InProceedings{wang2021audio2head,

author = Suzhen Wang, Lincheng Li, Yu Ding, Changjie Fan, Xin Yu

title = {Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motion},

booktitle = {the 30th International Joint Conference on Artificial Intelligence (IJCAI-21)},

year = {2021},

}

```

#### Acknowledgement

This codebase is based on [First Order Motion Model](https://github.com/AliaksandrSiarohin/first-order-model), thanks for their contribution.