Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

awesome-ai-talking-heads

A curated list of 'Talking Head Generation' resources. Features influential papers, groundbreaking algorithms, crucial GitHub repositories, insightful videos, and more. Ideal for AI enthusiasts, researchers, and graphics professionals
https://github.com/Curated-Awesome-Lists/awesome-ai-talking-heads

AudioGPT
SadTalker - Driven Single Image Talking Face Animation. 🎭🎶
Thin-Plate-Spline-Motion-Model - Plate Spline Motion Model for Image Animation. 🖼️
GeneFace - Fidelity 3D Talking Face Synthesis; ICLR 2023; Official code. 👤💬
CVPR2022-DaGAN - Aware Generative Adversarial Network for Talking Head Video Generation. 👥📹
sd-wav2lip-uhq
Text2Video - driven talking-head video synthesis with phonetic dictionary". 🔤🎞️
OTAvatar - shot Talking Face Avatar with Controllable Tri-plane Rendering [CVPR2023]. 👤🎭
Audio2Head - driven One-shot Talking-head Generation with Natural Head Motion" in the conference of IJCAI 2021. 🗣️👤
IP_LAP - Preserving Talking Face Generation With Landmark and Appearance Priors. 🔥🤖
Wunjo AI - time speech recognition, deepfake face & lips animation, face swap with one photo, change video by text prompts, segmentation, and retouching. Open-source, local & free. 🗣️👤💬
LIHQ - Inference, High Quality Synthetic Speaker (AI avatar/ AI presenter). 🎙️👤
Co-Speech-Motion-Generation
Neural Head Reenactment with Latent Pose Descriptors
NED
WACV23_TSNet - identity Video Motion Retargeting with Joint Transformation and Synthesis". 🎬✨
ICCV2023-MCNET
Speech2Video
StyleLipSync - based Personalized Lip-sync Video Generation". 💋🎥
How to Create Fake Talking Head Videos With Deep Learning (Code Tutorial)
AudioGPT: Understanding and Generating Speech, Music, Sound - modal AI system that can process complex audio information and understand and generate speech, music, sound, and talking head content.
Text-based Editing of Talking-head Video - head videos using text-based instructions.
Few-Shot Adversarial Learning of Realistic Neural Talking Head
DisCoHead: Audio-and-Video-Driven Talking Head Generation
Microsoft's 3D Photo Realistic Talking Head
Depth-Aware Generative Adversarial Network for Talking Head - based approach that leverages dense 3D facial geometry to generate realistic and accurate talking head videos.
Talking-head Generation with Rhythmic Head Motion - head videos with natural head movements, addressing the challenge of generating lip-synced videos while incorporating natural head motion. The proposed approach utilizes a 3D-aware generative network along with a hybrid embedding module and a non-linear composition module, resulting in controllable and photo-realistic talking-head videos with natural head movements.
Learned Spatial Representations for Few-shot Talking-Head Synthesis - shot talking-head synthesis by factorizing the representation of a subject into its spatial and style components. The proposed method predicts a dense spatial layout for the target image and utilizes it for synthesizing the target frame, achieving improved preservation of the subject's identity in the source images.
Efficient Emotional Adaptation for Audio-Driven Talking-Head - driven Talking-head (EAT) method, which transforms emotion-agnostic talking-head models into emotion-controllable ones in a cost-effective and efficient manner. The approach utilizes lightweight adaptations to enable precise and realistic emotion controls, achieving state-of-the-art performance on widely-used benchmarks.
High-Fidelity and Freely Controllable Talking Head Video Generation - quality and controllable talking-head videos. It introduces a novel model that leverages self-supervised learned landmarks and 3D face model-based landmarks to model the motion, along with a motion-aware multi-scale feature alignment module. The proposed method produces high-fidelity talking-head videos with free control over head pose and expression.
Implicit Identity Representation Conditioned Memory Compensation - fidelity talking head generation. The network module learns a unified spatial facial meta-memory bank, which compensates warped source facial features to overcome limitations due to complex motions in the driving video, resulting in improved generation quality.
Avatar Fingerprinting for Authorized Use of Synthetic Talking-Head - head videos. It proposes an embedding that groups the motion signatures of one identity together, allowing the identification of synthetic videos using the appearance of a specific individual driving the expressions.
Style Transfer for 2D Talking Head Animation - pattern construction, and a style-aware image generator. The method achieves better results than recent state-of-the-art methods in generating photo-realistic and fidelity 2D animation.
One-Shot Free-View Neural Talking-Head Synthesis for Video - head video synthesis model that learns to synthesize videos using a source image containing the target person's appearance and a driving video for motion. The model achieves high visual quality and bandwidth efficiency, outperforming competing methods on benchmark datasets.
Progressive Disentangled Representation Learning for Fine - shot talking head synthesis method that achieves disentangled control over lip motion, eye gaze & blink, head pose, and emotional expression. It utilizes a progressive disentangled representation learning strategy to isolate each motion factor, allowing for fine-grained control and high-quality speech and lip-motion synchronization.
VideoReTalking: Audio-based Lip Synchronization for Talking Head - world talking head videos according to input audio. It disentangles the editing task into face video generation, audio-driven lip-sync, and face enhancement, ultimately producing a high-quality and lip-syncing output video. The system utilizes learning-based approaches in a sequential pipeline, without requiring user intervention.
Video Production: You Can Make Simple Talking Head Video | Udemy
The Complete Talking Head Video Production Masterclass | Udemy - depth course.
Video Production - Inexpensive Talking Head Video - Business | Udemy
How to Create a Talking Head Video | Udemy - friendly course that covers the technicalities of filming and creating talking head videos.
Talking-Heads Attention - heads attention," a variation on multi-head attention that improves language modeling and comprehension tasks.
Few-Shot Adversarial Learning of Realistic Neural Talking Head
MakeItTalk: Speaker-Aware Talking-Head Animation
StyleTalk: One-shot Talking Head Generation with Controllable - shot talking heads with diverse personalized speaking styles.
DiffTalk: Crafting Diffusion Models for Generalized - driven denoising process using Latent Diffusion Models.
One-Shot High-Fidelity Talking-Head Synthesis with Deformable - fidelity talking heads by employing explicit 3D structural representations.
AD-NeRF: Audio Driven Neural Radiance Fields for Talking Head - fidelity talking head videos directly from input audio using neural scene representation networks.
Depth-Aware Generative Adversarial Network for Talking Head - supervised geometry learning method and leveraging dense 3D facial geometry for accurate talking head video generation.
What comprises a good talking-head video generation?: A Survey - head video generation, addressing the limitations of subjective evaluations. It explores desired properties such as identity preservation, lip synchronization, high video quality, and natural-spontaneous motion.
Text-based Editing of Talking-head Video - head videos based on their transcript, allowing for modifications in speech content while maintaining a seamless audio-visual flow. It utilizes annotations of facial features and a parametric face model for realistic video output.
MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation - preserving talking head generation framework that utilizes dense landmarks for accurate geometry-aware flow fields. It also proposes adaptive fusion of source identity during synthesis and a fast adaptation model using meta-learning for personalized fine-tuning.
Learning Dynamic Facial Radiance Fields for Few-Shot Talking Head Synthesis - shot talking head synthesis, which generalizes to unseen identities with limited training data. It conditions face radiance field on 2D appearance images, allowing flexible adjustment to new identities with few reference images.
Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motions - driven talking-head method that produces photo-realistic videos from a single reference image. It addresses challenges of producing natural head motions matching speech prosody while maintaining appearance during large head motions. It utilizes a head pose predictor and a motion field generator.
Efficient Emotional Adaptation for Audio-Driven Talking-Head Synthesis - driven Talking-head (EAT) method, which transforms emotion-agnostic talking-head models into emotion-controllable ones through parameter-efficient adaptations. It utilizes adaptations from different perspectives to enable realistic emotion controls.
Talking Head Generation with Probabilistic Audio-to-Visual Diffusion - shot audio-driven talking head generation using a probabilistic approach. It probabilistically generates facial motions matching input audio while maintaining audio-lip synchronization and overall photo-realism. It avoids the need for additional driving sources for controlled synthesis.
One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing - head video synthesis model for video conferencing. It utilizes a source image and a driving video to synthesize the talking-head video. It outperforms competing methods on benchmark datasets and enables video conferencing with high visual quality and low bandwidth usage.
Write-a-speaker: Text-based Emotional and Rhythmic Talking-head Video Generation - based talking-head video generation framework that synthesizes facial expressions and head motions according to contextual sentiments and speech rhythm. It consists of a speaker-independent stage and a speaker-specific stage, allowing tailored video synthesis for different individuals.
Style Transfer for 2D Talking Head Animation - head animation with learnable style references. It reconstructs 2D animation based on a single input image and an audio stream by extracting facial landmarks motion and incorporating style patterns from reference images.
LUCIA - 4 Talking Head Engine. 💻
Yepic Studio - style videos in minutes without expensive equipment. 🎥
Mel McGee's Talkbots - browser, multi-platform talking head application in SVG suitable for web sites or as an avatar. 🗣️
face3D_chung
CrazyTalk
tts avatar free download - SourceForge - browser, multi-platform talking head. (🔧👄)
Verbatim AI - Product Information, Latest Updates, and Reviews 2023 - time with Verbatim AI. Add interest, intrigue, and dynamism to your chat bots! (🔧👄)
Best Open Source BASIC 3D Modeling Software
DVDStyler / Discussion / Help: ffmpeg-vbr or internal
puffin web browser free download - SourceForge - browser, multi-platform talking head. (🔧👄)
12 best AI video generators to use in 2023 [Free and paid - quality videos from scratch. (🔧🎥)
(Paper Review)Few-Shot Adversarial Learning of Realistic Neural Talking Head Models - shot adversarial learning of realistic neural talking head models.
Nethania Michelle's Character | PPT
Presenting you: Top tips on presenting with Prezi Video – Prezi
Research Presentation | PPT
Adding narration to your presentation (using Prezi Video) – Prezi ...
Awesome List Generator - source Python package that uses the power of GPT models to automatically curate and generate starting points for resource lists related to a specific topic.

Programming Languages

Python 17 Jupyter Notebook 1

Keywords

talking-head 18 talking-face-generation 7 image-animation 6 gan 5 deepfake 5 motion-transfer 5 deep-learning 5 face 4 pytorch 4 face-reenactment 4 talking-face 4 talking-heads 3 video 3 cvpr2023 3 audio-driven-talking-face 3 deep-fake 3 avatar 3 speech 2 face-animation 2 pose-transfer 2 generative-ai 2 aigc 2 tts 2 vid2vid 2 deepfakes 2 deep-fakes 2 face-swapping 2 lip-sync 2 image-generation 2 audio 1 self-supervised-learning 1 voxceleb 1 gpt 1 expression-manipulation 1 music 1 sound 1 voxceleb2 1 3dmm 1 emotion 1 cvpr2022 1 pose-estimation 1 landmark-detection 1 head-reenactment 1 head-avatar 1 generative-model 1 facial-landmarks 1 gesture-controller 1 generation-algorithms 1 co-speech-gesture 1 wav2lip 1