Papers

Audio-Driven motion generation

• GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis
• DiffMotion: Speech-Driven Gesture Synthesis Using Denoising Diffusion Model
• DiffTalk: Crafting Diffusion Models for Generalized Talking Head Synthesis
• Generating Holistic 3D Human Motion from Speech
• Audio-Driven Co-Speech Gesture Video Generation
• Listen, denoise, action! Audio-driven motion synthesis with diffusion models
• ZeroEGGS: Zero-shot Example-based Gesture Generation from Speech
• EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model
• Learning Hierarchical Cross-Modal Association for Co-Speech Gesture Generation
• SEEG: Semantic Energized Co-speech Gesture Generation
• Freeform Body Motion Generation from Speech
• Learning Speech-driven 3D Conversational Gestures from Video
• Learning Individual Styles of Conversational Gesture
• here
• [Code
• [Code
• Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation
• [Incoming Code
• [Code
• BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis
• [Code
• [Code
• [Code
• [Code
• FaceFormer: Speech-Driven 3D Facial Animation with Transformers
• [Code
• [Code
• Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoders
• [Code
• [Code

Human motion prediction

• here
• HumanMAC: Masked Motion Completion for Human Motion Prediction
• NeMF: Neural Motion Fields for Kinematic Animation
• Multi-Person Extreme Motion Prediction
• MotionMixer: MLP-based 3D Human Body Pose Forecasting
• Multi-Person 3D Motion Prediction with Multi-Range Transformers
• InterDiff: Generating 3D Human-Object Interactions with Physics-Informed Diffusion
• [Code
• Stochastic Multi-Person 3D Motion Forecasting
• [Code
• [Code
• BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction
• [Upcoming Code
• Diverse Human Motion Prediction Guided by Multi-Level Spatial-Temporal Anchors
• [Code
• [Code
• [Code
• [Code
• [Code

Motion Applications

• MIME: Human-Aware 3D Scene Generation
• Scene Synthesis from Human Motion
• TEACH: Temporal Action Compositions for 3D Humans
• Motion In-betweening via Two-stage Transformers
• Conditional Motion In-betweening
• SkeletonMAE: Spatial-Temporal Masked Autoencoders for Self-supervised Skeleton Action Recognition
• A Unified Framework for Real Time Motion Completion
• Transformer based Motion In-betweening
• Generative Tweening: Long-term Inbetweening of 3D Human Motions
• [Code
• [Code
• Motion In-betweening via Two-stage Transformers
• Skeleton2Humanoid: Animating Simulated Characters for Physically-plausible Motion In-betweening
• [Upcoming Code
• [Code
• [Code

Music-Driven motion generation

• Taming Diffusion Models for Music-driven Conducting Motion Generation
• Magic: Multi Art Genre Intelligent Choreography Dataset and Network for 3D Dance Generation
• EDGE: Editable Dance Generation From Music
• You Never Stop Dancing: Non-freezing Dance Generation via Bank-constrained Manifold Projection
• A Brand New Dance Partner: Music-Conditioned Pluralistic Dancing Controlled by Multiple Dance Genres
• Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memory
• Dance Style Transfer with Cross-modal Transformer
• Music-driven Dance Regeneration with Controllable Key Pose Constraints
• [Code
• Music-Driven Group Choreography
• [Code
• Pretrained Diffusion Models for Unified Human Motion Synthesis
• [Code
• [Code
• [Upcoming Code
• AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
• [Code
• GroupDancer: Music to Multi-People Dance Synthesis with Style Collaboration
• Music-Driven Group Choreography

Text-3D Image Generation

• Point-E: A System for Generating 3D Point Clouds from Complex Prompts
• DreamFusion: Text-to-3D using 2D Diffusion

Text-Driven motion generation

• ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
• GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents
• Modiff: Action-Conditioned 3D Motion Generation with Denoising Diffusion Probabilistic Models
• Executing your Commands via Motion Diffusion in Latent Space
• MultiAct: Long-Term 3D Human Motion Generation from Multiple Action Labels
• MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis
• UDE: A Unified Driving Engine for Human Motion Generation
• MotionBERT: Unified Pretraining for Human Motion Analysis
• FLAME: Free-form Language-based Motion Synthesis & Editing
• MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
• TEMOS: Generating diverse human motions from textual descriptions
• GIMO: Gaze-Informed Human Motion Prediction in Context
• AvatarCLIP: Zero-Shot Text-Driven Generation and Animation of 3D Avatars
• Text2Gestures: A Transformer-Based Network for Generating Emotive Body Gestures for Virtual Agents
• [Code
• [Code
• T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations
• [Code
• [Upcoming Code
• [Code
• UDE: A Unified Driving Engine for Human Motion Generation
• [Upcoming Code
• [Code
• [Code
• [Code
• [Code
• [Code
• MotionCLIP: Exposing Human Motion Generation to CLIP Space
• [Code
• Generating Diverse and Natural 3D Human Motions from Text
• [Code
• [Code
• [Code

Text-Image Generation

• SpaText: Spatio-Textual Representation for Controllable Image Generation
• Sketch-Guided Text-to-Image Diffusion Models
• Make-A-Story: Visual Memory Conditioned Consistent Story Generation
• Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models
• InstructPix2Pix: Learning to Follow Image Editing Instructions
• Null-text Inversion for Editing Real Images using Guided Diffusion Models
• HumanDiffusion: a Coarse-to-Fine Alignment Diffusion Framework for Controllable Text-Driven Person Image Generation
• Imagic: Text-Based Real Image Editing with Diffusion Models
• Self-Guided Diffusion Models
• On Distillation of Guided Diffusion Models
• DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
• Prompt-to-Prompt Image Editing with Cross Attention Control
• Improved Vector Quantized Diffusion Models
• Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors
• Diffusion Autoencoders: Toward a Meaningful and Decodable Representation
• Vector Quantized Diffusion Model for Text-to-Image Synthesis
• High-Resolution Image Synthesis with Latent Diffusion Models
• here
• Adding Conditional Control to Text-to-Image Diffusion Models
• Sketch-Guided Text-to-Image Diffusion Models
• Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models
• [Code
• [Code
• [Code
• [Code
• [Code

Text-Video Generation

• Text-To-4D Dynamic Scene Generation
• Structure and Content-Guided Video Synthesis with Diffusion Models
• MagicVideo: Efficient Video Generation With Latent Diffusion Models
• Text2LIVE: Text-Driven Layered Image and Video Editing
• TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Models
• [Code
• Conditional Image-to-Video Generation with Latent Flow Diffusion Models
• [Code
• Latent Video Diffusion Models for High-Fidelity Video Generation with Arbitrary Lengths
• [Upcoming Code
• [Code
• Structure and Content-Guided Video Synthesis with Diffusion Models

Uncategorized

Uncategorized

• Tracking Papers on Diffusion Models