awesome-ai4animation

🔥🔥🔥 This repository includes latest papers, projects and datasets on GenAI for Cel-Animation.
https://github.com/yunlong10/awesome-ai4animation

Last synced: 22 days ago
JSON representation

🛠️ Methods
- 2️⃣ Production
  - **AnimeGamer: Infinite Anime Life Simulation with Next Game State Prediction**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation** - ntu.com/project/fresco/) [Code](https://github.com/williamyang1991/FRESCO) [Demo](https://www.youtube.com/watch?v=jLnGx5H-wLw) | CVPR 2024 |
  - **TokenFlow: Consistent Diffusion Features for Consistent Video Editing** - Tal, Shai Bagon, Tali Dekely | [Project](https://diffusion-tokenflow.github.io/) [Code](https://github.com/omerbt/TokenFlow) [Demo](https://huggingface.co/spaces/weizmannscience/tokenflow) | ICLR 2024 |
  - **LVCD: Reference-based Lineart Video Colorization with Diffusion Models**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Framer: Interactive Frame Interpolation** - uofa.github.io/Framer/) [Code](https://github.com/aim-uofa/Framer) [Demo](https://huggingface.co/spaces/wwen1997/Framer)| |
  - **CogCartoon: Towards Practical Story Visualization**
  - **Sketch-Guided Scene Image Generation**
  - **VideoComposer: Compositional Video Synthesis with Motion Controllability** - vilab/videocomposer) | NIPS 2023 |
  - **LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators** - Generation/layout-generation/tree/master/LayoutGAN) | ICLR 2019 |
  - **Sketch-Guided Scene Image Generation**
  - **VideoComposer: Compositional Video Synthesis with Motion Controllability** - vilab/videocomposer) | NIPS 2023 |
  - **LayoutGAN: Generating Graphic Layouts with Wireframe Discriminators** - Generation/layout-generation/tree/master/LayoutGAN) | ICLR 2019 |
  - **VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning** - hanlin/VideoDirectorGPT) | COLM 2024 |
  - **DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation**
  - **Manga Generation via Layout-controllable Diffusion** - fdu.github.io/MangaDiffusion/) [Code](https://github.com/siyuch-fdu/MangaDiffusion) | |
  - **CameraCtrl: Enabling Camera Control for Text-to-Video Generation** - CameraCtrl/) [Demo](https://huggingface.co/spaces/hehao13/CameraCtrl-svd) | |
  - **Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation** - anyone/) [Code](https://github.com/HumanAIGC/AnimateAnyone) | CVPR 2024 |
  - **Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance** - generative-vision.github.io/champ/#/) [Code](https://github.com/fudan-generative-vision/champ) | |
  - **CameraCtrl: Enabling Camera Control for Text-to-Video Generation** - CameraCtrl/) [Demo](https://huggingface.co/spaces/hehao13/CameraCtrl-svd) | |
  - **DiffSensei: Bridging Multi-Modal LLMs and Diffusion Models for Customized Manga Generation**
  - **Manga Generation via Layout-controllable Diffusion** - fdu.github.io/MangaDiffusion/) [Code](https://github.com/siyuch-fdu/MangaDiffusion) | |
  - **Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation** - anyone/) [Code](https://github.com/HumanAIGC/AnimateAnyone) | CVPR 2024 |
  - **Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidance** - generative-vision.github.io/champ/#/) [Code](https://github.com/fudan-generative-vision/champ) | |
  - **MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance** - Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, Fangyuan Zou | [Project Page](https://tencent.github.io/MimicMotion/) [Code](https://github.com/tencent/MimicMotion) | |
  - **Animate-X: Universal Character Image Animation with Enhanced Motion Representation** - x) | |
  - **MikuDance: Animating Character Art with Mixed Motion Dynamics**
  - **Textoon: Generating Vivid 2D Cartoon Characters from Text Descriptions**
  - **Joint Stroke Tracing and Correspondence for 2D Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Enhanced Deep Animation Video Interpolation**
  - **MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance** - Wen Wang, Han Wang, Junqi Cheng, Yuefeng Zhu, Fangyuan Zou | [Project Page](https://tencent.github.io/MimicMotion/) [Code](https://github.com/tencent/MimicMotion) | |
  - **Animate-X: Universal Character Image Animation with Enhanced Motion Representation** - x) | |
  - **MikuDance: Animating Character Art with Mixed Motion Dynamics**
  - **Textoon: Generating Vivid 2D Cartoon Characters from Text Descriptions**
  - **Joint Stroke Tracing and Correspondence for 2D Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Enhanced Deep Animation Video Interpolation**
  - **Improving the Perceptual Quality of 2D Animation Interpolation**
  - **Improving the Perceptual Quality of 2D Animation Interpolation**
  - **Deep Animation Video Interpolation in the Wild**
  - **Deep Sketch-Guided Cartoon Video Inbetweening**
  - **Optical Flow Based Line Drawing Frame Interpolation Using Distance Transform to Support Inbetweenings**
  - **DiLight: Digital Light Table – Inbetweening for 2D Animations Using Guidelines**
  - **AniDoc: Animation Creation Made Easier** - meng.github.io/AniDoc_demo/) [Code](https://github.com/yihao-meng/AniDoc) | arXiv 2024 |
  - **DiLight: Digital Light Table – Inbetweening for 2D Animations Using Guidelines**
  - **AniDoc: Animation Creation Made Easier** - meng.github.io/AniDoc_demo/) [Code](https://github.com/yihao-meng/AniDoc) | arXiv 2024 |
  - **VToonify: Controllable High-Resolution Portrait Video Style Transfer** - ntu.com/project/vtoonify/) [Code](https://github.com/williamyang1991/VToonify) | TOG 2022 |
  - **VToonify: Controllable High-Resolution Portrait Video Style Transfer** - ntu.com/project/vtoonify/) [Code](https://github.com/williamyang1991/VToonify) | TOG 2022 |
  - **StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces** - ntu.com/project/styleganex/) [Code](https://github.com/williamyang1991/StyleGANEX) [Demo](https://www.youtube.com/watch?v=8oK0TXQmxg8) | ICCV 2023 |
  - **StyleGANEX: StyleGAN-Based Manipulation Beyond Cropped Aligned Faces** - ntu.com/project/styleganex/) [Code](https://github.com/williamyang1991/StyleGANEX) [Demo](https://www.youtube.com/watch?v=8oK0TXQmxg8) | ICCV 2023 |
  - **PromptFix: You Prompt and We Fix the Photo** - Page/) [Code](https://github.com/yeates/PromptFix) | NIPS 2024 |
  - **Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization**
  - **Coloring Anime Line Art Videos with Transformation Region Enhancement Network**
  - **SketchBetween: Video-to-Video Synthesis for Sprite Animation via Sketches**
  - **The Animation Transformer: Visual Correspondence via Segment Matching**
  - **Deep Line Art Video Colorization with a Few References** - Qi Zhang, Shu-Yu Chen, Lin Gao, Yu-Kun Lai, Fang-Lue Zhang | | arXiv 2020 |
  - **Artist-Guided Semiautomatic Animation Colorization**
  - **Automatic Temporally Coherent Video Colorization** - Thasarathan/TCVC) | arXiv 2019 |
  - **The Animation Transformer: Visual Correspondence via Segment Matching**
  - **Artist-Guided Semiautomatic Animation Colorization**
  - **Line Art Correlation Matching Feature Transfer Network for Automatic Animation Colorization**
  - **Deep Line Art Video Colorization with a Few References** - Qi Zhang, Shu-Yu Chen, Lin Gao, Yu-Kun Lai, Fang-Lue Zhang | | arXiv 2020 |
  - **Automatic Temporally Coherent Video Colorization** - Thasarathan/TCVC) | arXiv 2019 |
  - **Toona**
  - **MangaNinja: Line Art Colorization with Precise Reference Following** - vilab/MangaNinjia) | |
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **ToonCrafter: Generative Cartoon Interpolation** - Tsin Wong | [Project](https://doubiiu.github.io/projects/ToonCrafter/) [Code](https://github.com/Doubiiu/ToonCrafter) | TOG 2024 |
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **ToonCrafter: Generative Cartoon Interpolation** - Tsin Wong | [Project](https://doubiiu.github.io/projects/ToonCrafter/) [Code](https://github.com/Doubiiu/ToonCrafter) | TOG 2024 |
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
  - **Deep Animation Video Interpolation in the Wild**
  - **Deep Sketch-Guided Cartoon Video Inbetweening**
  - **Optical Flow Based Line Drawing Frame Interpolation Using Distance Transform to Support Inbetweenings**
  - **FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation** - ntu.com/project/fresco/) [Code](https://github.com/williamyang1991/FRESCO) [Demo](https://www.youtube.com/watch?v=jLnGx5H-wLw) | CVPR 2024 |
  - **TokenFlow: Consistent Diffusion Features for Consistent Video Editing** - Tal, Shai Bagon, Tali Dekely | [Project](https://diffusion-tokenflow.github.io/) [Code](https://github.com/omerbt/TokenFlow) [Demo](https://huggingface.co/spaces/weizmannscience/tokenflow) | ICLR 2024 |
  - **PromptFix: You Prompt and We Fix the Photo** - Page/) [Code](https://github.com/yeates/PromptFix) | NIPS 2024 |
  - **LVCD: Reference-based Lineart Video Colorization with Diffusion Models**
  - **Coloring Anime Line Art Videos with Transformation Region Enhancement Network**
  - **SketchBetween: Video-to-Video Synthesis for Sprite Animation via Sketches**
  - **Animation Line Art Colorization Based on Optical Flow Method**
  - **Toona**
  - **MangaNinja: Line Art Colorization with Precise Reference Following** - vilab/MangaNinjia) | |
  - **Exploring Inbetween Charts with Trajectory-Guided Sliders for Cutout Animation**
- 1️⃣ Pre-production
  - **CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities** - CS/CustomCrafter) | AAAI 2025 |
  - **VideoStudio: Generating Consistent-Content and Multi-Scene Videos**
  - **Mind the Time: Temporally-Controlled Multi-Event Video Generation** - video.github.io/) | |
  - **HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing**
  - **Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context**
  - **Claude 3.5 Sonnet**
  - **HoLLMwood: Unleashing the Creativity of Large Language Models in Screenwriting via Role Playing**
  - **Claude 3.5 Sonnet**
  - **Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context**
  - **InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks**
  - **Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond** - VL) | |
  - **InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks**
  - **Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond** - VL) | |
  - **GPT-4**
  - **High-Resolution Image Synthesis with Latent Diffusion Models** - diffusion) | CVPR 2022 |
  - **MidJourney**
  - **Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models**
  - **Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation** - cvc.github.io/Animate-A-Story/) [Code](https://github.com/AILab-CVC/Animate-A-Story) | |
  - **GPT-4**
  - **High-Resolution Image Synthesis with Latent Diffusion Models** - diffusion) | CVPR 2022 |
  - **MidJourney**
  - **StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character Customization** - Zhangjl/StoryWeaver) | |
  - **Intelligent Grimm -- Open-ended Visual Storytelling via Latent Diffusion Models**
  - **StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completion** - Kun Bao, Hao Tang, Yaowei Wang, Changsheng Xu | [Code](https://github.com/tobran/StoryImager) | ECCV 2024 |
  - **Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models**
  - **SEED-Story: Multimodal Long Story Generation with Large Language Model** - Story) [Dataset](https://huggingface.co/datasets/TencentARC/StoryStream) | |
  - **Make-A-Story: Visual Memory Conditioned Consistent Story Generation** - Ying Lee, Jian Ren, Sergey Tulyakov, Shweta Mahajan, Leonid Sigal | [Code](https://github.com/ubc-vision/Make-A-Story) | |
  - **Make-A-Storyboard: A General Framework for Storyboard with Disentangled and Merged Control**
  - **Make-A-Story: Visual Memory Conditioned Consistent Story Generation** - Ying Lee, Jian Ren, Sergey Tulyakov, Shweta Mahajan, Leonid Sigal | [Code](https://github.com/ubc-vision/Make-A-Story) | |
  - **Make-A-Storyboard: A General Framework for Storyboard with Disentangled and Merged Control**
  - **Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation** - cvc.github.io/Animate-A-Story/) [Code](https://github.com/AILab-CVC/Animate-A-Story) | |
  - **StoryImager: A Unified and Efficient Framework for Coherent Story Visualization and Completion** - Kun Bao, Hao Tang, Yaowei Wang, Changsheng Xu | [Code](https://github.com/tobran/StoryImager) | ECCV 2024 |
  - **CustomCrafter: Customized Video Generation with Preserving Motion and Concept Composition Abilities** - CS/CustomCrafter) | AAAI 2025 |
  - **VideoStudio: Generating Consistent-Content and Multi-Scene Videos**
  - **Mind the Time: Temporally-Controlled Multi-Event Video Generation** - video.github.io/) | |
  - **DreamRunner: Fine-Grained Storytelling Video Generation with Retrieval-Augmented Motion Adaptation**
  - **Vlogger: Make Your Dream A Vlog**
  - **Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation** - TMG/Anim-Director) | SIGGRAPH Asia 2024 |
  - **Storyboarder.ai**
  - **DreamRunner: Fine-Grained Storytelling Video Generation with Retrieval-Augmented Motion Adaptation**
  - **Vlogger: Make Your Dream A Vlog**
  - **Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation** - TMG/Anim-Director) | SIGGRAPH Asia 2024 |
  - **Storyboarder.ai**
  - **CogCartoon: Towards Practical Story Visualization**
  - **VideoDirectorGPT: Consistent Multi-scene Video Generation via LLM-Guided Planning** - hanlin/VideoDirectorGPT) | COLM 2024 |
  - **StoryWeaver: A Unified World Model for Knowledge-Enhanced Story Character Customization** - Zhangjl/StoryWeaver) | |
  - **Synthesizing Coherent Story with Auto-Regressive Latent Diffusion Models**
  - **SEED-Story: Multimodal Long Story Generation with Large Language Model** - Story) [Dataset](https://huggingface.co/datasets/TencentARC/StoryStream) | |
- 3️⃣ Post-Production
  - **Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model** - lab/video2music) [Project Page](https://amaai-lab.github.io/Video2Music/) [Code](https://github.com/amaai-lab/video2music) [Dataset](https://zenodo.org/records/10057093) | |
  - **MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models** - my.sharepoint.com/personal/sanjoyc_umd_edu/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fsanjoyc%5Fumd%5Fedu%2FDocuments%2FMeLFusion%20datasets&ga=1) | CVPR 2024 |
  - **Scaling In-the-Wild Training for Diffusion-based Illumination Harmonization and Editing by Imposing Consistent Light Transport**
  - **PCT-Net: Full Resolution Image Harmonization Using Pixel-Wise Color Transformations** - Net-Image-Harmonization/) | CVPR 2023 |
  - **Thinking Outside the BBox: Unconstrained Generative Object Compositing**
  - **Scaling In-the-Wild Training for Diffusion-based Illumination Harmonization and Editing by Imposing Consistent Light Transport**
  - **High-Resolution Image Harmonization via Collaborative Dual Transformations** - High-Resolution-Image-Harmonization) | CVPR 2022 |
  - **SSH: A Self-Supervised Framework for Image Harmonization** - Group/SSHarmonization) | ICCV 2021 |
  - **Thinking Outside the BBox: Unconstrained Generative Object Compositing**
  - **Dr.Bokeh: DiffeRentiable Occlusion-aware Bokeh Rendering** - Src) | |
  - **ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion** - Acha, Yedid Hoshen | [Project Page](https://objectdrop.github.io/) | |
  - **Dr.Bokeh: DiffeRentiable Occlusion-aware Bokeh Rendering** - Src) | |
  - **Floating No More: Object-Ground Reconstruction from a Single Image** - Yan Gui, Yu-Xiong Wang | [Project Page](https://yunzeman.github.io/ORG/) | |
  - **ObjectDrop: Bootstrapping Counterfactuals for Photorealistic Object Removal and Insertion** - Acha, Yedid Hoshen | [Project Page](https://objectdrop.github.io/) | |
  - **DisenStudio: Customized Multi-Subject Text-to-Video Generation with Disentangled Spatial Control** - ov-file) | ACMMM 2024 |
  - **SSN: Soft Shadow Network for Image Compositing**
  - **DisenStudio: Customized Multi-Subject Text-to-Video Generation with Disentangled Spatial Control** - ov-file) | ACMMM 2024 |
  - **SSN: Soft Shadow Network for Image Compositing**
  - **Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward** - 1k) | ACCV 2022 |
  - **Foley Music: Learning to Generate Music from Videos** - Music) | ECCV 2020 |
  - **Multi-modal Segment Assemblage Network for Ad Video Editing with Importance-Coherence Reward** - 1k) | ACCV 2022 |
  - **Reframe Anything: LLM Agent for Open World Video Reframing**
  - **OpusClip**
  - **Reframe Anything: LLM Agent for Open World Video Reframing**
  - **OpusClip**
  - **Foley Music: Learning to Generate Music from Videos** - Music) | ECCV 2020 |
  - **V2Meow: Meowing to the Visual Beat via Video-to-Music Generation** - research.github.io/noise2music/v2meow/) | AAAI 2024 |
  - **VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling**
  - **Taming Visually Guided Sound Generation** - iashin.github.io/SpecVQGAN) [Demo](https://colab.research.google.com/drive/1pxTIMweAKApJZ3ZFqyBee3HtMqFpnwQ0?usp=sharing) [Code](https://github.com/v-iashin/SpecVQGAN) | BMVC 2021 |
  - **VidMuse: A Simple Video-to-Music Generation Framework with Long-Short-Term Modeling**
  - **Taming Visually Guided Sound Generation** - iashin.github.io/SpecVQGAN) [Demo](https://colab.research.google.com/drive/1pxTIMweAKApJZ3ZFqyBee3HtMqFpnwQ0?usp=sharing) [Code](https://github.com/v-iashin/SpecVQGAN) | BMVC 2021 |
  - **I Hear Your True Colors: Image Guided Audio Generation** - lab/im2wav/) [Code](https://github.com/RoySheffer/im2wav) | ICASSP 2023 |
  - **FoleyGen: Visually-Guided Audio Generation**
  - **I Hear Your True Colors: Image Guided Audio Generation** - lab/im2wav/) [Code](https://github.com/RoySheffer/im2wav) | ICASSP 2023 |
  - **FoleyGen: Visually-Guided Audio Generation**
  - **Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models** - foley.github.io/) [Code](https://github.com/luosiallen/Diff-Foley) | NeurIPS 2023 |
  - **Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos** - Ning Hsu, David Harwath, Kristen Grauman | [Project Page](https://vision.cs.utexas.edu/projects/action2sound/) [Code](https://github.com/ChanganVR/action2sound) | ECCV 2024 |
  - **StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing** - Hsuan Yang, Chenggang Yan, Qingming Huang | [Code](https://github.com/GalaxyCong/StyleDubber) | ACL 2024 |
  - **EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing**
  - **Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos** - Ning Hsu, David Harwath, Kristen Grauman | [Project Page](https://vision.cs.utexas.edu/projects/action2sound/) [Code](https://github.com/ChanganVR/action2sound) | ECCV 2024 |
  - **StyleDubber: Towards Multi-Scale Style Learning for Movie Dubbing** - Hsuan Yang, Chenggang Yan, Qingming Huang | [Code](https://github.com/GalaxyCong/StyleDubber) | ACL 2024 |
  - **EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing**
  - **Learning to Dub Movies via Hierarchical Prosody Models** - Hsuan Yang, Qingming Huang | [Code](https://github.com/GalaxyCong/HPMDubbing) | CVPR 2023 |
  - **From Speaker to Dubber: Movie Dubbing with Prosody and Duration Consistency Learning**
  - **Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model** - lab/video2music) [Project Page](https://amaai-lab.github.io/Video2Music/) [Code](https://github.com/amaai-lab/video2music) [Dataset](https://zenodo.org/records/10057093) | |
  - **V2Meow: Meowing to the Visual Beat via Video-to-Music Generation** - research.github.io/noise2music/v2meow/) | AAAI 2024 |
  - **MeLFusion: Synthesizing Music from Image and Language Cues using Diffusion Models** - my.sharepoint.com/personal/sanjoyc_umd_edu/_layouts/15/onedrive.aspx?id=%2Fpersonal%2Fsanjoyc%5Fumd%5Fedu%2FDocuments%2FMeLFusion%20datasets&ga=1) | CVPR 2024 |
  - **Diff-Foley: Synchronized Video-to-Audio Synthesis with Latent Diffusion Models** - foley.github.io/) [Code](https://github.com/luosiallen/Diff-Foley) | NeurIPS 2023 |
  - **Learning to Dub Movies via Hierarchical Prosody Models** - Hsuan Yang, Qingming Huang | [Code](https://github.com/GalaxyCong/HPMDubbing) | CVPR 2023 |
- Others
  - **TransPixar: Advancing Text-to-Video Generation with Transparency** - Hsien Wang, Zhifei Zhang, He Zhang, Zhe Lin, Yingcong Chen | [Project Page](https://wileewang.github.io/TransPixar/) [Code](https://github.com/wileewang/TransPixar) | |
  - **LayerAnimate: Layer-specific Control for Animation**
  - **Re:Draw -- Context Aware Translation as a Controllable Method for Artistic Production**
  - **DrawingSpinUp: 3D Animation from Single Character Drawings** - Ling Lam, Hongbo Fu | [Code](https://github.com/LordLiang/DrawingSpinUp) | Siggraph Asia 2024 |
  - **Re:Draw -- Context Aware Translation as a Controllable Method for Artistic Production**
  - **Scaling Concept With Text-Guided Diffusion Models**
  - **Generative Omnimatte: Learning to Decompose Video into Layers** - Chih Lee, Erika Lu, Sarah Rumbley, Michal Geyer, Jia-Bin Huang, Tali Dekel, Forrester Cole | [Project Page](https://gen-omnimatte.github.io/) | |
  - **Scaling Concept With Text-Guided Diffusion Models**
  - **Generative Omnimatte: Learning to Decompose Video into Layers** - Chih Lee, Erika Lu, Sarah Rumbley, Michal Geyer, Jia-Bin Huang, Tali Dekel, Forrester Cole | [Project Page](https://gen-omnimatte.github.io/) | |
  - **LayerAnimate: Layer-specific Control for Animation**
  - **TransPixar: Advancing Text-to-Video Generation with Transparency** - Hsien Wang, Zhifei Zhang, He Zhang, Zhe Lin, Yingcong Chen | [Project Page](https://wileewang.github.io/TransPixar/) [Code](https://github.com/wileewang/TransPixar) | |
  - **Collaborative Neural Rendering using Anime Character Sheets** - research/IJCAI2023-CoNR) [Dataset](https://github.com/P2Oileen/CoNR_Dataset) | IJCAI 2023 |
  - **DrawingSpinUp: 3D Animation from Single Character Drawings** - Ling Lam, Hongbo Fu | [Code](https://github.com/LordLiang/DrawingSpinUp) | Siggraph Asia 2024 |
  - **Collaborative Neural Rendering using Anime Character Sheets** - research/IJCAI2023-CoNR) [Dataset](https://github.com/P2Oileen/CoNR_Dataset) | IJCAI 2023 |
  - **Sprite-from-Sprite: Cartoon Animation Decomposition with Self-supervised Sprite Estimation** - Tsin Wong, Yuxin Liu | [Code](https://lllyasviel.github.io/GitPageToonDecompose/) | ACM 2022 |
  - **Toonsynth: Example-based Synthesis of Hand-Colored Cartoon Animations**
♥️ Contributors
- Others
  - Yizhi Song
  - Liu He
  - Xinyang Li
  - Chenliang Xu
  - Junjia Guo
  - Zhiyuan Wang
  - Hang Hua
  - Jia-Xing Zhong
  - Yunzhong Xiao
  - Zhiyuan Wang
  - Hang Hua
  - Jia-Xing Zhong
  - Yunzhong Xiao
  - Yizhi Song
  - Liu He
  - Xinyang Li
  - Chenliang Xu
📊 Datasets
- Others
  - **SSH: A Self-Supervised Framework for Image Harmonization** - Group/SSHarmonization) [Dataset](https://drive.google.com/file/d/1jBx-DBtRX8GaqMvMv-CZutK4jn9tz-fT/view) | ICCV 2021 |
  - **Intrinsic Image Harmonization** - pa_9BNgIkuR0j1gcCxh8GI3XSWZN0e7/view?usp=sharing) | CVPR 2021 |
  - **ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Video**
  - **Sakuga-42M Dataset: Scaling Up Cartoon Research** - v1.0) | arXiv 2024 |
  - **ANIM-400K: A Large-Scale Dataset for Automated End-To-End Dubbing of Video**
  - **Sakuga-42M Dataset: Scaling Up Cartoon Research** - v1.0) | arXiv 2024 |
  - **Anisora: Exploring the frontiers of animation video generation in the sora era** - anisora) | |
  - **Alchemist: Parametric Control of Material Properties with Diffusion Models**
  - **Intrinsic Image Harmonization** - pa_9BNgIkuR0j1gcCxh8GI3XSWZN0e7/view?usp=sharing) | CVPR 2021 |
  - **Learning Inclusion Matching for Animation Paint Bucket Colorization**
  - **Deep Animation Video Interpolation in the Wild** - g/view) [Data (OneDrive)](https://entuedu-my.sharepoint.com/:u:/g/personal/siyao002_e_ntu_edu_sg/EY3SG0-IajxKj9HMPz__zOMBvyJdrA-SlwpyHYFkDsQtng?e=q7nGlu) [Video Demo](https://www.youtube.com/watch?v=2bbujT-ZXr8) | CVPR 2021 |
  - **AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies**
  - **AnimeRun: 2D Animation Visual Correspondence from Open Source 3D Movies**
  - **Anisora: Exploring the frontiers of animation video generation in the sora era** - anisora) | |
  - **V2C: Visual Voice Cloning**
  - **DoveNet: Deep Image Harmonization via Domain Verification** - Harmonization-Dataset-iHarmony4) [Demo](https://bcmi.sjtu.edu.cn/home/niuli/demo_image_composition/) [Dataset (Baidu Cloud)](https://pan.baidu.com/s/1xEN0Xrv_MbuKT0ZqsipeEg) [Dataset (OneDrive)](https://1drv.ms/f/s!AohNSvvkuxZmgTHOraRzo5-X3nMp?e=bQQKkR) | CVPR 2020 |
  - **V2C: Visual Voice Cloning**
  - **Alchemist: Parametric Control of Material Properties with Diffusion Models**
  - **Deep Geometrized Cartoon Line Inbetweening** - LsqFKpI&feature=youtu.be) [Code](https://github.com/lisiyao21/AnimeInbet) [Dataset](https://drive.google.com/file/d/1SNRGajIECxNwRp6ZJ0IlY7AEl2mRm2DR/view) | ICCV 2023 |
  - **Learning Inclusion Matching for Animation Paint Bucket Colorization**
  - **Deep Geometrized Cartoon Line Inbetweening** - LsqFKpI&feature=youtu.be) [Code](https://github.com/lisiyao21/AnimeInbet) [Dataset](https://drive.google.com/file/d/1SNRGajIECxNwRp6ZJ0IlY7AEl2mRm2DR/view) | ICCV 2023 |
  - **Deep Animation Video Interpolation in the Wild** - g/view) [Data (OneDrive)](https://entuedu-my.sharepoint.com/:u:/g/personal/siyao002_e_ntu_edu_sg/EY3SG0-IajxKj9HMPz__zOMBvyJdrA-SlwpyHYFkDsQtng?e=q7nGlu) [Video Demo](https://www.youtube.com/watch?v=2bbujT-ZXr8) | CVPR 2021 |
🌟 Star History
- Others
  - ![Star History Chart - history.com/#yunlong10/Awesome-AI4Animation&Date)
  - ![Star History Chart - history.com/#yunlong10/Awesome-AI4Animation&Date)
  - ![Star History Chart - history.com/#yunlong10/Awesome-AI4Animation&Date)

Programming Languages

Categories

🛠️ Methods 209 📊 Datasets 22 ♥️ Contributors 17 🌟 Star History 3

Sub Categories

2️⃣ Production 96 Others 58 3️⃣ Post-Production 49 1️⃣ Pre-production 48