Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/vincentbonnetcg-zz/awesome-ml-character

A collection of papers about characters generation with machine learning
https://github.com/vincentbonnetcg-zz/awesome-ml-character
List: awesome-ml-character
awesome awesome-list character machine-learning neural-network
Last synced: about 1 month ago
JSON representation
A collection of papers about characters generation with machine learning
Host: GitHub
URL: https://github.com/vincentbonnetcg-zz/awesome-ml-character
Owner: vincentbonnetcg-zz
License: cc0-1.0
Created: 2021-07-29T08:03:50.000Z (over 3 years ago)
Default Branch: main
Last Pushed: 2022-04-08T20:29:07.000Z (over 2 years ago)
Last Synced: 2024-05-19T18:15:20.424Z (7 months ago)
Topics: awesome, awesome-list, character, machine-learning, neural-network
Homepage:
Size: 117 KB
Stars: 19
Watchers: 3
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: readme.md
- Contributing: contributing.md
- License: LICENSE
- Code of conduct: code-of-conduct.md
Awesome Lists containing this project

more-awesome - ML Character - Digital characters with machine learning. (Data Science)
ultimate-awesome - awesome-ml-character - A collection of papers about characters generation with machine learning. (Other Lists / Monkey C Lists)
README

        # Awesome ML Character 

> A collection of papers about characters generation with machine learning

[![Awesome](https://awesome.re/badge.svg)](https://awesome.re)

[![made-with-Markdown](https://img.shields.io/badge/Made%20with-Markdown-1f425f.svg)](http://commonmark.org)

## Contents

- [Non-Machine Learning](#non-machine-learning)

- [Parametric Model](#parametric-model)

- [Sparse Decomposition](#sparse-decomposition)

- [Avatar Creation](#avatar-creation)

- [Rigging](#rigging)

- [Deformation](#deformation)

- [Body Simulation](#body-simulation)

- [Cloth Simulation](#cloth-simulation)

- [Motion Controller](#motion-controller)

- [Motion Generation](#motion-generation)

- [Motion Inbetweening](#motion-inbetweening)

- [Motion Retargeting](#motion-retargeting)

- [Dataset Generation](#dataset-generation)

- [Datasets](#datasets)

## Non-Machine Learning

It's odd that non-ML is included in the ML list. The non-ML papers serve as a foundation and important to know.

### Skinning

**Real-time skeletal skinning with optimized centers of rotation.**


*Le, Binh Huy, and Jessica K. Hodgins*


2016. [[PDF](https://s3-us-west-1.amazonaws.com/disneyresearch/wp-content/uploads/20160705174939/Real-time-Skeletal-Skinning-with-Optimized-Centers-of-Rotation-Paper.pdf)]

**Learning from the Artist: Theory and Practice of Example-Based Character Deformation.**


*J Lewis*


2016. [[PDF](https://researcharchive.vuw.ac.nz/xmlui/bitstream/handle/10063/5255/thesis.pdf)]

**SMPL: A Skinned Multi-Person Linear Model.**


*Loper, Matthew, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black*


2015. [[PDF](https://files.is.tue.mpg.de/black/papers/SMPL2015.pdf)]

**Delta Mush: smoothing deformations while preserving detail.**


*J Mancewicz, ML Derksen, H Rijpkema, et al.*


2014. [[PDF](https://on-demand.gputechconf.com/gtc/2015/presentation/S5641-Joe-Mancewicz.pdf)]

**Pose-space animation and transfer of facial details.**


*Bickel, Bernd, Manuel Lang, Mario Botsch, Miguel A. Otaduy, and Markus Gross*


2008. [[PDF](https://www.gmrv.es/Publications/2008/BLBOG08/BLBOG08.pdf)]

**Skinning with dual quaternions.**


*Kavan, Ladislav, Steven Collins, Jiří Žára, and Carol O'Sullivan*


2007. [[PDF](https://www.cs.utah.edu/~ladislav/kavan07skinning/kavan07skinning.pdf)]

**Real‐time weighted pose‐space deformation on the GPU.**


*Rhee, Taehyun, John P. Lewis, and Ulrich Neumann*


2006. [[PDF](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.135.2149&rep=rep1&type=pdf)]

**Pose space deformation: a unified approach to shape interpolation and skeleton-driven deformation.**


*Lewis, John P., Matt Cordner, and Nickson Fong*


2000. [[PDF](http://scribblethink.org/Work/PSD/PSD.pdf)]

**Joint-dependent local deformations for hand animation and object grasping.**


*Magnenat-Thalmann, Nadia, Richard Laperrire, and Daniel Thalmann*


1988. [[PDF](https://graphicsinterface.org/wp-content/uploads/gi1988-4.pdf)]

### Physics Simulation

**Enriching facial blendshape rigs with physical simulation.**


*Kozlov, Yeara, Derek Bradley, Moritz Bächer, Bernhard Thomaszewski, Thabo Beeler, and Markus Gross*


2017. [[PDF](http://disneyresearch.s3.amazonaws.com/wp-content/uploads/20170425165601/Enriching-Facial-Blendshape-Rigs-with-Physical-Simulation-Paper2.pdf)]

**Blendforces: A dynamic framework for facial animation.**


*Barrielle, Vincent, Nicolas Stoiber, and Cédric Cagniart*


2016. [[PDF](https://www.researchgate.net/profile/Nicolas-Stoiber/publication/303443665_BlendForces_A_Dynamic_Framework_for_Facial_Animation/links/574309c308ae9f741b379c05/BlendForces-A-Dynamic-Framework-for-Facial-Animation.pdf)]

**Reconstructing personalized anatomical models for physics-based body animation.**


*Kadleček, Petr, Alexandru-Eugen Ichim, Tiantian Liu, Jaroslav Křivánek, and Ladislav Kavan*


2016. [[PDF](https://www.cs.utah.edu/~ladislav/kadlecek16reconstructing/kadlecek16reconstructing.pdf)]

**Efficient elasticity for character skinning with contact and collisions.**


*McAdams, Aleka, Yongning Zhu, Andrew Selle, Mark Empey, Rasmus Tamstorf, Joseph Teran, and Eftychios Sifakis*


2011. [[PDF](https://www.andyselle.com/papers/18/elasticity-skinning.pdf)]

**Efficient Simulation of Inextensible Cloth.**


*Goldenthal, Rony, David Harmon, Raanan Fattal, Michel Bercovier, and Eitan Grinspun*


2007. [[PDF](http://www.cs.columbia.edu/cg/pdfs/131-ESIC.pdf)]

**Invertible Finite Elements For Robust Simulation of Large Deformation.**


*Irving, Geoffrey, Joseph Teran, and Ronald Fedkiw*


2004. [[PDF](https://www.math.ucla.edu/~jteran/papers/ITF04.pdf)]

### Miscellaneous

**Doug Roble - YouTube channel**


[[Channel](https://www.youtube.com/c/DougRoble)]

**Ladislav Kavan - YouTube channel**


[[Channel](https://www.youtube.com/user/kavanl1)]

## Parametric Model

**Learning a model of facial shape and expression from 4D scans.**


*Li, Tianye, Timo Bolkart, Michael J. Black, Hao Li, and Javier Romero*


2017. [[PDF](https://www.is.mpg.de/uploads_file/attachment/attachment/400/paper.pdf)]

**Building Statistical Shape Spaces for 3D Human Modeling.**


*Pishchulin, Leonid, Stefanie Wuhrer, Thomas Helten, Christian Theobalt, and Bernt Schiele.*


2017. [[PDF](https://arxiv.org/pdf/1503.05860)]

**SMPL: A skinned multi-person linear model.**


*Loper, Matthew, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black.*


SIGGRAPH 2015. [[PDF](https://files.is.tue.mpg.de/black/papers/SMPL2015.pdf)][[Course](https://smpl-made-simple.is.tue.mpg.de/index.html)]

**Parametric modeling of 3D human body shape—A survey.**


*Cheng, Zhi-Quan, Yin Chen, Ralph R. Martin, Tong Wu, and Zhan Song.*


2015. [[Website](https://www.sciencedirect.com/science/article/abs/pii/S0097849317301929)]

**A Statistical Model of Human Pose and Body Shape.**


*Hasler, Nils, Carsten Stoll, Martin Sunkel, Bodo Rosenhahn, and H‐P. Seidel.* 


2009. [[PDF](https://www.cs.princeton.edu/courses/archive/spr11/cos598A/pdfs/Hasler09.pdf)]

**SCAPE: Shape Completion and Animation of People.**


*Anguelov, Dragomir, Praveen Srinivasan, Daphne Koller, Sebastian Thrun, Jim Rodgers, and James Davis.*


SIGGRAPH 2005. [[PDF](https://ai.stanford.edu/~drago/Papers/shapecomp.pdf)]

**The space of human body shapes: reconstruction and parameterization from range scans.**


*Allen, Brett, Brian Curless, and Zoran Popović.*


2003. [[PDF](https://www.is.mpg.de/uploads_file/attachment/attachment/400/paper.pdf)]

**A morphable model for the synthesis of 3D faces.**


*Blanz, Volker, and Thomas Vetter.*


1999. [[PDF](https://www.face-rec.org/algorithms/3d_morph/morphmod2.pdf)]

## Sparse Decomposition

**Generating 3D faces using convolutional mesh autoencoders.**


*Ranjan, Anurag, Timo Bolkart, Soubhik Sanyal, and Michael J. Black.*


2018. [[PDF](https://arxiv.org/pdf/1807.10267.pdf)]

**Mesh-based Autoencoders for Localized Deformation Component Analysis.**


*Tan, Qingyang, Lin Gao, Yu-Kun Lai, Jie Yang, and Shihong Xia*.


2018.[[PDF](https://arxiv.org/pdf/1709.04304.pdf)]

**Sparse Data Driven Mesh Deformation.**


*Lin Gao, Yu-Kun Lai, Jie Yang, Ling-Xiao Zhang, Leif Kobbelt, Shihong Xia.*


2017. [[PDF](https://arxiv.org/pdf/1709.01250)]

**Linear Shape Deformation Models with Local Support using Graph-based Structured Matrix Factorisation.**


*Bernard, Florian, Peter Gemmar, Frank Hertel, Jorge Goncalves, and Johan Thunberg.*


2016. [[PDF](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Bernard_Linear_Shape_Deformation_CVPR_2016_paper.pdf)]

**Sparse Localized Decomposition of Deformation Gradients.**


*Huang, Zhichao, Junfeng Yao, Zichun Zhong, Yang Liu, and Xiaohu Guo.*


SPLOCS with deformation gradients


2014. [[PDF](https://personal.utdallas.edu/~xxg061000/CGF14.pdf)]

**Sparse localized deformation components.**


Introduces SPLOCS.


*Neumann, Thomas, Kiran Varanasi, Stephan Wenger, Markus Wacker, Marcus Magnor, and Christian Theobalt.*


2013. [[PDF](https://vcai.mpi-inf.mpg.de/files/splocs.pdf)]

## Avatar Creation

**SMPLpix: Neural Avatars from 3D Human Models.**


*Prokudin, Sergey, Michael J. Black, and Javier Romero.*


2021.  [[PDF](https://arxiv.org/pdf/2008.06872.pdf)][[GitHub](https://github.com/sergeyprokudin/smplpix)]

**Expressive body capture: 3d hands, face, and body from a single image.**


*Pavlakos, Georgios, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J. Black.*


2019. [[PDF](https://arxiv.org/pdf/1904.05866.pdf)][[GitHub](https://github.com/vchoutas/smplify-x)]

**Direct Manipulation Blendshapes.**


*Lewis, John P., and Ken-ichi Anjyo.*


2010. [[PDF](http://scribblethink.org/Work/DirectManipBlendshapes/DMBpreprint.pdf)]

## Rigging

**Learning Skeletal Articulations with Neural Blend Shapes.**


*Li, Peizhuo, Kfir Aberman, Rana Hanocka, Libin Liu, Olga Sorkine-Hornung, and Baoquan Chen.*


SIGGRAPH 2021. [[PDF](https://arxiv.org/pdf/2105.02451)]

**HeterSkinNet: A Heterogeneous Network for Skin Weights Prediction.**


*Pan, Xiaoyu, Jiancong Huang, Jiaming Mai, He Wang, Honglin Li, Tongkui Su, Wenjun Wang, and Xiaogang Jin.*


I3D 2021. [[PDF](https://arxiv.org/pdf/2103.10602)]

**Rignet: Neural rigging for articulated characters.**


*Xu, Zhan, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh.*


SIGGRAPH 2020. [[PDF](https://arxiv.org/pdf/2005.00559)]

**NiLBS: Neural Inverse Linear Blend Skinning.**


*Jeruzalski, Timothy, David IW Levin, Alec Jacobson, Paul Lalonde, Mohammad Norouzi, and Andrea Tagliasacchi.*


SIGGRAPH 2020. [[PDF](https://arxiv.org/pdf/2004.05980)]

## Deformation

**PBNS: Physically Based Neural Simulator for Unsupervised Garment Pose Space Deformation.**


*Bertiche, Hugo, Meysam Madadi, and Sergio Escalera.*


2020.[[PDF](https://arxiv.org/pdf/2012.11310)]

**Fast and deep deformation approximations.**


*Bailey, Stephen W., Dave Otte, Paul Dilorenzo, and James F. O'Brien.*


2018.[[PDF](https://research.dreamworks.com/wp-content/uploads/2018/08/Rig_Approximation-Edited.pdf)]

**Fast and deep facial deformations.**


*Bailey, Stephen W., Dalton Omens, Paul Dilorenzo, and James F. O'Brien.*


2020. [[PDF](http://graphics.berkeley.edu/papers/Bailey-FDF-2020-07/Bailey-FDF-2020-07.pdf)]

## Body Simulation

**SoftSMPL: Data-driven Modeling of Nonlinear Soft-tissue Dynamics for Parametric Humans.**


*Santesteban, Igor, Elena Garces, Miguel A. Otaduy, and Dan Casas.*


Eurographics 2020.[[PDF](https://arxiv.org/pdf/2004.00326)]

## Cloth Simulation

**SNUG: Self-Supervised Neural Dynamic Garments.**


*Igor Santesteban, Miguel A. Otaduy, and Dan Casas.*


2022. [[PDF](http://mslab.es/projects/SNUG/contents/santesteban_CVPR2022.pdf)]

**Self-Supervised Collision Handling via Generative 3D Garment Models for Virtual Try-On.**


*Igor Santesteban, Nils Thuerey, Miguel A. Otaduy and Dan Casas.*


2021. [[PDF](http://mslab.es/projects/SelfSupervisedGarmentCollisions/contents/santesteban_CVPR2021.pdf)]

**Swish: Neural Network Cloth Simulation on Madden NFL 21.**


*Lewin, Chris, James Power, and James Cobb.*


2021.[[PDF](https://media.contentapi.ea.com/content/dam/ea/seed/presentations/seed-swish-cloth-simulation-madden-nfl.pdf)]

**TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style.**


*Patel, Chaitanya, Zhouyingcheng Liao, and Gerard Pons-Moll.*


2020. [[PDF](https://arxiv.org/pdf/2003.04583)]

**Learning Mesh-Based Simulation with Graph Networks.**


*Pfaff, Tobias, Meire Fortunato, Alvaro Sanchez-Gonzalez, and Peter W. Battaglia.*


2020. [[PDF](https://arxiv.org/pdf/2010.03409)]

**Subspace neural physics: Fast data-driven interactive simulation.**


*Holden, Daniel, Bang Chi Duong, Sayantan Datta, and Derek Nowrouzezahrai..*


2020. [[PDF](http://cim.mcgill.ca/~derek/files/Deep-Cloth-paper.pdf)]

## Motion Controller

### Data-Driven Controller

**ProtoRes: Proto-Residual Architecture for Deep Modeling of Human Pose.**


*Oreshkin, Boris N., Florent Bocquelet, Félix H. Harvey, Bay Raitt, and Dominic Laflamme.*


2021. [[PDF](https://arxiv.org/pdf/2106.01981)][[Website](https://unity-technologies.github.io/Labs/protores.html)]

**Neural animation layering for synthesizing martial arts movements.**


*Starke, Sebastian, Yiwei Zhao, Fabio Zinno, and Taku Komura.*


2021. [[PDF](http://www.ipab.inf.ed.ac.uk/cgvu/starke2021.pdf)]

**AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control.**


*Peng, Xue Bin, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa.*


2021. [[PDF](https://arxiv.org/pdf/2104.02180)]

**A GAN-Like Approach for Physics-Based Imitation Learning and Interactive Character Control.**


*Xu, Pei, and Ioannis Karamouzas.*


2021. [[PDF](https://arxiv.org/pdf/2105.10066)]

**Learned motion matching.**


*Xu, Pei, and Ioannis Karamouzas.*


2020. [[PDF](http://theorangeduck.com/media/uploads/other_stuff/Learned_Motion_Matching.pdf)]

**Local motion phases for learning multi-contact character movements.**


*Starke, Sebastian, Yiwei Zhao, Taku Komura, and Kazi Zaman.*


2020. [[PDF](https://www.pure.ed.ac.uk/ws/portalfiles/portal/157671564/Local_Motion_Phases_STARKE_DOA27042020_AFV.pdf)]

**Neural state machine for character-scene interactions.**


*Starke, Sebastian, He Zhang, Taku Komura, and Jun Saito.*


SIGGRAPH Asia 2019. [[PDF](https://www.pure.ed.ac.uk/ws/files/112627363/papers_168s4_file2.pdf)]

**Mode-adaptive neural networks for quadruped motion control .**


*Zhang, He, Sebastian Starke, Taku Komura, and Jun Saito.*


2018. [[PDF](https://www.pure.ed.ac.uk/ws/files/60838109/dog2.pdf)]

**Phase-functioned neural networks for character control.**


*Holden, Daniel, Taku Komura, and Jun Saito.*


2017. [[PDF](http://theorangeduck.com/media/uploads/other_stuff/phasefunction.pdf)]

### Physics-Based Controller

**CARL: Controllable Agent with Reinforcement Learning for Quadruped Locomotion.**


*Luo, Ying-Sheng, Jonathan Hans Soeseno, Trista Pei-Chun Chen, and Wei-Chao Chen.*


2020. [[PDF](https://arxiv.org/pdf/2005.03288)]

## Motion Generation

**From Motor Control to Team Play in Simulated Humanoid Football.**


*Liu, Siqi, Guy Lever, Zhe Wang, Josh Merel, S. M. Eslami, Daniel Hennes, Wojciech M. Czarnecki et al.*


2021.[[PDF](https://arxiv.org/pdf/2105.12196.pdf)]

**Deeploco: Dynamic locomotion skills using hierarchical deep reinforcement learning.**


*Peng, Xue Bin, Glen Berseth, KangKang Yin, and Michiel Van De Panne.*


2017.[[PDF](https://www.cs.ubc.ca/~van/papers/2017-TOG-deepLoco/2017-TOG-deepLoco.pdf)]

**Emergence of locomotion behaviours in rich environments.**


*Heess, Nicolas, Dhruva TB, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa et al..*


2017.[[PDF](https://arxiv.org/pdf/1707.02286)]

**Flexible muscle-based locomotion for bipedal creatures.**


*Geijtenbeek, Thomas, Michiel Van De Panne, and A. Frank Van Der Stappen*


2013.[[PDF](https://www.cs.ubc.ca/~van/papers/2013-TOG-MuscleBasedBipeds/2013-TOG-MuscleBasedBipeds.pdf)]

## Motion Inbetweening

**Recurrent transition networks for character locomotion.**


*Harvey, Félix G., and Christopher Pal*


SIGGRAPH Asia 2018.[[PDF](https://arxiv.org/pdf/1810.02363)]

**Robust motion in-betweening.**


*Harvey, Félix G., Mike Yurick, Derek Nowrouzezahrai, and Christopher Pal*


SIGGRAPH 2020.[[PDF](https://arxiv.org/pdf/2102.04942)]

## Motion Retargeting

**Skeleton-Aware Networks for Deep Motion Retargeting.**


*Aberman, Kfir, Peizhuo Li, Dani Lischinski, Olga Sorkine-Hornung, Daniel Cohen-Or, and Baoquan Chen*


SIGGRAPH 2020.[[PDF](https://arxiv.org/pdf/2005.05732)]

## Dataset Generation

**Learning from Synthetic Humans.**


*Varol, Gul, Javier Romero, Xavier Martin, Naureen Mahmood, Michael J. Black, Ivan Laptev, and Cordelia Schmid.*


2017. [[PDF](https://arxiv.org/pdf/1701.01370.pdf)]

## Datasets

**LAFAN1 - Ubisoft La Forge Animation Dataset.**


*Félix G. Harvey and Mike Yurick and Derek Nowrouzezahrai and Christopher Pal.*


2020. [[Website](https://github.com/ubisoft/ubisoft-laforge-animation-dataset)]

**MPII Human Pose - 2D Human Pose Estimation: New Benchmark and State of the Art Analysis.**


*Mykhaylo Andriluka and Leonid Pishchulin and Peter Gehler and Schiele, Bernt*.


2014. [[Website](http://human-pose.mpi-inf.mpg.de/)]

**Human3.6M Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments.**


*Ionescu, Catalin, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu.*


2013. [[Website](http://vision.imar.ro/human3.6m/description.php)]

**HumanEva I/II**
 

*Ionescu, Catalin, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu.*


[[Website](http://humaneva.is.tue.mpg.de/)]

**Caesar**


[[Website](https://humanshape.mpi-inf.mpg.de)]

**DensePose**


*Facebook Research*


50K humans, collecting more then 5 million manually annotated correspondences


[[Website](https://github.com/facebookresearch/DensePose)]

**SURREAL : Synthetic hUmans foR REAL tasks**


50K humans, collecting more then 5 million manually annotated correspondences


6M RGB frames of synthetic humans.


[[Website](https://www.di.ens.fr/willow/research/surreal/data/)]

## Interisting Links

- Dan Casas : http://dancasas.github.io

- Sebastian Starke : https://github.com/sebastianstarke/AI4Animation

- Computer Graphics in the Era of AI : http://cs348i.stanford.edu/

- https://khanhha.github.io/posts/3D-human-datasets/

- https://graphics.soe.ucsc.edu/data/BodyModels/index.html

- https://sric.me/Learning-to-Reconstruct-People/

## Contribute

Contributions welcome! Read the [contribution guidelines](contributing.md) first.