{"id":15127671,"url":"https://github.com/numessanguis/facsvatar","last_synced_at":"2025-04-05T15:09:23.698Z","repository":{"id":94696387,"uuid":"114080451","full_name":"NumesSanguis/FACSvatar","owner":"NumesSanguis","description":"An Open Source Modular Framework From Face to FACS Based Avatar Animation (Unity3D / Blender)","archived":false,"fork":false,"pushed_at":"2023-06-29T02:51:34.000Z","size":131102,"stargazers_count":442,"open_issues_count":25,"forks_count":102,"subscribers_count":25,"default_branch":"master","last_synced_at":"2025-04-05T15:08:57.564Z","etag":null,"topics":["animation","avatar-animation","blender","dnn","docker-container","facial-expressions","facs","modular","openface","pyzmq","unity3d","zeromq"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"lgpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/NumesSanguis.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-12-13T05:59:42.000Z","updated_at":"2025-03-25T08:18:25.000Z","dependencies_parsed_at":"2023-03-05T10:45:26.720Z","dependency_job_id":"2ff140c3-1ef7-4cc7-ae42-2149581b47bc","html_url":"https://github.com/NumesSanguis/FACSvatar","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumesSanguis%2FFACSvatar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumesSanguis%2FFACSvatar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumesSanguis%2FFACSvatar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/NumesSanguis%2FFACSvatar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/NumesSanguis","download_url":"https://codeload.github.com/NumesSanguis/FACSvatar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247353749,"owners_count":20925329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["animation","avatar-animation","blender","dnn","docker-container","facial-expressions","facs","modular","openface","pyzmq","unity3d","zeromq"],"created_at":"2024-09-26T02:05:16.079Z","updated_at":"2025-04-05T15:09:23.673Z","avatar_url":"https://github.com/NumesSanguis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"Notes 2023-01-12\n- I wanted to simplify the quick start and smooth some rough edges before releasing version v0.4.0, but life held me up and I never merged this into master/main branch. This version is in all aspects better than v0.3.4, so I'm merging it after all in its current state.\n- Version v0.5.0 is on its way though (no release date yet). Small spoilers:\n    - Python version 3.10+\n    - ~~Support for Blender 3.3 LTS~~ (2023-06-17) Updated FACSvatar-Blender Blender add-on to support Blender LTS v3.3 \u0026 v3.6\n    - Will have a proper GUI (written in Vue3 (JavaScript Framework)) that communicates with Python.\n\n\n# What is FACSvatar? (v0.4.0-Alpha)\n\u003e FACSvatar is An Open Source Modular Framework for Real-Time FACS based Facial Animation\n\nOr in plain English:\n\n\u003e Track facial expressions with any software and visualize that data on any avatar in real-time,\npowered by the FACS representation.\nNo more need to modify your avatar to support your tracking software.\nAll written in your favorite programming language, on any OS, and across machines.\n\n![Diagram FACS advantage](docs/img/why_FACS_control_trans.png \"FACS advantage\")\nMuscle [image source](https://pixabay.com/illustrations/skull-anatomy-skull-and-crossbones-3012250/).\n\n- Facial Action Coding System (**FACS**): A description of how muscle groups in the human face contract/relax\n  to make any facial configuration possible.\n  ([learn more](https://facsvatar.readthedocs.io/en/latest/facs_theory.html)).\n  * Action Unit (**AU**): The strength of contraction of a single muscle group.\n- **Modular**: Software and OS independent. You only need to know what data goes in and what comes out.\n- **Extendable**: Write your code, add a ZeroMQ message socket, and let it talk to other modules.\n- **Real-time**: Create lively avatars that respond to your user.\n- **Machine/Deep Learning**: Input/output data-fied facial configurations.\n\n[![FACSvatar demo 2018-09](https://img.youtube.com/vi/J2FvrIl-ypU/0.jpg)](https://www.youtube.com/watch?v=J2FvrIl-ypU)\n\n(Above demo video link: https://www.youtube.com/watch?v=J2FvrIl-ypU)\n\nMessage to:\n- **Animators**: Copy facial expressions from a video/webcam to your avatar.\n- **Affective Computing**: Enable Human-Agent Interaction (HAI) by inputting your human-analysis into a ML-model,\noutput FACS values, and have your Embodied Conversational Agent (ECA) display it.\n- **Psychologists**: Create stimuli with the same facial configurations across avatars of different sex, age and ethnicity.\n\nFACSvatar is already operable with:\n- Tracking software:\n  * [OpenFace](https://github.com/TadasBaltrusaitis/OpenFace): Extract facial AUs from videos/webcam.\n- Visualization software:\n  * [Blender 2.80+](https://www.blender.org/)\n    * [MB-Lab](https://mb-lab-community.github.io/MB-Lab.github.io/) 3D avatar generation\n  * [Unity3D](https://unity.com/)\n  * [FACSHuman](https://www.michaelgilbert.fr/facshuman/)\n- Modules for additional data processing, and allowing `m trackers - to - n avatars` (`modules` folder)\n- [ZeroMQ](https://zeromq.org/): This framework's glue, allowing modules to communicate with each other.\n- [Containerization with Docker](https://www.docker.com/) to run FACSvatar modules everywhere.\n\n**Disclaimers**: This is an open-source project, hopefully being flexible enough for your facial animation needs.\nThis is not software supported by a company / commercially, but by users like you.\nIf you need some new capability, you likely have to code it yourself (or ask/hire someone),\nbut questions for guidance are always welcome (make a [GitHub issue](https://github.com/NumesSanguis/FACSvatar/issues))!\nFor commercial usage, please check the [license page](https://facsvatar.readthedocs.io/en/latest/misc/license.html).\nRead more about FACSvatar's limitations (TODO doc link).\n\n## Full documentation\nRead the Docs: https://facsvatar.readthedocs.io/\n\n## Paper\nPlease cite the following paper when using this framework in a paper:\n\n[van der Struijk, Stef and Huang, Hung-Hsuan and Mirzaei, Maryam Sadat and Nishida, Toyoaki \"FACSvatar: An Open Source Modular Framework for Real-Time FACS based Facial Animation\" In Proceedings of 18th ACM International Conference on Intelligent Virtual Agents (pp. 159-164). ACM, 2018.](https://dl.acm.org/citation.cfm?id=3267918)\n\n- DOI: https://doi.org/10.1145/10.1145/3267851.3267918\n- ISBN: 978-1-4503-6013-5/18/11\n\n## New in v0.4.0-alpha (2020-07-??)  TODO UNFINISHED\n\n* COMPLETE re-write of the documentation: [Check it out!](https://facsvatar.readthedocs.io/en/v0.4.0/)\n* Python modules:\n    * Standardization pass over all modules / code clean-up\n    * Consistency fix: ROUTER / DEALER sockets use JSON formatted data\n    * DOC string per class and function\n    * Logger instead of print() statements\n    * Debug as option to enable logger\n    * File structure for proper import of modules / pip?\n    * Use config file (in addition to command line arguments) + config filepath argument\n* Easy run: Docker container per module + Docker Compose\n* Demo video\n\nSee [all changelogs](https://facsvatar.readthedocs.io/en/latest/changelog.html)\n\n\n# Quickstart\nFACSvatar is tested on Ubuntu and Windows, but should work on MacOS.\n\nThis quickstart has 2 parts:\n1. Start FACSvatar modules using Docker - modules in containers\n(see [here for Python instructions](https://facsvatar.readthedocs.io/en/latest/firstrun))\n2. Visualize in Unity3D or Blender\n\n## Dockerized modules\n\u003c!-- \u003cdetails\u003e\u003csummary\u003eExpand (CLICK ME)\u003c/summary\u003e\n\u003cp\u003e --\u003e\n\n0. Downloads - Go to the [release page of this GitHub repo](https://github.com/NumesSanguis/FACSvatar/releases) and download:\n    * (Real-time only) openface_2.1.0_zeromq.zip\n        * Unzip and execute `download_models.sh or .ps1` to download trained models\n    * Windows 7 / 8 / 10 Home version \u003c2004 : unity_FACSvatar_standalone_docker-ip.zip\n    * Windows 10 Home v2004+ / Pro / Enterprise / Education: unity_FACSvatar_standalone.zip\n    * Windows / Linux / Mac: [Unity3D editor (documentation)](https://facsvatar.readthedocs.io/en/latest/defaultsetup.html#unity3d-game-engine)\n    * Source code (zip / tar.gz) or download this repository with:\n        * `git clone https://github.com/NumesSanguis/FACSvatar.git`\n        * Press the green `Clone or Download` button on this page --\u003e `Download ZIP`\n\n1. Docker Install - Let's you execute applications without worrying about OS or programming language.\n    * [General Docker instructions](https://docs.docker.com/install/#supported-platforms)\n    * [Docker Toolbox for Windows 7/8/10 Home version \u003c2004](https://docs.docker.com/toolbox/overview/)\n    * [Docker for Windows 10 Home v2004+](https://docs.docker.com/docker-for-windows/install-windows-home/)\n    * [Docker for Windows 10 Pro, Enterprise or Education](https://docs.docker.com/docker-for-windows/install/#what-to-know-before-you-install)\n    * Ubuntu: [Docker](https://docs.docker.com/install/linux/docker-ce/ubuntu/) and [docker-compose](https://docs.docker.com/compose/install/) and `sudo usermod -a -G docker $USER`\n\n2. Docker Modules - Open a terminal (W7/8: cmd.exe / W10: PowerShell) and navigate to folder `FACSvatar/modules`, then execute:\n    1. `docker-compose pull`  (Downloads FACSvatar Docker containers)\n    2. `docker-compose up`  (Starts downloaded Docker containers)\n    \n3.  See visualization engine instructions\n\n### Offline version:\n\n4. Open a 2nd terminal in folder `FACSvatar/modules` and execute: `docker-compose exec facsvatar_facsfromcsv bash`\n5. Inside Docker container - Start facial animation with: `python main.py --pub_ip facsvatar_bridge`\n\n### With webcam for real-time (Windows-only for now):\n\n4. Navigate inside folder `openface_x.x.x_zeromq`\n5. (Windows 7/8/10 Home version \u003c2004 - only) Get Docker machine ip by opening a 2nd terminal and execute: `docker-machine ip` (likely to be 192.168.99.100)\n6. (Windows 7/8/10 Home version \u003c2004 - only) Open `config.xml`, change `\u003cIP\u003e127.0.0.1\u003c/IP\u003e` to `\u003cIP\u003emachine ip from step 3\u003c/IP\u003e` (`\u003cIP\u003e192.168.99.100\u003c/IP\u003e`) and save and close.\n7. Double click `OpenFaceOffline.exe` –\u003e menu: File –\u003e Open Webcam\n\n\u003c!--  \u003c/p\u003e\n\u003c/details\u003e  --\u003e\n\n\n## Visualization engines\n### Unity3D\nTested on version: 2018.2.20f1\n\n1. Open the folder `unity_FACSvatar` as a project with Unity3D\n2. Press play (now it's waiting for facial data)\n\nOR (Windows-only TODO):\n\n1. Navigate inside unzipped folder unity_FACSvatar_standalone(_docker-ip) and double-click `unity_FACSvatar.exe`\n\nExtra: Use the numbers 0, 1, 2 on your keyboard to change camera.\n\n### FACSvatar Blender add-on\nFollow instructions here: https://github.com/NumesSanguis/FACSvatar-Blender\n\n## Quickstart video\nSee the quickstart video (:warning: note that the **Blender script part is outdated** (from 15:15) due the new FACSvatar Blender add-on):\n\n[![FACSvatar Quickstart 2019-01 (v0.3.4)](https://img.youtube.com/vi/OOoXDfkn8fk/0.jpg)](https://www.youtube.com/watch?v=OOoXDfkn8fk)\n\n# Find out more\n## Full documentation\n[Read the FACSvatar documentation](https://facsvatar.readthedocs.io/en/latest/)!\n\n## 2017 promotion poster (English \u0026 日本語)\n[![FACSvatar details in English and 日本語](https://surafusoft.eu/facsvatar/files/2018/10/FACSvatar_poster_25_A4-724x1024.png)](https://surafusoft.eu/facsvatar/files/2018/10/FACSvatar_poster_25_A4.png)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumessanguis%2Ffacsvatar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnumessanguis%2Ffacsvatar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnumessanguis%2Ffacsvatar/lists"}