Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/lingdong-/handpose-facemesh-demos

🎥🤟 8 minimalistic templates for tfjs mediapipe handpose and facemesh
https://github.com/lingdong-/handpose-facemesh-demos

computer-vision handpose machine-learning mediapipe networking p5js tensorflowjs threejs

Last synced: 3 months ago
JSON representation

🎥🤟 8 minimalistic templates for tfjs mediapipe handpose and facemesh

Awesome Lists containing this project

README 

        
# Mediapipe Handpose and Facemesh Demos



## Table of Contnets



- [Handpose Demos](#handpose-demos)

- [Facemesh Demos](#facemesh-demos)



# Handpose Demos



![](images/icon-handpose.png)



Collection of minimalistic demos/templates using [mediapipe handpose with tensorflow.js](https://blog.tensorflow.org/2020/03/face-and-hand-tracking-in-browser-with-mediapipe-and-tensorflowjs.html). Created for and supported by [the Frank-Ratchye STUDIO for Creative Inquiry](https://studioforcreativeinquiry.org/).



> The handpose package detects hands in an input image or video stream, and returns twenty-one 3-dimensional landmarks locating features within each hand. Such landmarks include the locations of each finger joint and the palm. -- blog.tensorflow.org



There are 4 templates included in this repo, 2 of them with [three.js](https://threejs.org) and 2 with [p5.js](https://p5js.org), 2 are multiplyer via socket.io + express, and 2 are singleplayer static sites. They're all well commented and explained for beginners.



All demos are hosted on [glitch.com](https://glitch.com/) 🎏



## Online multiplayer 3D



### [Live](https://networked-hand-3js-tf174-handv1.glitch.me) | [Remix](https://glitch.com/edit/#!/networked-hand-3js-tf174-handv1) | [Source](./networked-hand-3js-tf174-handv1)



- tensorflow.js + mediapipe handpose

- three.js

- socket.io + express



## Online multiplayer 2D



### [Live](https://networked-hand-p5-tf174-handv1.glitch.me) | [Remix](https://glitch.com/edit/#!/networked-hand-p5-tf174-handv1) | [Source](./networked-hand-p5-tf174-handv1)



- tensorflow.js + mediapipe handpose

- p5.js

- socket.io + express



## Singleplayer 3D



### [Live](https://mediapipe-hand-3js-tf174-handv1.glitch.me) | [Remix](https://glitch.com/edit/#!/mediapipe-hand-3js-tf174-handv1) | [Source](./mediapipe-hand-3js-tf174-handv1)



- tensorflow.js + mediapipe handpose

- three.js



## Singleplayer 2D



### [Live](https://mediapipe-hand-p5-tf174-handv1.glitch.me) | [Remix](https://glitch.com/edit/#!/mediapipe-hand-p5-tf174-handv1) | [Source](./mediapipe-hand-p5-tf174-handv1)



- tensorflow.js + mediapipe handpose

- p5.js



Also available on editor.p5js.org: [https://editor.p5js.org/lingdong/sketches/1viPqbRMv](https://editor.p5js.org/lingdong/sketches/1viPqbRMv)



## 4 in 1



Glued together with iframes: [https://handpose-demos.glitch.me/](https://handpose-demos.glitch.me/)



# Facemesh Demos



![](images/icon-facemesh.png)



Collection of minimalistic demos/templates using [mediapipe facemesh with tensorflow.js](https://blog.tensorflow.org/2020/03/face-and-hand-tracking-in-browser-with-mediapipe-and-tensorflowjs.html). Created for and supported by [the Frank-Ratchye STUDIO for Creative Inquiry](https://studioforcreativeinquiry.org/).



> The facemesh package infers approximate 3D facial surface geometry from an image or video stream, requiring only a single camera input without the need for a depth sensor. -- blog.tensorflow.org



There are 4 templates included in this repo, 2 of them with [three.js](https://threejs.org) and 2 with [p5.js](https://p5js.org), 2 are multiplyer via socket.io + express, and 2 are singleplayer static sites. They're all well commented and explained for beginners.



All demos are hosted on [glitch.com](https://glitch.com/) 🎏



## Keypoint Subsets



There're 3 additional keypoint subsets available in each demo to select from, as alternatives to the facemesh 468 vertices. See image below for indices and triangulation for each of them:



![](images/facemesh-subsets.png)



From left to right:



- Minimal 7 points

- Important features 33 points

- Standard facial landmark 68 points

- Full facemesh 468 points. A larger resolution and flattened version of the illustration is available in [tfjs facemesh repo](https://github.com/tensorflow/tfjs-models/tree/master/facemesh).



## Online multiplayer 3D



### [Live](https://networked-facemesh-3js-tf2.glitch.me) | [Remix](https://glitch.com/edit/#!/networked-facemesh-3js-tf2) | [Source](./networked-facemesh-3js-tf2)



- tensorflow.js + mediapipe facemseh

- three.js

- socket.io + express



## Online multiplayer 2D



### [Live](https://networked-facemesh-p5-tf2.glitch.me) | [Remix](https://glitch.com/edit/#!/networked-facemesh-p5-tf2) | [Source](./networked-facemesh-p5-tf2)



- tensorflow.js + mediapipe facemseh

- p5.js

- socket.io + express



## Singleplayer 3D



### [Live](https://mediapipe-facemesh-3js-tf2.glitch.me) | [Remix](https://glitch.com/edit/#!/mediapipe-facemesh-3js-tf2) | [Source](./mediapipe-facemesh-3js-tf2)



- tensorflow.js + mediapipe facemseh

- three.js



## Singleplayer 2D



### [Live](https://mediapipe-facemesh-p5-tf2.glitch.me) | [Remix](https://glitch.com/edit/#!/mediapipe-facemesh-p5-tf2) | [Source](./mediapipe-facemesh-p5-tf2)



- tensorflow.js + mediapipe facemseh

- p5.js



Also available on editor.p5js.org: [https://editor.p5js.org/lingdong/sketches/ef6FB-uNq](https://editor.p5js.org/lingdong/sketches/ef6FB-uNq)



**Made at [the Frank-Ratchye STUDIO for Creative Inquiry](https://studioforcreativeinquiry.org/) at Carnegie Mellon University.**