https://github.com/vladmandic/human-motion

Realtime Face/Pose/Hand Motion 3D Model Visualization and 2D Overlay for using Human Library and BabylonJS
https://github.com/vladmandic/human-motion

body-tracking face-tracking hand-tracking human

Last synced: 2 months ago
JSON representation

Realtime Face/Pose/Hand Motion 3D Model Visualization and 2D Overlay for using Human Library and BabylonJS

• Host: GitHub
• URL: https://github.com/vladmandic/human-motion
• Owner: vladmandic
• License: mit
• Created: 2021-11-25T12:37:30.000Z (over 3 years ago)
• Default Branch: main
• Last Pushed: 2023-01-30T13:14:47.000Z (over 2 years ago)
• Last Synced: 2025-03-16T02:04:49.474Z (2 months ago)
• Topics: body-tracking, face-tracking, hand-tracking, human
• Language: TypeScript
• Homepage: https://vladmandic.github.io/human-motion/src/index.html
• Size: 128 MB
• Stars: 104
• Watchers: 5
• Forks: 15
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

        
# Human: 3D Motion Visualization

- Input  

  **Video file**  

  **Live webcam**  

- Analyze and track all keypoints in a *web worker* thread  

  Using [**Human**](https://github.com/vladmandic/human) library  

  **Face**, **Body**, **Hands**

- Display  

  **Video Overlay** with keypoints  

  **3D Mesh** with *pan/zoom/rotate* functionality




![**Screenshot-Face**](assets/screenshot-face.jpg)

![**Screenshot-Body**](assets/screenshot-body.jpg)

![**Screenshot-Hand**](assets/screenshot-hand.jpg)




## Run




### Online

> [**Live Demo**](https://vladmandic.github.io/human-motion/src/index.html)

### Offline

- Clone or fork the repository

- Install dependencies: `npm install`

- Start HTTPS server using precompiled code: `npm start`

- Navigate to 

### Development Mode

- Clone or fork the repository

- Compile code on-demand & start HTTPS server: `npm run dev`

- Navigate to 

## Notes

- All heavy-lifting (machine learning processing) is performed using [**Human**](https://github.com/vladmandic/human) library  

- Enabled models are: **BlazeFace-Back, FaceMesh-with-Attention, BlazePose-Heavy, HandTrack, HandDetect**  

- **2D** visualization is performed using built-in draw methods in `Human` library  

- **3D** visualization is performed using [BabylonJS](https://www.babylonjs.com/)  

## Processing

Process flow is separated into two loops:

- **Process loop**  

  Runs in web-worker so its not blocking the main thread  

  Supports `WASM`, `WebGL` and `WebGPU` backends  

  Runs as fast as possible: frame is submited to worker and as soon as results are available a new frame is submitted  

- **Render loop**  

  Runs at fixed 60 FPS  

  Draws both **2D** canvas overlay and **3D** GL mesh  

  Uses intepolated (for smoother output) results based on last known processed results from process loop