https://github.com/vladmandic/blazepose
BlazePose: Body Segmentation for TFJS and NodeJS
https://github.com/vladmandic/blazepose
blazepose body-segmentation pose-detection pose-estimation pose-tracking
Last synced: 2 months ago
JSON representation
BlazePose: Body Segmentation for TFJS and NodeJS
- Host: GitHub
- URL: https://github.com/vladmandic/blazepose
- Owner: vladmandic
- Archived: true
- Created: 2021-04-24T23:02:43.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2022-11-20T19:59:56.000Z (over 2 years ago)
- Last Synced: 2024-10-27T22:45:01.769Z (6 months ago)
- Topics: blazepose, body-segmentation, pose-detection, pose-estimation, pose-tracking
- Language: TypeScript
- Homepage:
- Size: 79.4 MB
- Stars: 17
- Watchers: 3
- Forks: 5
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# BlazePose: Body Segmentation for TFJS
*Updated for BlazePose v2 variations*
Included models:
- BlazePose Detector
- BlazePose Lite, Full & Heavy variations
Models included in `/model/*` were manually converted and quantized to TFJS Graph model format
from the original repository and are not identical to TFJS Graph models published on [TF Hub](https://tfhub.dev/s?q=blazepose)
Models descriptors and signature have been additionally parsed for readability
## Implementation
Actual model parsing implementation in `/src/blazepose.ts` does not follow [reference implementation](https://github.com/tensorflow/tfjs-models/tree/master/pose-detection)
BlazePose is a two-phase model:
- Detector
- Pose analysis
Pose analysis is a fast, clean and accurate model
However, detector is much slower and requires a lot of post-processing - thus its usage has been eliminated from this implementation
Instead, this implementation prepares *virtual* detected tensor padded and resized to fit pose analysis model
Implementation is in `src/blazepose.ts` with keypoint definitions and annotations in `src/blazeposecoords.ts`
The rest of sources in `src/util` is for simplicity of usage and testing only
Ideal implementation includes additional results caching and temporal interpolation with smoothing functionality
See for details
## Result
Implementation defines results a:
- `keypoints`: array of 39 detected keypoints
- `part`: body part label
- `position`: [x, y, z] normalized to input size
- `postitionRaw`: [x, y, z] normalized to 0..1
- `score`: detection score for the body part
- `box`: [x, y, width, height] box around detected body normalized to input size
- `boxRaw`: [x, y, width, height] box around detected body normalized to 0..1
- `annotations`: annotated arrays of points to help define higher level entities such as `arm` or `leg`
- `score`: average score for body
## Demo
Demo app that initializes webcam and uses `BlazePose` is in `demo/index.html`
To run image analysis instead of default analysis on webcam input, edit `demo/index.js`
Example:
```js
await detectImage('daz3d-ella.jpg');
```
## Credits & Links
- Blog:
- Docs:
- Paper:
- Card:
## Todo
- Enhance virtual tensor with virtual box around previous body center
For details, see disabled code in `calculateBoxes` method
- HeatMap processing