https://github.com/808vita/fingerpose-ext

Tensorflowjs handpose gesture recognition - This package provides utility function to create "landmark" array from keypoints & keypoints3D arrays; for the latest model of handpose.
https://github.com/808vita/fingerpose-ext

ai gesture-recognition hand-pose-estimation hand-pose-recognition handpose javascript tensorflow tensorflowjs

Last synced: 16 days ago
JSON representation

Tensorflowjs handpose gesture recognition - This package provides utility function to create "landmark" array from keypoints & keypoints3D arrays; for the latest model of handpose.

• Host: GitHub
• URL: https://github.com/808vita/fingerpose-ext
• Owner: 808vita
• Created: 2023-10-08T11:03:41.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2023-10-08T12:02:06.000Z (about 1 year ago)
• Last Synced: 2024-11-16T03:37:34.922Z (about 1 month ago)
• Topics: ai, gesture-recognition, hand-pose-estimation, hand-pose-recognition, handpose, javascript, tensorflow, tensorflowjs
• Language: JavaScript
• Homepage: https://www.npmjs.com/package/fingerpose-ext
• Size: 3.91 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        
# fingerpose-ext

This package provides utility function to create "landmark" array from keypoints & keypoints3D arrays.

fingerpose & fingerpose-gestures npm packages both requires "landmarks" array to detect the gestures.

The newer improved version of hand pose "@tensorflow-models/hand-pose-detection": "^2.0.1" returns "keypoints" and "keypoints3D" instead of "landmarks".

## Installation

Install the module via NPM:

```sh

npm i fingerpose-ext

```

Install the module via yarn:

```sh

yarn add fingerpose-ext

```

## Usage

### Include "handpose" (newer version), "tfjsWasm","fingerpose",etc. and this library

```js

import * as fp from "fingerpose";

import createLandmarks from "fingerpose-ext";

```

### Use "handpose" to estimate the landmarks

```js

const model = handPoseDetection.SupportedModels.MediaPipeHands;

const detectorConfig = {

  runtime: "mediapipe", // or 'tfjs',

  solutionPath: "https://cdn.jsdelivr.net/npm/@mediapipe/hands",

  modelType: "full",

};

const detector = await handPoseDetection.createDetector(model, detectorConfig);

const hands = await detector.estimateHands(image);

```

### Example output

```json

[

  {

    score: 0.8,

    handedness: ‘Right’,

    keypoints: [

      {x: 105, y: 107, name: "wrist"},

      {x: 108, y: 160, name: "pinky_finger_tip"},

      ...

    ],

    keypoints3D: [

      {x: 0.00388, y: -0.0205, z: 0.0217, name: "wrist"},

      {x: -0.025138, y: -0.0255, z: -0.0051, name: "pinky_finger_tip"},

      ...

    ]

  }

]

```

### Estimate the gestures

```js

let landmarks = createLandmarks(hands[0]);

const estimatedGestures = GE.estimate(landmarks, 8.5);

```

The result is an object containing possible gestures and their confidence, for example:

```json

{

    "poseData": [ ... ],

    "gestures": [

        { "name": "thumbs_up", "confidence": 9.25 },

        { ... }

    ]

}

```