Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/dmackenz/HTML5-Gesture-Recognition
Gesture recognition for the browser
https://github.com/dmackenz/HTML5-Gesture-Recognition
gesture-recognition neural-network swipe tensorflowjs
Last synced: 16 days ago
JSON representation
Gesture recognition for the browser
- Host: GitHub
- URL: https://github.com/dmackenz/HTML5-Gesture-Recognition
- Owner: dmackenz
- Created: 2018-11-29T21:11:16.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2019-04-01T18:22:17.000Z (over 5 years ago)
- Last Synced: 2024-08-01T05:22:19.415Z (3 months ago)
- Topics: gesture-recognition, neural-network, swipe, tensorflowjs
- Language: JavaScript
- Homepage:
- Size: 21.2 MB
- Stars: 4
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# HTML5 Gesture Recognition
### Overview
The purpose of this project is to demonstrate how an artificial neural network can be used to recognize gestures on webpages. The idea for this project was heavily inspired by Google’s Quick, Draw! game.
This concept provides client-side code for tracking mouse and touch based movements for both mobile and desktop. A swipe on the screen by either a finger or mouse is collected by the browser and classified/encoded to a specific set of gestures by a neural network. The current prototype is a proof of concept and will only be trained on four types of gestures (up, down, left, right). The current plan is to implement the neural network in Tensorflow.js.
### Example
### Files
The important locations in the project folders can be found at:
| Description | Directory Location |
|-------------------------------------------------|-------------------------------------|
| Example of the model working in a browser. | /example/ |
| Program for procedurally generating swipe data. | /training/dataGeneration/index.html |
| Training the model. | /training/training.js |
### Procedurally Generated Training Set
The training set was created automatically by a JavaScript client that generates artificial swipe data and normalizes it for the viewport. This data was then used to train the neural network.
##### The structure of the collected data
```JSON
{
"swipeType":"right",
"xs": [
0.2469352014010508,
0.2679509632224168,
0.29334500875656744,
0.318739054290718,
0.3380035026269702,
0.34851138353765326,
0.3502626970227671
],
"ys": [
0.23309352517985613,
0.23453237410071942,
0.23597122302158274,
0.23597122302158274,
0.23453237410071942,
0.23309352517985613,
0.2316546762589928
]
}
```
# Report
