{"id":13826008,"url":"https://github.com/charliegerard/gestures-ml-js","last_synced_at":"2025-04-14T02:33:49.094Z","repository":{"id":78179644,"uuid":"186580595","full_name":"charliegerard/gestures-ml-js","owner":"charliegerard","description":"[WIP] - Gesture recognition using hardware and Tensorflow.js ","archived":false,"fork":false,"pushed_at":"2019-09-01T12:29:15.000Z","size":20338,"stargazers_count":80,"open_issues_count":2,"forks_count":11,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-27T16:40:02.068Z","etag":null,"topics":["arduino","creative-coding","front-end-development","javascript","machine-learning","tensorflow-js"],"latest_commit_sha":null,"homepage":"https://bit.ly/sf-ml","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/charliegerard.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2019-05-14T08:38:13.000Z","updated_at":"2025-03-02T22:46:30.000Z","dependencies_parsed_at":"2023-03-22T21:48:12.287Z","dependency_job_id":null,"html_url":"https://github.com/charliegerard/gestures-ml-js","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgestures-ml-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgestures-ml-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgestures-ml-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/charliegerard%2Fgestures-ml-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/charliegerard","download_url":"https://codeload.github.com/charliegerard/gestures-ml-js/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248811163,"owners_count":21165244,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["arduino","creative-coding","front-end-development","javascript","machine-learning","tensorflow-js"],"created_at":"2024-08-04T09:01:30.702Z","updated_at":"2025-04-14T02:33:49.052Z","avatar_url":"https://github.com/charliegerard.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"# Gesture recognition using hardware and Tensorflow.js\n\n**This project is very experimental and in active development.**\n\nExperiments building a gesture recognition system using an Arduino, a Daydream controller and a mobile phone, with Tensorflow.js\n\n*Inspired by a [similar project](https://blog.mgechev.com/2018/10/20/transfer-learning-tensorflow-js-data-augmentation-mobile-net/) by [Minko Gechev](https://twitter.com/mgechev) using the webcam*.\n\n## Demo:\n\n![Demo](demo.gif)\n\nEach project has 2 demos: one to play a game of street fighter, and one to predict magic wand movements.\n\nSprites used in the Street Fighter demo come from this [Codepen](https://codepen.io/jkneb/pen/smtHA) and [this repo](https://github.com/jkneb/street-fighter-css)\n\n## How it works:\n\nUsing an accelerometer/gyroscope (MPU6050 for the Arduino, and built-in sensors for the Daydream and phone), we can record data streamed while performing a gesture. By repeating and recording gestures multiple times, we can feed all this data to a machine learning algorithm to find patterns in the data. Once a model is created, we can use it to predict new live data and classify it to use as input for an interface or device.\n\n## Arduino project:\n\nSee [arduino-mkr1000](arduino-mkr1000/) folder.\n\n## Daydream project:\n\nSee [daydream](daydream/) folder.\n\n## Phone project:\n\nSee [phone](phone/) folder.\n\n---\n\n\n## Blog post\n\nMore details on in [this blog post](https://dev.to/devdevcharlie/play-street-fighter-with-body-movements-using-arduino-and-tensorflow-js-4kbi).\n\n\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharliegerard%2Fgestures-ml-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcharliegerard%2Fgestures-ml-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcharliegerard%2Fgestures-ml-js/lists"}