Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/charlielito/teachable-machines-audio-demo
An audio model for recognizing a whistle pattern was trained to toggle a Sonoff/Ewelink socket device connected to a room light
https://github.com/charlielito/teachable-machines-audio-demo
audio deep-learning microphone teachable-machine
Last synced: about 2 months ago
JSON representation
An audio model for recognizing a whistle pattern was trained to toggle a Sonoff/Ewelink socket device connected to a room light
- Host: GitHub
- URL: https://github.com/charlielito/teachable-machines-audio-demo
- Owner: charlielito
- Created: 2020-02-21T22:10:54.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2023-05-08T02:36:29.000Z (over 1 year ago)
- Last Synced: 2024-03-15T19:01:45.552Z (9 months ago)
- Topics: audio, deep-learning, microphone, teachable-machine
- Language: JavaScript
- Homepage:
- Size: 295 KB
- Stars: 11
- Watchers: 1
- Forks: 1
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Using Audio to toggle a WiFi connected device with [Teachable Machines](https://teachablemachine.withgoogle.com/) by Google
https://user-images.githubusercontent.com/8033598/117604569-5d26f680-b11b-11eb-958b-fc94078ec963.mp4
An audio model for recognizing a whistle pattern was trained to toggle a [Sonoff/Ewelink socket](https://www.amazon.com/-/es/Slampher-Bombilla-inteligente-compatible-Assistant/dp/B07TRSYJGB) device connected to a room light. Basically, you whistle and the light turns on/off.
This is also a starting point for those wo want to deploy an audio model in a headless device like a RaspberryPi, since at May 2021 there is no API in python for using an audio trained model. The current options are on the browser or on Android.
## API trick/hack
To be able to use an audio model, we use node.js with puppeteer that launches a headless chrome to be able to run the model. Chrome runs the model and outputs the predictions as logs which are parsed inside the node.js script. After that the user can do whatever with the predictions. In this case toggling a light.
## Requirements
* Node
* Ewelink connected device and account
* Teachable machines audio trained model
* A working microphone
## Installation
### Node installation
```
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.38.0/install.sh | bash
nvm install node
source ~/.bashrc
```
Install now the project:
```
npm install .
```
## Running the demo
First you need to identify the `ID` of the device you want to control. For that just get the list of all devices for you `ewelink` account with:
```
node get_devices.js -e -p
```
Look for the property `deviceid`.
Now you are ready to run the demo. If you want to change to another model, change https://github.com/charlielito/teachable-machines-audio-demo/blob/master/index.html#L18 with the URL of your trained model.
Finally run:
```
node index.js -e -p --deviceid
```
There are additional parameters like `--toggle_class` and `--threshold`. For more info run `node index.js -h`.
### Local or LAN mode
By default the demo calls the API through Internet, which causes a little bit of lag when triggering the state of the device. If you want to control a device that is in your LAN, you can follow this extra step that will speed up the control of the devices.
First cache the devices info locally with:
```
node save_cache.js -e -p
```
That command would generate 2 files: `arp-table.json` and `devices-cache.json`.
Finally just run everything with:
```
node index.js --lan --deviceid
```
## Training the model
This should be very straight forward following the instructions at https://teachablemachine.withgoogle.com/train/audio.
Once you have the model trained you should see something like the following:
## Hardware setup
You can run this demo in you laptop, but if you want to control your lights you'll have to have the laptop 24/7 turned on. Because of that, I am running this Demo in a Jetson TX2 for continuos evaluation. I used a Jetson because I had one at hand, but this should work without problem in a RaspberryPi for example.
I used an USB audio card to read the audio from an analog microphone, for example any headphones that have a microphone built-in (almost all). The set up can be seen in the following picture:
