{"id":15630104,"url":"https://github.com/jonnor/embeddedml","last_synced_at":"2025-04-12T23:30:23.027Z","repository":{"id":52956984,"uuid":"197437798","full_name":"jonnor/embeddedml","owner":"jonnor","description":"Notes on Machine Learning on edge for embedded/sensor/IoT uses","archived":false,"fork":false,"pushed_at":"2025-03-28T22:14:24.000Z","size":170155,"stargazers_count":266,"open_issues_count":1,"forks_count":32,"subscribers_count":17,"default_branch":"master","last_synced_at":"2025-04-04T02:48:49.792Z","etag":null,"topics":["embedded-systems","machine-learning","microcontroller","sensor-data","sensor-network","tinyml"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonnor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["jonnor"]}},"created_at":"2019-07-17T17:55:31.000Z","updated_at":"2025-04-01T16:31:24.000Z","dependencies_parsed_at":"2023-02-01T01:02:06.745Z","dependency_job_id":"ddf683e7-f4ee-48ad-9405-e0b507423c54","html_url":"https://github.com/jonnor/embeddedml","commit_stats":{"total_commits":459,"total_committers":1,"mean_commits":459.0,"dds":0.0,"last_synced_commit":"6aaa814e41335acc1e9cd487747573a0fe6cbbd0"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fembeddedml","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fembeddedml/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fembeddedml/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2Fembeddedml/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonnor","download_url":"https://codeload.github.com/jonnor/embeddedml/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248647249,"owners_count":21139081,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["embedded-systems","machine-learning","microcontroller","sensor-data","sensor-network","tinyml"],"created_at":"2024-10-03T10:30:12.056Z","updated_at":"2025-04-12T23:30:18.017Z","avatar_url":"https://github.com/jonnor.png","language":"Jupyter Notebook","funding_links":["https://github.com/sponsors/jonnor"],"categories":[],"sub_categories":[],"readme":"# Machine learning on embedded devices\n\nFocused primarily on running inference/prediction/feed-forward part on a microcontroller (or small embedded device).\nTraining phase can run on a standard computer/server, using existing tools as much as possible.\n\n# Background\n\n## What and when to use machine learning\n\nThe defaults right now are to do conventional signal processing (no learning) in sensor,\nand stream raw data to the cloud for storage and processing. Machine learning happens in the cloud.\nIf gateways are used, they mostly forward communication (no data processing). \n\nOn-edge processing valueable when\n\n* Local response needed. Autonomy\n* Adaptable response needed. Over time, in context.\n* Low/predictable latency needed\n* Sending raw sensor data has privacy implications.\nAudio, video.\n* Unreliable connection\n* High bandwidth sensor input.\nAudio, video, accelerometer/IMU, current sensor, radiowaves.\n* Low bandwidth algorithm output\n* Events of interest are rare\n* Low energy usage needed\n* Full/raw sensor data is not valuable to store\n* Sensor system should be low cost\n\nExample usecases\n\n* Predictive maintenance, using audio/vibration data\n* Activitity detection for people, using audio/accelerometer data. Assistive tech, medical\n* Appliance disaggregation, using aggregated power consumption data. \"Non-Intrusive Load Monitoring\" (NILM)\n* Anomaly/change detection for predictive maintenance, using audio/vibration data, or electrical data\n* Gesture recognition as human input device, using accelerometer/gyro data.\n* Speech/command recognition as human input device, using microphone. Keyword/Wake-word detection\n* Battery saving in wireless sensors. Normally sending day/week aggregates, on event/anomaly detection send data immediately\n* Health status of animals via activity detected using accelerometer\n* Monitoring eating activity using accelerometer [1](https://www.sciencedirect.com/science/article/pii/S0010482515000086)\n* Environmental monitoring, using microphone to detect unwanted activity like cutting down trees\n* Adaptive signalling and routing for wireless transmission in Wireless Sensor networks\n* Electronic nose using arrays of MEMS detectors\n* Material identification using reflecive spectrometer [1](https://hackaday.io/project/143014-compact-25-spectrometer/)\n\nMore notes on [Applications](./applications)\n\n### Motivation\n\n* [Why the Future of Machine Learning is Tiny (devices)](https://petewarden.com/2018/06/11/why-the-future-of-machine-learning-is-tiny)\nTiny Computers are Already Cheap and Everywhere. Energy is the Limiting Factor.We Capture Much More Sensor Data Than We Use.\n* [embedded.com: Bringing machine learning to the edge](https://www.embedded.com/electronics-blogs/say-what-/4460873/Bringing-machine-learning-to-the-edge--A-Q-A-with-Neurala-s-Anatoli-Gorshechnikov-)\nPredictions are much lower bandwidth than the raw sensor data (e.g. video)\nIt allows for local adaptation in the AI logic (L-DNN)\nIt achieves lower latency between observed event and action resulting from AI logic\n\"the most important question is what is the least amount accuracy and computation complexity we can do\nwhile still delivering the business value?\"\nTop mistake: \"Continuing with the top down approach ‘let’s make it perform the task first and then squeeze it on device`\ninstead of switching to bottom up ‘let’s make it run on device and fulfill all hardware constraints first,\nand then tune it for the task at hand’.\"\n* [How to run deep learning model on microcontroller with CMSIS-NN](https://www.dlology.com/blog/how-to-run-deep-learning-model-on-microcontroller-with-cmsis-nn/).\nWhy run deep learning model on a microcontroller?\nSensitive data gets to the cloud, photos, and audio recordings.\nThe company who sells this may charge a service fee to use its service and even worse sell your private data.\nIt won't work without the network connection to the server.\nData traveling back and forth between the device and server introduces lag.\nRequire network and wireless hardware components on the circuit design which increase the cost.\nIt might waste bandwidth sending useless data.\n\n\n## State of the Art in 2019\nOf ML inference on general-purpose microcontrollers.\n\n```\nTODO: update for 2023\n```\n\n- Deep models have efficient implementations for ARM Cortex-M. Ex: CNN and RNN in CMSIS-NN, FC in uTensor\n- Some implementations available for non-neural models that *can* be used. Ex: SVM,RF,AdaBoost in sklearn-porter\n- A few special-designed ML algorithms made. Ex: ProtoNN, Bonsai\n- Basic tools available for converting Tensorflow models\n- Keyword-spotting/wake-word on audio well established. Used in commercial products (Alexa etc)\n- Human activity detecton on accelerometers.\n- Computer vision is actively developed\n- Lots of research and many announcements of low-power co-processors, but little on market yet\n\nLimitations\n\n- Neural models lacking for non-ARM micros. ESP8266/ESP32\n- Non-neural models missing inference engines designed for microcontrollers\n- \"Small DNN\" work mostly on computer vision for mobile phones (model size 1000x of uC)\n- Few/no pretrained models available. Transfer learning little explored?\n- Very little documentation of entire development process.\nFrom planning, data aquisition, model design\n- Best practices underdocumented (or underdeveloped?) \n\nWays of advancing, make contributions\n\n- Faster inference. Power saving, or bigger problems.\n- Smaller models. Cheaper MCU, or bigger problems.\n- Better accuracy on a problem. Better user experience, new usecases\n- Solve a concrete usecase. Easier to deploy similar usecases\n- Comparison between approaches. Microcontroller, ML model\n- Libraries or tools. Lower time to market, enable more developers\n\n\n# Learning material\n\nBooks\n\n* [Learning in Embedded Systems](https://mitpress.mit.edu/books/learning-embedded-systems), May 1993.\n* [TinyML: Machine Learning with TensorFlow on Arduino, and Ultra-Low Power Micro-Controllers](https://tinymlbook.com/). January, 2020.\n* [TinyML Cookbook](https://www.packtpub.com/product/tinyml-cookbook/9781801814973)\n\nArticles\n\n- [Embedded.com: Applying machine learning in embedded systems](https://www.embedded.com/applying-machine-learning-in-embedded-systems)\n- [Embedded.com: Transfer learning for the IoT](https://www.embedded.com/transfer-learning-for-the-iot/)\n\n# Tools\n\nOpen-source\n\n* [emlearn](http://github.com/emlearn/emlearn). Inference engine for microcontrollers.\nSupports converting scikit-learn models to plain C code.\nNo dynamic allocations. No runtime needed.\n* [TensorFlow Lite for Microcontrollers](https://www.tensorflow.org/lite/microcontrollers).\nSupports neural network models made with TensorFlow (including Keras).\nCan run on wide range of platforms.\nSince November 2018. Supports ARM Cortex M, RISC-V, ESP32/Xtensa and Linux/MacOS host.\nRequires a runtime.\n* [nnom](https://github.com/majianjia/nnom) - Fixed-point neural network compiler for microcontrollers.\nSupports wide range of networks. Outputs plain C code. Can use CMSIS-NN on ARM Cortex M.\n- [micromlgen](https://github.com/eloquentarduino/micromlgen)\n* [Embedded Learning Library](https://github.com/Microsoft/ELL) by Microsoft.\nSet of C++ libraries for machine learning on embedded platforms. Includes code for kNN, RandomForest etc.\nAlso has some node-based dataflow system in place it seems. JavaScript and Python bindings.\n* ONNC project has a [backend for ARM Cortex M](https://github.com/ONNC/onnc-tutorial/blob/master/lab_2_Digit_Recognition_with_ARM_CortexM/lab_2.md) (using CMSIS-NN)\nand a [C backend](https://github.com/ONNC/onnc/blob/74e59908b2881844329c3d330eea7a7c306e1e22/docs/ONNC-C-Backend-Guide.md).\nAllows to convert an ONNX models to run on devices.\n- [nn4mc_cpp](https://github.com/correlllab/nn4mc_cpp).\nNeural Networks for Microcontrollers.\nSupports Keras natively. Provides instructions for PyTorch et.c via ONNX.\nDocumentation for using from Python is lacking, as well as the type of networks supported.\nDoes not seem maintained since 2020.\n* [uTensor](https://github.com/uTensor/uTensor).\nExport Tensorflow model to mbed/ARM microcontrollers.\nNot supported on ESP32 or RISC-V or similar.\n* [Embedded Classification Software Toolbox](https://github.com/ma2th/ecst).\nUnmaintained since 2018.\n- [sklearn-porter](https://github.com/nok/sklearn-porter).\nCan compile DecisionTreeClassifier and SVC models to C.\nUses dynamic memory.\nNot optimized for use on embedded devices.\n* [microTVM](https://tvm.apache.org/docs/topic/microtvm/index.html).\nDepends only on the C standard library, and runs bare metal such as STM32 and NRF52 microcontrollers.\nIs under development. Which models are supported on microcontrollers not specified.\n\nProprietary\n\n* X-CUBE-AI for STM32\n\n\n# Models\n\nA range of Machine Learning models are useful in an embedded devices setting.\nClassical methods are used when the amount of data is quite small,\nand neural networks for large datasets and complex inputs.\n\nBelow are notes on the various models in the context of embedded Machine Learning,\nincluding model size and compute-time optimization.\n\n- [Tree-based methods](./models/tree-based.md).\nRandom Forest, Extratrees, Decision Trees, et.c.\n- [Neural Networks](./models/neural-networks.md).\nConvolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Autoencoders\n- [Support Vector Machines](./models/support-vector-machine.md) (SVM).\n- [Mixture models](./models/mixtures.md).\nGaussian Mixture Models (GMM).\n- [Nearest Neighbours](./models/k-nearest-neighbours.md). kNN et.c.\n\n\n# More topics\n\n- [Privacy](./topics/privacy.md)\n- [Energy usage](./topics/energy-usage.md)\n- [Model size](./topics/model-size.md)\n- [On-device learning](./on-device-learning)\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonnor%2Fembeddedml","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonnor%2Fembeddedml","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonnor%2Fembeddedml/lists"}