{"id":15638720,"url":"https://github.com/jonnor/esc-cnn-microcontroller","last_synced_at":"2025-10-06T05:02:07.028Z","repository":{"id":138840802,"uuid":"166037992","full_name":"jonnor/ESC-CNN-microcontroller","owner":"jonnor","description":" Environmental Sound Classification on Microcontrollers using Convolutional Neural Networks","archived":false,"fork":false,"pushed_at":"2024-08-02T16:22:52.000Z","size":34040,"stargazers_count":102,"open_issues_count":5,"forks_count":20,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-26T20:21:33.022Z","etag":null,"topics":["audio-classification","embedded-devices","machine-learning","master-thesis","microcontroller","thesis"],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonnor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-01-16T12:38:53.000Z","updated_at":"2025-03-21T16:08:04.000Z","dependencies_parsed_at":"2024-10-22T20:24:18.373Z","dependency_job_id":null,"html_url":"https://github.com/jonnor/ESC-CNN-microcontroller","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2FESC-CNN-microcontroller","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2FESC-CNN-microcontroller/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2FESC-CNN-microcontroller/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonnor%2FESC-CNN-microcontroller/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonnor","download_url":"https://codeload.github.com/jonnor/ESC-CNN-microcontroller/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248658392,"owners_count":21140936,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-classification","embedded-devices","machine-learning","master-thesis","microcontroller","thesis"],"created_at":"2024-10-03T11:22:50.610Z","updated_at":"2025-10-06T05:02:01.986Z","avatar_url":"https://github.com/jonnor.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Environmental Sound Classification on Microcontrollers using Convolutional Neural Networks\n\n## Thesis report\n\n\u003ca href=\"https://github.com/jonnor/ESC-CNN-microcontroller/releases/download/print1/report-print1.pdf\"\u003e\u003cimg src=\"https://github.com/jonnor/ESC-CNN-microcontroller/raw/master/report/img/frontpage.png\" height=\"200\" alt=\"Download thesis report (PDF)\"\u003e\u003c/a\u003e\n\n\u003ca href=\"https://github.com/jonnor/ESC-CNN-microcontroller/releases/download/print1/report-print1.pdf\"\u003eDownload thesis report (PDF)\u003c/a\u003e\n\n### Errata\n\nIn `print1` version of report\n\n- Fig 5.1. DS-Strided-24 result is missing\n- Fig 5.1. No-information-rate should be 11.5% instead of 10%.\nDid not take class-imbalance into account\n- Fig 2.10. Labels EffNet and ShuffleNet swapped\n- Fig 5.3. Missing description of model used. Uses Stride-DS-24\n- Table 4.1. Nesterov momentum shows NaN. Should be 0.9\n\n## Citing\n\nYou can use the following BibTeX entry\n\n```bibtex\n@mastersthesis{esc_micro_cnn_nordby2019,\n    title={Environmental Sound Classification on Microcontrollers using Convolutional Neural Networks},\n    author={Jon Nordby},\n    year=2019,\n    month=5,\n    school={Norwegian University of Life Sciences},\n    url={http://hdl.handle.net/11250/2611624}\n}\n```\n\n## Keywords\n\n    Wireless Sensor Networks, Embedded Systems\n    Edge Computing, Edge Machine Learning\n    Noise classification, Environmental Sound Classification (ESC), Urbansounds\n    Tensorflow, Keras, librosa\n\n## Abstract\n\nNoise is a growing problem in urban areas,\nand according to the WHO is the second environmental cause of health problems in Europe.\nNoise monitoring using Wireless Sensor Networks are\nbeing applied in order to understand and help mitigate these noise problems.\nIt is desirable that these sensor systems, in addition to logging the sound level,\ncan indicate what the likely sound source is.\nHowever, transmitting audio to a cloud system for classification is\nenergy-intensive and may cause privacy issues.\nIt is also critical for widespread adoption and dense sensor coverage that\nindividual sensor nodes are low-cost.\nTherefore we propose to perform the noise classification on the sensor node,\nusing a low-cost microcontroller.\n\nSeveral Convolutional Neural Networks were designed for the\nSTM32L476 low-power microcontroller using the Keras deep-learning framework,\nand deployed using the vendor-provided X-CUBE-AI inference engine.\nThe resource budget for the model was set at maximum 50% utilization of CPU, RAM, and FLASH.\n10 model variations were evaluated on the Environmental Sound Classification task\nusing the standard Urbansound8k dataset.\n\nThe best models used Depthwise-Separable convolutions with striding for downsampling,\nand were able to reach 70.9\\% mean 10-fold accuracy while consuming only 20% CPU.\nTo our knowledge, this is the highest reported performance on Urbansound8k using a microcontroller.\nOne of the models was also tested on a microcontroller development device,\ndemonstrating the classification of environmental sounds in real-time.\n\nThese results indicate that it is computationally feasible to classify environmental sound\non low-power microcontrollers.\nFurther development should make it possible to create wireless sensor-networks\nfor noise monitoring with on-edge noise source classification.\n\n\n## Run experiments\n\n### Setting up\n\nRecommend using [miniconda](https://conda.io/en/latest/miniconda.html) for\n\n    conda env create -f environment.yml\n    conda activate microesc\n\nAs an altenative, one can use pip\n\n    #pip install -r requirements.txt\n\nPreprocess audio files into features\n\n    python3 preprocess.py\n\nCheck that the environment is working.\nThis will run training process, but only for a few minutes.\n\n    python3 jobs.py --check\n\n### Running\n\nTrain the models\n\n    python3 jobs.py\n\nEvaluate the resulting models\n\n    python3 test.py\n\nPlot the results\n\n    python3 report.py\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonnor%2Fesc-cnn-microcontroller","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonnor%2Fesc-cnn-microcontroller","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonnor%2Fesc-cnn-microcontroller/lists"}