{"id":48037936,"url":"https://github.com/absent42/esphome-audio-reactive","last_synced_at":"2026-04-04T14:00:55.436Z","repository":{"id":344192548,"uuid":"1180910427","full_name":"absent42/esphome-audio-reactive","owner":"absent42","description":"ESP32 audio analysis component with beat detection, frequency and amplitude energy","archived":false,"fork":false,"pushed_at":"2026-03-25T13:10:08.000Z","size":408,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-25T13:38:59.770Z","etag":null,"topics":["audio-analysis","audio-processing","esp32","esphome","home-assistant"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/absent42.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-13T14:43:27.000Z","updated_at":"2026-03-25T13:10:12.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/absent42/esphome-audio-reactive","commit_stats":null,"previous_names":["absent42/esphome-audio-reactive"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/absent42/esphome-audio-reactive","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/absent42%2Fesphome-audio-reactive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/absent42%2Fesphome-audio-reactive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/absent42%2Fesphome-audio-reactive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/absent42%2Fesphome-audio-reactive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/absent42","download_url":"https://codeload.github.com/absent42/esphome-audio-reactive/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/absent42%2Fesphome-audio-reactive/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31402277,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T10:20:44.708Z","status":"ssl_error","status_checked_at":"2026-04-04T10:20:06.846Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-analysis","audio-processing","esp32","esphome","home-assistant"],"created_at":"2026-04-04T14:00:32.228Z","updated_at":"2026-04-04T14:00:55.402Z","avatar_url":"https://github.com/absent42.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ESPHome Audio Reactive\n\nESP32 audio analysis component for ESPHome. Provides real-time onset/beat detection,\nfrequency band energy, amplitude, BPM estimation with phase tracking, spectral\ndescriptors (centroid, rolloff), and silence detection via on-device FFT processing\nwith a dedicated FreeRTOS task. Supports three detection modes: spectral flux,\nbass energy, and complex domain (phase + magnitude).\n\nDesigned as the audio source for the \n[Aqara Advanced Lighting](https://github.com/absent42/aqara-advanced-lighting) Home Assistant integration, enabling music syncing of dynamic lighting scenes. The devices output an array of Home Assistant sensors which can also be used with any automation.\n\n## Hardware\n\nCan be adapted for any ESP32 with an I2S digital microphone. Configurations currently available for:\n\n### M5Stack ATOM Echo\n\nThe best starting point for most users, readily available.\n\n![M5Stack ATOM Echo](https://raw.githubusercontent.com/absent42/esphome-audio-reactive/refs/heads/main/static/images/atom-echo.jpg)\n\n| | |\n|---|---|\n| **Price** | ~$13 |\n| **Chipset** | ESP32-PICO-D4 |\n| **Microphone** | Built-in SPM1423 PDM |\n| **Feedback** | LED |\n| **Power** | USB-C |\n| **Size** | 24 x 24 x 17mm |\n| **Where to buy** | [M5Stack store](https://shop.m5stack.com/products/atom-echo-smart-speaker-dev-kit), [Pi Hut](https://thepihut.com/products/atom-echo-smart-speaker-dev-kit), [Amazon US](https://amzn.to/4dnA7GH), [Amazon UK](https://amzn.to/4bdYJQM), [Amazon DE](https://amzn.to/47lapii), [Amazon FR](https://amzn.to/4rU1Ja6), [Amazon IT](https://amzn.to/4rSoJWO), [AliExpress](https://s.click.aliexpress.com/e/_c3ULpt9b) |\n\n### M5Stack ATOM Echo S3R\n\nHigher-quality audio with an ES8311 codec and speaker feedback for on-device status tones.\n\n![M5Stack ATOM Echo S3R](https://raw.githubusercontent.com/absent42/esphome-audio-reactive/refs/heads/main/static/images/atom-echo-s3r.jpg)\n\n| | |\n|---|---|\n| **Price** | ~$15 |\n| **Chipset** | ESP32-S3 |\n| **Microphone** | MEMS via ES8311 ADC (I2S, 44.1kHz) |\n| **Audio codec** | ES8311 (mic ADC + speaker DAC) |\n| **Feedback** | Speaker tones (no LED) |\n| **Power** | USB-C |\n| **Size** | 24 x 24 x 17mm |\n| **Where to buy** | [M5Stack store](https://shop.m5stack.com/products/atom-echos3r-smart-speaker-dev-kit?variant=46751279710465), [Pi Hut](https://thepihut.com/products/atom-echos3r-smart-speaker-dev-kit), [AliExpress](https://s.click.aliexpress.com/e/_c4oQ7XMD) |\n\n### Waveshare ESP32-S3 Audio Board\n\nFeature-rich board with dual MEMS microphones, 7-LED ring, and optional battery power.\n\n![Waveshare ESP32-S3](https://raw.githubusercontent.com/absent42/esphome-audio-reactive/refs/heads/main/static/images/waveshare-esp32-s3.jpg)\n\n| | |\n|---|---|\n| **Price** | ~$16 |\n| **Chipset** | ESP32-S3R8 (8MB PSRAM) |\n| **Microphone** | Dual MEMS via ES7210 ADC (I2S, 44.1kHz) |\n| **Audio codec** | ES7210 (mic) + ES8311 (speaker) |\n| **Feedback** | LED ring |\n| **Power** | USB-C, optional battery |\n| **Size** | 58 x 58 x 49mm |\n| **Where to buy** | [Waveshare store](https://www.waveshare.com/esp32-s3-audio-board.htm), [Amazon US](https://amzn.to/4lqAoKU), [Amazon UK](https://amzn.to/414SkRU), [Amazon DE](https://amzn.to/4bvKK9b), [Amazon FR](https://amzn.to/4uQIu3U), [AliExpress](https://s.click.aliexpress.com/e/_c3nj7nSd) |\n\n### M5StickC Plus2\n\nA compact development kit with a built-in screen, battery, and PDM microphone.\n\n![M5StickC Plus2](https://raw.githubusercontent.com/absent42/esphome-audio-reactive/refs/heads/main/static/images/m5stack-stick2.jpg)\n\n| | |\n|---|---|\n| **Price** | ~$20 |\n| **Chipset** | ESP32-PICO-V3-02 |\n| **Microphone** | Built-in SPM1423 PDM |\n| **Feedback** | Screen |\n| **Power** | USB-C, built-in battery |\n| **Size** | 54 x 25 x 16mm |\n| **Where to buy** | [M5Stack store](https://shop.m5stack.com/products/m5stickc-plus2-esp32-mini-iot-development-kit), [Pi Hut](https://thepihut.com/products/m5stickc-plus2-esp32-mini-iot-development-kit), [Amazon US](https://amzn.to/470ydYE), [Amazon UK](https://amzn.to/4brZ0i4), [Amazon DE](https://amzn.to/3PZFwde), [Amazon FR](https://amzn.to/4uM3Dwh), [Amazon IT](https://amzn.to/4c3nJdU), [AliExpress](https://s.click.aliexpress.com/e/_c3liKXJr) |\n\n## Installation\n\n### One-click install (recommended)\n\nVisit **[absent42.github.io/esphome-audio-reactive](https://absent42.github.io/esphome-audio-reactive/)**\nand click Install. Connect your ESP32 via USB — no ESPHome knowledge required.\n\n### Manual ESPHome setup\n\nIf you prefer to compile yourself or need custom configuration:\n\nAdd to your ESPHome YAML:\n\n    external_components:\n      - source: github://absent42/esphome-audio-reactive\n        components: [audio_reactive]\n\nSee `atom-echo.yaml`, `atom-echo-s3r.yaml`, and `waveshare-s3-audio.yaml` for complete device-specific configs.\n\n### Security (recommended)\n\nAfter installing, add API encryption and an OTA password to your device config.\nIf you use the ESPHome Dashboard (Home Assistant add-on), it generates encryption\nkeys automatically when you add the `encryption:` block.\n\nAdd these to your YAML:\n\n    api:\n      encryption:\n        key: !secret api_encryption_key\n\n    ota:\n      - platform: esphome\n        password: !secret ota_password\n\nThen add matching values to your `secrets.yaml` (managed by the ESPHome Dashboard,\nor created manually alongside your device YAML).\n\nAPI encryption secures communication between the device and Home Assistant.\nThe OTA password prevents unauthorized over-the-network firmware updates.\nNeither affects the USB web installer.\n\n## Exposed Entities\n\n| Entity | Type | Update Rate | Description |\n|--------|------|-------------|-------------|\n| Audio Sensor | binary_sensor | On event | Pulses on when a musical onset is detected (beat, cymbal, vocal entrance, etc.) |\n| Silence | binary_sensor | On change | On when the environment is quiet (noise gate active for \u003e1 second) |\n| Bass Energy | sensor (0-1) | ~50ms | Smoothed, AGC-normalized bass band energy |\n| Mid Energy | sensor (0-1) | ~50ms | Smoothed, AGC-normalized mid band energy |\n| High Energy | sensor (0-1) | ~50ms | Smoothed, AGC-normalized high band energy |\n| Amplitude | sensor (0-1) | ~50ms | Overall smoothed amplitude with dynamics limiting |\n| BPM | sensor | ~1s | Estimated beats per minute from autocorrelation beat tracker |\n| Beat Confidence | sensor (0-1) | ~1s | Confidence in the current BPM estimate (0 = unknown, 1 = locked) |\n| Beat Phase | sensor (0-1) | ~50ms | Position within the current beat cycle (0 = on beat, approaches 1 before next beat) |\n| Spectral Centroid | sensor (0-1) | ~50ms | Spectral \"brightness\" — weighted average frequency of the spectrum |\n| Spectral Rolloff | sensor (0-1) | ~50ms | Frequency below which 85% of spectral energy is concentrated |\n| Onset Strength | sensor (0-1) | On event | Magnitude of the most recent onset detection (0 = weak, 1 = strong) |\n| Beat Sensitivity | number (1-100) | On change | Controls onset detection threshold |\n| Squelch | number (0-100) | On change | Noise gate threshold (higher = requires louder signal) |\n| Detection Mode | select | On change | `spectral_flux` (all genres), `bass_energy` (rhythmic), or `complex_domain` (phase+magnitude) |\n| Microphone Mute | switch | On change | Mute/unmute the microphone (LED turns red when muted) |\n| Reset AGC | button | On press | Resets automatic gain control and onset detector |\n| Calibrate Quiet Room | button | On press | Calibrates noise floor from quiet room (3 seconds) |\n| Calibrate Music Level | button | On press | Calibrates signal scaling from music playback (5 seconds) |\n| Status LED | light | — | On-device RGB LED (ATOM Echo) |\n\n## Configuration\n\n```yaml\naudio_reactive:\n  id: audio_analysis\n  microphone: mic                 # Required: I2S microphone component ID\n  update_interval: 50ms           # Processing interval (default: 50ms)\n  beat_sensitivity: 50            # 1-100, higher = reacts to quieter onsets (default: 50)\n  squelch: 10                     # 0-100, noise gate threshold (default: 10)\n  sample_rate: 22050              # Sample rate in Hz, must match microphone config (default: 22050)\n  fft_size: 512                   # FFT window size: 256 or 512 (default: 512)\n  debug_logging: false            # Enable comprehensive DSP pipeline logging (default: false)\n\n  # Automation triggers (all optional)\n  on_mute_changed:                # Fired when mute state changes (button, switch, or HA)\n    - ...\n  on_quiet_calibration_started:   # Fired when quiet room calibration begins\n    - ...\n  on_quiet_calibration_complete:  # Fired when quiet room calibration finishes\n    - ...\n  on_music_calibration_started:   # Fired when music calibration begins\n    - ...\n  on_music_calibration_complete:  # Fired when music calibration finishes\n    - ...\n  on_silence_changed:             # Fired when silence state transitions\n    - ...\n\n# Sensors (all optional — include only what you need)\nsensor:\n  - platform: audio_reactive\n    audio_reactive_id: audio_analysis\n    bass_energy:\n      name: \"Bass Energy\"\n    mid_energy:\n      name: \"Mid Energy\"\n    high_energy:\n      name: \"High Energy\"\n    amplitude:\n      name: \"Amplitude\"\n    bpm:\n      name: \"BPM\"\n    beat_confidence:\n      name: \"Beat Confidence\"\n    beat_phase:\n      name: \"Beat Phase\"\n    centroid:\n      name: \"Spectral Centroid\"\n    rolloff:\n      name: \"Spectral Rolloff\"\n    onset_strength:\n      name: \"Onset Strength\"\n\n# Binary sensors\nbinary_sensor:\n  - platform: audio_reactive\n    audio_reactive_id: audio_analysis\n    onset_detected:\n      name: \"Audio Sensor\"\n    silence:\n      name: \"Silence\"\n\n# Control entities\nnumber:\n  - platform: audio_reactive\n    audio_reactive_id: audio_analysis\n    beat_sensitivity:\n      name: \"Beat Sensitivity\"\n    squelch:\n      name: \"Squelch\"\n\nselect:\n  - platform: audio_reactive\n    audio_reactive_id: audio_analysis\n    detection_mode:\n      name: \"Detection Mode\"      # spectral_flux, bass_energy, or complex_domain\n\nswitch:\n  - platform: audio_reactive\n    audio_reactive_id: audio_analysis\n    microphone_mute:\n      name: \"Microphone Mute\"\n\nbutton:\n  - platform: audio_reactive\n    audio_reactive_id: audio_analysis\n    reset_agc:\n      name: \"Reset AGC\"\n    calibrate_quiet:\n      name: \"Calibrate Quiet Room\"\n    calibrate_music:\n      name: \"Calibrate Music Level\"\n```\n\nSee the device YAML files and [example-annotated.yaml](example-annotated.yaml) for complete working examples with feedback wiring.\n\n## Debug Logging\n\nEnable comprehensive DSP pipeline logging for troubleshooting:\n\n```yaml\naudio_reactive:\n  debug_logging: true\n```\n\nWhen enabled, logs every 2 seconds: raw FFT magnitudes, scaled values, AGC gains, silence state, calibration state, published sensor values, sample rate, FFT size, and ring buffer fill level. Disable for production use.\n\n## Calibration\n\nFor best results, calibrate the device to your environment. Calibration data persists across reboots.\n\n### Quiet room calibration (recommended)\n\nEnsures the device correctly identifies silence and doesn't react to ambient noise.\n\n1. Make sure the room is quiet (no music, minimal background noise)\n2. **Double-click** the device button (Button A on M5StickC Plus2), or press **Calibrate Quiet Room** in Home Assistant\n3. The device shows green feedback (LED or screen) for 3 seconds while sampling\n4. Feedback flashes briefly to confirm calibration is complete\n\nThis sets the noise gate threshold and per-band noise floors based on your room's actual ambient noise level.\n\n### Music level calibration (recommended)\n\nTeaches the device what typical music levels look like in your setup, so the sensors produce a useful 0-1 range.\n\n1. Play music at your typical listening volume\n2. **Triple-click** the device button (Button A on M5StickC Plus2), or press **Calibrate Music Level** in Home Assistant\n3. The device shows blue feedback (LED or screen) for 5 seconds while sampling\n4. Feedback flashes briefly to confirm calibration is complete\n\nThis sets the signal scaling factor so that typical music maps to mid-range sensor values (~0.5), giving the AGC room to normalize both quiet and loud passages.\n\n### Calibration order\n\nRun quiet room calibration first, then music calibration. If you change rooms, speaker setup, or device placement, re-run both calibrations.\n\n## Button Actions\n\n### ATOM Echo / ATOM Echo S3R\n\nThese devices have a single button — actions are distinguished by click pattern:\n\n| Action | ATOM Echo | ATOM Echo S3R |\n|--------|-----------|---------------|\n| **Double click** | Calibrate quiet (green LED) | Calibrate quiet (speaker tone) |\n| **Triple click** | Calibrate music (blue LED) | Calibrate music (speaker tone) |\n| **Long press (1s+)** | Toggle mute (red LED) | Toggle mute (speaker tone) |\n\n### M5StickC Plus2\n\nButton A (front) handles all actions via click pattern. Button B (side) is exposed for custom use.\n\n| Action | Button A | Feedback |\n|--------|----------|----------|\n| **Double click** | Calibrate quiet room | Green screen |\n| **Triple click** | Calibrate music level | Blue screen |\n| **Long press (1s+)** | Toggle mute | Red screen |\n\n### Waveshare ESP32-S3 Audio Board\n\nThe Waveshare has three dedicated buttons (K1, K2, K3) — one action per button:\n\n| Button | Action | Feedback |\n|--------|--------|----------|\n| **K1** | Calibrate quiet room | Green LEDs |\n| **K2** | Calibrate music level | Blue LEDs |\n| **K3** | Toggle mute | Red LEDs |\n\n## Detection Modes\n\n### Spectral flux (default)\n\nDetects any sudden change in the frequency spectrum — kick drums, snare hits, cymbal crashes, piano attacks, violin pizzicato, vocal entrances. Works with all music genres including classical, jazz, and ambient.\n\n### Bass energy\n\nOnly detects bass energy threshold crossings. Optimized for rhythmic music with a prominent low-frequency beat (EDM, pop, rock, hip-hop). Includes hysteresis to prevent rapid re-triggering.\n\n### Complex domain\n\nUses both phase and magnitude information (Dixon 2006 phase advance algorithm) with spectral whitening. Distinguishes transients from sustained tones, making it better at detecting soft or subtle onsets — gentle percussion, fingerpicked guitar, or quiet vocal entrances that spectral flux might miss.\n\nSwitch between modes via the Detection Mode select entity, or the integration sets it automatically per scene.\n\n## How It Works\n\n1. Audio is captured at the configured sample rate (default 22,050 Hz for ATOM Echo, 44,100 Hz for S3R and Waveshare devices)\n2. Ring buffer feeds samples to a dedicated FreeRTOS FFT task on core 0\n3. Configurable FFT window (256, 512, or 1024 samples; default 512) with 75% overlap produces frequency magnitudes\n4. 16 frequency bands are computed with pink noise correction\n5. PI-controller AGC normalizes energy values with configurable attack/release\n6. Spectral flux onset detection identifies musical events across all frequency bands\n7. Dynamics limiter and asymmetric EMA smoothing prevent jittery sensor output\n8. Silence detector gates output when mid+high energy is below the calibrated threshold","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabsent42%2Fesphome-audio-reactive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fabsent42%2Fesphome-audio-reactive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fabsent42%2Fesphome-audio-reactive/lists"}