{"id":30105149,"url":"https://github.com/mzakharo/lg_detector","last_synced_at":"2026-06-22T02:32:04.075Z","repository":{"id":308257233,"uuid":"1032173279","full_name":"mzakharo/lg_detector","owner":"mzakharo","description":"Detect LG Appliance Completion with a Microphone, Machine Learning, and ESP32","archived":false,"fork":false,"pushed_at":"2025-08-04T23:55:03.000Z","size":1920,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-08-05T01:21:04.340Z","etag":null,"topics":["audio-processing","esp32","lg-appliances","machine-learning"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mzakharo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-04T23:39:56.000Z","updated_at":"2025-08-04T23:55:07.000Z","dependencies_parsed_at":"2025-08-05T01:21:05.853Z","dependency_job_id":"7edee452-8b66-402d-8110-a4a46a547517","html_url":"https://github.com/mzakharo/lg_detector","commit_stats":null,"previous_names":["mzakharo/lg_detector"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/mzakharo/lg_detector","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mzakharo%2Flg_detector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mzakharo%2Flg_detector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mzakharo%2Flg_detector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mzakharo%2Flg_detector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mzakharo","download_url":"https://codeload.github.com/mzakharo/lg_detector/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mzakharo%2Flg_detector/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34632550,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-22T02:00:06.391Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["audio-processing","esp32","lg-appliances","machine-learning"],"created_at":"2025-08-09T23:46:25.429Z","updated_at":"2026-06-22T02:32:04.070Z","avatar_url":"https://github.com/mzakharo.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LG Appliance Sound Detection System\n\nA machine learning-powered system that detects the completion melody of LG washing machines and other appliances using audio analysis. The system consists of a Python-based training and real-time detection pipeline, plus an ESP32-based embedded detector for standalone operation.\n\n## Overview\n\nThis project uses deep learning to identify the distinctive melody that LG appliances play when they complete their cycles. The system can run in two modes:\n- **Python Real-time Detector**: Uses a computer's microphone or MQTT data stream\n- **ESP32 Embedded Detector**: Standalone device with PDM microphone and WiFi connectivity\n- **Tested** with Seeed Studio XIAO ESP32S3 Sense board\t\n- **Home Assistant** integration over MQTT\n\n## Features\n\n- **Real-time Audio Processing**: Converts audio to mel spectrograms for ML inference\n- **Confidence-based Detection**: Uses accumulative scoring to reduce false positives\n- **Multiple Input Sources**: Microphone input or MQTT streaming\n- **ESP32 Implementation**: Low-power embedded solution with WiFi/MQTT\n- **Visualization**: Real-time spectrogram display for debugging\n- **Model Training Pipeline**: Complete workflow from data collection to deployment\n\n## System Architecture\n\n```\nAudio Input → Spectrogram Generation → ML Model → Confidence Accumulator → Detection Alert\n```\n\n### Key Components\n\n1. **Audio Processing**: 16kHz sampling, 1.5s windows, 64 mel bands\n2. **CNN Model**: Lightweight 3-layer CNN optimized for embedded deployment\n3. **Confidence System**: Accumulative scoring with cooldown periods\n4. **Communication**: MQTT for remote monitoring and data streaming\n\n## Quick Start\n\n### Python Environment Setup\n\n```bash\n# Install dependencies\npip install -r requirements.txt\n\n# Run real-time detection with microphone\npython realtime_detector.py --input mic --visualize\n\n# Run with MQTT input (for ESP32 integration)\npython realtime_detector.py --input mqtt --broker nas.local --topic lg-detector/spectrogram\n```\n\n### ESP32 Setup\n\n1. **Hardware Requirements**:\n   - ESP32-S3 or similar with PSRAM\n   - PDM microphone (CLK: GPIO42, DATA: GPIO41)\n   - WiFi connectivity\n\n2. **Build and Flash**:\n    - source esp-idf (v5.4 tested)\n   ```bash\n   cd esp-lg-detector\n   idf.py set-target esp32s3\n   idf.py build\n   idf.py flash monitor\n   ```\n\n3. **Configuration**:\n   - Update WiFi credentials in `main.cpp`\n   - Set MQTT broker address\n   - Adjust detection thresholds if needed\n\n4. **Home Assistant**:\n    - Ensure MQTT integration is configured\n    - in configuration.yaml\n    ```yaml\n    mqtt:\n            sensor:\n                - name: LG_conf \n                state_topic: lg-detector/detection\n                unit_of_measurement: \"%\"\n                unique_id: lg_conf \n                value_template: \"{{ value_json.confidence_score }}\"\n                force_update: true\n                expire_after: 60\n    ```\n    - Perform a quick restart of Home Assistant to reload MQTT entities\n    - Add Helper Entity. Threshold. Set Trigger threshold % (60). \n\n## Model Training\n\nThe system uses a CNN trained on mel spectrograms:\n\n### Data Structure\n```\ndata/\n├── lg_melody/          # Target LG appliance sounds\n│   ├── lg.wav\n│   ├── lg_washer.wav\n│   └── lg_washer_machine.wav\n└── other_sounds/       # Background/negative samples\n    └── ... (150+ samples)\n```\n\n### Training Process\n\n1. **Data Preprocessing**: Audio files → Mel spectrograms (64×48)\n2. **Model Architecture**: 3-layer CNN with dropout\n3. **Training**: 30 epochs with early stopping\n4. **Conversion**: Keras → TensorFlow Lite for deployment\n\nRun the training notebook:\n```bash\njupyter notebook notebook.ipynb\n```\n\n### ESP32 Operation\nThe ESP32 automatically:\n1. Connects to configured WiFi network\n2. Starts PDM microphone capture\n3. Processes audio in real-time\n4. Publishes detection results via MQTT\n5. Logs detection events to serial console\n\n### MQTT Topics\n- `lg-detector/spectrogram`: Raw spectrogram data (ESP32 → Python)\n- `lg-detector/detection`: Detection results JSON\n\n## Model Performance\n\n- **Input**: 64×48 mel spectrogram (1.5s audio window)\n- **Output**: Binary classification [other_sounds, lg_melody]\n- **Model Size**: ~50KB TensorFlow Lite\n- **Memory Usage**: ~128KB tensor arena\n\n## Hardware Requirements\n\n### Python Version\n- Python 3.7+\n- Audio input device (microphone)\n\n### ESP32 Version\n- ESP32-S3 or similar with PSRAM\n- PDM microphone module\n- WiFi connectivity\n\n## Troubleshooting\n\n### Common Issues\n\n### Debug Mode\nEnable debug output in ESP32:\n```cpp\n#define DEBUG_MODE 1  // Sine wave test mode\n\n#define SEPCTOGRAM_DEBUG  // Output spectrogram over MQTT\n```\n\n### Visualization\nUse the `--visualize` flag to see real-time spectrograms and debug audio processing.\n\n## Contributing\n\n## Technical Details\n\n### Signal Processing Pipeline\n1. **Audio Capture**: 16kHz mono PCM\n2. **Windowing**: 1.5s Hann windows\n3. **FFT**: 1024-point real FFT\n4. **Mel Filtering**: 64-band mel filterbank (0-8kHz)\n5. **Log Conversion**: Power to dB scale\n6. **Normalization**: Reference-based scaling\n\n### Machine Learning Pipeline\n1. **Feature Extraction**: Mel spectrograms as 2D images\n2. **Model Architecture**: CNN with 3 conv layers + dense classifier\n3. **Training**: Supervised learning on labeled audio samples\n4. **Deployment**: TensorFlow Lite for cross-platform inference\n5. **Post-processing**: Confidence accumulation with temporal smoothing\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmzakharo%2Flg_detector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmzakharo%2Flg_detector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmzakharo%2Flg_detector/lists"}