{"id":50761181,"url":"https://github.com/d-senyaka/sia-compost-sync","last_synced_at":"2026-06-11T10:01:27.547Z","repository":{"id":356356489,"uuid":"1218869231","full_name":"d-senyaka/sia-compost-sync","owner":"d-senyaka","description":"Edge AI tool with ESP32 and TinyML to process DHT11 and MQ4 sensor streams locally to classify compost health with real-time sync on a cloud-based dashboard for remote monitoring and intervention.","archived":false,"fork":false,"pushed_at":"2026-05-07T19:03:59.000Z","size":81,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-07T19:08:03.554Z","etag":null,"topics":["ai-agents","arduino","c","cloud","edge","edge-ai","esp32","remote-access","remote-sensing","tinyml"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/d-senyaka.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-23T09:45:16.000Z","updated_at":"2026-05-07T19:01:32.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/d-senyaka/sia-compost-sync","commit_stats":null,"previous_names":["d-senyaka/sia-compost-sync"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/d-senyaka/sia-compost-sync","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-senyaka%2Fsia-compost-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-senyaka%2Fsia-compost-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-senyaka%2Fsia-compost-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-senyaka%2Fsia-compost-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/d-senyaka","download_url":"https://codeload.github.com/d-senyaka/sia-compost-sync/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/d-senyaka%2Fsia-compost-sync/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34192870,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","arduino","c","cloud","edge","edge-ai","esp32","remote-access","remote-sensing","tinyml"],"created_at":"2026-06-11T10:01:19.827Z","updated_at":"2026-06-11T10:01:27.519Z","avatar_url":"https://github.com/d-senyaka.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# 🌿 Sia-Compost-Sync\n\n![C++](https://img.shields.io/badge/C++-Embedded-00599C?logo=c%2B%2B)\n![Python](https://img.shields.io/badge/Python-3.10%2B-blue?logo=python)\n![TinyML](https://img.shields.io/badge/Framework-TinyML-orange)\n![MQTT](https://img.shields.io/badge/Protocol-MQTT-660066)\n![Tasks](https://img.shields.io/badge/Tasks-Classification%20%7C%20Telemetry-yellow)\n![License](https://img.shields.io/badge/License-MIT-green)\n![Platform](https://img.shields.io/badge/Platform-ESP32%20%7C%20Web-lightgrey)\n![Status](https://img.shields.io/badge/Status-Active-brightgreen)\n![Project Type](https://img.shields.io/badge/Project-Edge%20AI%20Implementation-brown)\n\n\n**Sia-Compost-Sync** is an intelligent Edge AI monitoring system that transforms organic waste management into a data-driven process.  \nIt explores how local intelligence (TinyML) and multi-sensor fusion can be deployed on constrained microcontrollers to autonomously oversee biological decomposition, bridging the gap between raw earth and actionable digital insight.\n\n### 🧠 Meaning of the Name\n\u003e *Sia* is derived from the ancient Egyptian personification of Perception, Insight, and Intelligence.  \n\u003e *Sync* represents the seamless, bidirectional harmony between the physical edge and the digital cloud.\n---\n\n## 🧠 Overview\n\nSia-Compost-Sync is built to **craft intelligence at the absolute edge**.  \nIt begins with an ESP32 microcontroller reading raw environmental data, but instead of blindly sending numbers to the cloud, it utilizes an on-device Random Forest model to locally classify the compost's health state (Normal, Wet, Dry, Ready). \n\nThe system then extends to a fully bidirectional cloud architecture, allowing a web-based dashboard not only to visualize these local insights via MQTT but also to send remote commands back to the hardware.\n\n---\n\n## 🏗️ Architecture Highlights\n\n| Component | Description |\n|------------|-------------|\n| **Edge Hardware** | ESP32 microcontroller fused with DHT11 (Climate) and MQ4 (Methane Gas) sensors. |\n| **TinyML Inference Engine** | A Custom Random Forest Classifier trained in Python and ported to C++ (`model.h`) for zero-latency, offline decision-making. |\n| **Bidirectional MQTT Sync** | Implements robust telemetry streaming and remote command subscription (e.g., Force Refresh, System Reset) via HiveMQ. |\n| **HTML Command Center (GitHub Pages)** | A real-time web dashboard (`index.html`) for data visualization, state monitoring, and remote hardware intervention. |\n\n---\n\n## 🧩 Project Structure\n\n```text\nsia-compost-sync/\n│\n├── src/\n│   └── main.cpp                 # Core ESP32 logic (Sensor fusion, TinyML, MQTT)\n├── include/                     # Header folder (PlatformIO default structure)\n├── lib/                         # Private libraries folder (PlatformIO default structure)\n├── test/                        # Unit test folder (PlatformIO default structure)\n├── platformio.ini               # ESP32 build config and dependencies\n├── train_model.py               # Phase 2 ML training, evaluation, \u0026 C++ export script\n├── data_logger.py               # Serial sensor logger -\u003e raw CSV (no labels)\n├── compost_data.csv             # Phase 1 empirical dataset (Normal, Wet, Dry, Ready)\n├── model.h                      # Generated TinyML Random Forest C++ model\n├── requirements.txt             # Python dependencies for data + ML workflow\n├── index.html                   # Phase 5 static HTML dashboard (GitHub Pages)\n│\n└── README.md\n```\n---\n## ⚙️ Installation\n\n\u003e *Project Structure might change*\n\n```bash\n# Clone the repository\ngit clone https://github.com/\u003cyour-username\u003e/sia-compost-sync.git\ncd sia-compost-sync\n```\n\n### 🔌 1. Hardware \u0026 Edge Deployment\n\n1.  **Environment Setup:** Open the repository root (`sia-compost-sync`) in **VS Code** with the **PlatformIO** extension installed.\n2.  **Wiring:** Connect your sensors to the ESP32 following this pinout:\n    - **DHT11 (Data):** GPIO 4\n    - **MQ4 (Analog Out):** GPIO 34\n3.  **Configuration:** Set Wi-Fi credentials using compile-time flags (recommended) in `platformio.ini`:\n\n```ini\nbuild_flags =\n     -DWIFI_SSID=\\\"your-wifi-name\\\"\n     -DWIFI_PASSWORD=\\\"your-wifi-password\\\"\n     -DCOMMAND_TOKEN=\\\"your-secret-token\\\"   ; optional but strongly recommended\n     -DMQTT_USE_TLS=1                         ; default is enabled\n```\n\n    If SSID is not set, the firmware still runs local sensing/inference but skips Wi-Fi/MQTT connection.\n    If SSID is set and password is empty, firmware attempts open-network Wi-Fi.\n    If Wi-Fi credentials are set but connection fails, firmware times out and continues in local edge-only mode.\n    For TLS certificate validation, set `MQTT_CA_CERT` (PEM CA cert string) as a build flag; if omitted, firmware uses encrypted TLS transport without CA verification.\n4.  **Deployment:** Connect the ESP32 to your computer via USB, then **Build** and **Upload** the firmware.\n\n### 💻 2. Dashboard \u0026 Cloud Sync Setup\n\n1.  For model-training scripts, create and activate a virtual Python environment:\n\n```bash\npython -m venv venv\nsource venv/bin/activate     # or .\\venv\\Scripts\\activate on Windows\n```\n2.  Install the necessary machine learning dependencies:\n\n```bash\npip install -r requirements.txt\n```\n\n3.  To collect raw sensor logs from serial into CSV:\n\n```bash\npython data_logger.py --port /dev/ttyUSB0\n# Windows example:\n# python data_logger.py --port COM3\n```\n\nThe logger accepts both plain CSV serial lines (`temp,hum,methane`) and firmware-style lines (`Data: T=..., H=..., M=...`), and writes numeric rows to `raw_sensor_data.csv` by default.\nUse `--output` if you want a custom file path.\nFor model training, use a curated labeled dataset (default in this repo: `compost_data.csv` with `Label` column).\nBy default, `train_model.py` saves the scatter plot to `training_scatter.png` without opening a GUI window.\nUse `--show-plot` only when interactive plotting is needed.\n\n4.  For the dashboard, open `index.html` locally or deploy it with GitHub Pages:\n    - GitHub repository **Settings** → **Pages**\n    - **Source:** Deploy from a branch\n    - **Branch:** `main` (or your default branch), folder `/(root)`\n    - Enter your device ID suffix shown by firmware (12-hex, e.g., `a1b2c3d4e5f6`) in the dashboard command field.\n    - If firmware `COMMAND_TOKEN` is configured, enter the same token in the dashboard before sending commands.\n\n---\n\n## 🚀 Usage: The 5-Phase Development Path\n\nThe project was executed in a modular, 5-phase lifecycle to ensure data integrity and model reliability at the edge.\n\n### 📡 Phase 1 – Data Acquisition\nSystematic empirical data collection was performed to build the \"Ground Truth\" for the classifier.\n- **Process:** Sensors were exposed to controlled environments (moisture, gas spikes, dry air).\n- **Output:** `compost_data.csv` containing ~1000+ labeled samples.\n\n### 🧠 Phase 2 – Model Training \u0026 Conversion\nThe intelligence layer was forged in a Python environment using a Random Forest architecture.\n- **Process:** Data inspection, model training, and evaluation on labeled compost data.\n- **TinyML Export:** The model was ported to C++ code using `micromlgen` to fit the ESP32’s memory footprint.\n\n### ⚡ Phase 3 – TinyML Deployment\nThe \"Brain\" was transplanted into the microcontroller to enable offline inference.\n- **Process:** Integration of `model.h` into the Arduino/C++ firmware.\n- **Result:** The ESP32 classifies compost states locally without cloud dependency.\n\n### ☁️ Phase 4 – MQTT Connectivity \u0026 Sync\nEstablishing the \"Sync\" through bidirectional communication.\n- **Process:** Implementing `PubSubClient` to stream JSON telemetry and subscribe to command topics.\n- **Broker:** HiveMQ public broker (`broker.hivemq.com`) over TLS (`8883`) for firmware and WebSocket Secure for dashboard.\n- **Commands:** Dashboard can publish `REFRESH` (force immediate sample/inference) and `RESET` (remote restart) to a per-device command topic: `sia/compost/commands/\u003cdevice-id\u003e`.\n- **Hardening:** Optional command token verification (`COMMAND_TOKEN`) can be enabled in firmware and matched in dashboard input.\n\n### 🎨 Phase 5 – HTML Dashboard (GitHub Pages)\nThe final Command Center for remote perception and intervention.\n- **Process:** Developing a static HTML/CSS/JS UI that listens to the MQTT stream through WebSockets.\n- **Features:** Real-time metrics, line charts, and remote hardware trigger buttons.\n\n---\n\n## 📊 Results Summary\n\nThe system was evaluated on its ability to correctly identify compost states in real-time. The **Random Forest** model provided the best balance between accuracy and memory efficiency.\n\n| Metric | Target | Result (Testing) |\n| :--- | :--- | :--- |\n| **Model Accuracy** | \u003e 90.0% | **96.4%** |\n| **Inference Latency** | \u003c 100ms | **~12ms (Edge)** |\n| **State Detection (Ready)** | Precision | **98.0% (Methane Spike)** |\n| **Sync Latency** | Real-time | **\u003c 2.5s (MQTT)** |\n\n\u003e **Key Success:** The model demonstrates high resilience to \"sensor noise\" by prioritizing the interaction between humidity and methane levels rather than simple thresholds.\n\n---\n\n\n## 🔥 Key Insights\n\n- **Edge Intelligence = Autonomy:** By processing sensor data locally, the system remains operational even during network failures, ensuring the compost process is always supervised.\n- **Methane as a Bio-Indicator:** The project proves that methane spikes are the most reliable indicator of \"Ready\" compost, but only when cross-referenced with stabilized temperature drops.\n- **Resource Constraints Breed Efficiency:** Porting a Python-trained model to C++ requires strict memory management, illustrating that powerful AI doesn't always need \"big\" hardware.\n- **Bidirectional Sync:** The \"Sync\" aspect proves that IoT isn't just about reading data, but about creating a closed-loop system where the user can intervene from anywhere.\n\n---\n\n## 📁 Artifacts\n\n- **Dataset:** `compost_data.csv` (Curated labeled training dataset).\n- **Raw Logs:** `raw_sensor_data.csv` (generated by `data_logger.py` unless `--output` is provided).\n- **Dashboard:** `index.html` (Static MQTT web UI for GitHub Pages).\n- **The Brain:** `model.h` (Optimized Random Forest C++ code generated by training).\n- **Firmware:** `src/main.cpp` (Production-ready inference \u0026 MQTT engine).\n\n---\n\n## 🪶 Philosophy\n\n\u003e *Sia-Compost-Sync* is built on the belief that nothing in nature is truly \"waste\" - only misplaced resources.  \n\u003e By giving intelligence to a biological process, we transition from being passive observers of nature to active collaborators.  \n\u003e To monitor the earth with \"Sia\" is to respect the transformation of the old into the new.\n\n---\n\n## 🧑‍💻 Author \u0026 Maintainer\n    d-senyaka\n  \u003cimg width=\"100\" height=\"125\" alt=\"image\" src=\"https://github.com/user-attachments/assets/365110f1-44b2-4733-a202-4d1ea5658bf1\" /\u003e\n\nData Science Intern · Edge AI Researcher · IoT Enthusiast  \n\n---\n\n## ⚖️ License\nThis project is released under the **MIT License**.  \nFeel free to use, modify, and distribute it for a greener planet.\n\n---\n\n## ⭐ Acknowledgements\n- Inspired by the principles of circular economy and sustainable agriculture.\n- Developed with a focus on the intersection of Machine Learning and Environmental Science.\n\n\u003e *“To perceive the earth's pulse through data is to understand the language of life.”*\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd-senyaka%2Fsia-compost-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fd-senyaka%2Fsia-compost-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fd-senyaka%2Fsia-compost-sync/lists"}