https://github.com/felixgalindo/tinymldelta
TinyMLDelta is an incremental model-update system for TinyML and embedded AI devices.
https://github.com/felixgalindo/tinymldelta
edgeai embedded-systems tensorflow tensorflow-lite tinyml
Last synced: 4 months ago
JSON representation
TinyMLDelta is an incremental model-update system for TinyML and embedded AI devices.
- Host: GitHub
- URL: https://github.com/felixgalindo/tinymldelta
- Owner: felixgalindo
- License: other
- Created: 2025-11-22T19:44:21.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2025-11-22T23:51:21.000Z (4 months ago)
- Last Synced: 2025-11-23T01:16:38.328Z (4 months ago)
- Topics: edgeai, embedded-systems, tensorflow, tensorflow-lite, tinyml
- Language: C
- Homepage: https://medium.com/@felixgalindo91/introducing-tinymldelta-incremental-ml-model-updates-for-tiny-devices-96663edd1991
- Size: 42 KB
- Stars: 2
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project
README
# TinyMLDelta
TinyMLDelta is an **incremental model-update system** for TinyML and embedded AI devices.
Instead of shipping an entire TensorFlow Lite Micro model (often **20–200+ KB**),
you ship a **small binary patch** that mutates the existing model in flash into a new one.
This reduces:
- OTA bandwidth
- Cellular / satellite data costs
- Flash wear
- Update latency
- Fleet fragmentation
- Bootloader complexity
TinyMLDelta performs **safe, atomic, guardrail-checked updates** on
extremely low-resource MCUs.
------------------------------------------------------------------------
## Supported Today
- TensorFlow Lite Micro
- POSIX/macOS simulated flash environment
- CRC32 integrity checking
- A/B slot updates
- Crash-safe journaling
## Planned
- Edge Impulse frontend
- SHA-256 and AES-CMAC signatures
- Model versioning
- Vendor metadata
- MCU integrations (Zephyr, Arduino UNO R4 WiFi, Tachyon)
------------------------------------------------------------------------
## Patch vs Firmware Update --- What Requires What?
TinyMLDelta can safely update models **only when the firmware remains
compatible** with the new model.
Compatibility is enforced using **metadata TLVs** generated by PatchGen
and validated by the MCU runtime.
### ✔ Patch-friendly changes (safe, incremental)
No firmware update required:
- Weight updates
- Bias updates
- Quantization parameter changes
- Re-training the same architecture
- Minor graph edits without operator changes
- Same opset, same ABI, same I/O schema
- Similar tensor shapes
- Same input/output dtypes
These produce compact patches.
## ❌ Changes requiring a full firmware update
| Change Type | Firmware Update Required? | Why |
|------------|---------------------------|-----|
| **New operators** | ✔ Yes | Firmware must link kernels |
| **Opset version changes** | ✔ Yes | Operator implementations differ |
| **TFLM ABI changes** | ✔ Yes | Interpreter ABI mismatch |
| **Arena requirement > compiled arena** | ✔ Yes | Arena lives in static `.bss`; fixed at build |
| **Tensor I/O schema changes** | ✔ Yes | Application code relies on shapes/dtypes |
| **Flex ops / newer TFLite schema** | ✔ Yes | Requires Flex delegate or newer interpreter |
TinyMLDelta rejects incompatible patches automatically.
------------------------------------------------------------------------
## Architecture Overview
PatchGen (PC/CI) runs off-target and is **stateless**.
TinyMLDelta Core enforces all safety on-device.
### PatchGen (PC/CI)
- Loads the base & target models
- Computes byte-level differences
- Merges and compresses diff chunks
- Calculates integrity digests
- Extracts metadata (arena, ABI, opset hash, I/O hash)
- Assembles everything into a `.tmd` patch
### TinyMLDelta Core (MCU)
- Parses the patch header & TLVs
- Verifies compatibility guardrails
- Copies active → inactive slot
- Applies diff chunks to the inactive slot
- Verifies CRCs / digests
- Atomically flips the active slot
### High-Level Data Flow
┌────────────────────────┐
│ Base Model (flash) │
└──────────┬─────────────┘
│ read
▼
┌────────────────────────┐
│ Patch Generator (PC) │
│ • diff engine │
│ • metadata TLVs │
│ • integrity checks │
└──────────┬─────────────┘
│ OTA send
▼
┌────────────────────────┐
│ TinyMLDelta Core (MCU) │
│ • copy active→inactive│
│ • apply diff chunks │
│ • verify CRC/digests │
│ • enforce guardrails │
└─────────┬───────────── ┘
│ atomically flip
▼
┌────────────────────────┐
│ Target Model (flash) │
└────────────────────────┘
The MCU **never** receives a full model --- only a delta.
------------------------------------------------------------------------
## Wire Format
### Patch Header (`tmd_hdr_t`)
``` c
typedef struct __attribute__((packed)) {
uint8_t v;
uint8_t algo;
uint16_t chunks_n;
uint32_t base_len;
uint32_t target_len;
uint8_t base_chk[32];
uint8_t target_chk[32];
uint16_t meta_len;
uint16_t flags;
} tmd_hdr_t;
```
### Metadata TLVs
Each TLV is encoded as:
[tag][len][value...]
Current TLVs:
- `REQ_ARENA_BYTES` (u32)
- `TFLM_ABI` (u16)
- `OPSET_HASH` (u32)
- `IO_HASH` (u32)
Planned TLVs:
- Model version, model family/architecture
- Certificate + signature blocks
- Dataset lineage
- Firmware feature flags
- Strict/relaxed validation modes
### Chunk Header (`tmd_chunk_hdr_t`)
``` c
typedef struct __attribute__((packed)) {
uint32_t off;
uint16_t len;
uint8_t enc; // 0 = RAW, 1 = RLE
uint8_t has_crc;
} tmd_chunk_hdr_t;
```
------------------------------------------------------------------------
## Installation (CLI)
``` bash
cd cli/
./install.sh
source .tinyenv/bin/activate
```
Installer output example:
=== TinyMLDelta CLI Installer ===
Creating virtual environment .tinyenv ...
Installing TinyMLDelta CLI requirements...
...
TinyMLDelta CLI environment is ready!
------------------------------------------------------------------------
## Quickstart: Run the POSIX Demo
The POSIX demo simulates a full MCU update flow end-to-end:
model generation → patch generation → flash creation → patch apply →
verification.
A **full standalone walkthrough** (with complete logs, explanations, and
verification steps) is here:
👉 **`examples/posix/README.md`**
To run the demo:
``` bash
cd cli
./install.sh
source .tinyenv/bin/activate
cd ../examples/posix
./run_demo.sh
```
------------------------------------------------------------------------
### Demo Results
- **Base model:** 66,368 bytes
- **Target model:** 66,368 bytes
- **Patch file:** 474 bytes total
- **Encoded diff data:** 382 bytes
- **Single diff chunk:** offset 62,728, length 382 bytes
- **Flash slots:** 131,072 bytes each (A/B)
**Result:** patch applied successfully and flash matches the target
model exactly.
------------------------------------------------------------------------
## Directory Layout
```text
TinyMLDelta/
├── cli/
│ ├── install.sh # Optional: create a local venv + install CLI deps
│ ├── requirements.txt # Python deps for PatchGen + demo tooling
│ ├── tinymldelta_patchgen.py # PatchGen: build .tmd patches from base/target models
│ └── tinymldelta_meta_compute.py# (optional) TFLite-aware metadata helper (future use)
│
├── examples/
│ ├── modelgen/
│ │ ├── make_models.py # Generates small TFLite sensor models for the demo
│ │ ├── README.md # Notes on model generation / usage
│ │ └── (base.tflite, target.tflite are generated artifacts)
│ │
│ └── posix/
│ ├── README.md # Notes on running demo
│ ├── Makefile # Builds demo_apply using POSIX ports + core
│ ├── run_demo.sh # End-to-end demo: generate models → patch → apply → verify
│ ├── make_flash.py # Creates flash.bin with A/B slots + journal region
│ ├── flash_layout.h # Simulated flash layout (tmd_layout_t) for the demo
│ ├── demo_apply.c # TinyMLDelta POSIX sample app (patch applier)
│ ├── tinymldelta_ports_posix.c # POSIX implementation of tmd_ports_t (flash/journal/log)
│ ├── verify_flash.py # Verifies flash.bin contains the exact target.tflite bytes
│ └── (flash.bin, patch.tmd, *.o, demo_apply are build/generated artifacts)
│
├── runtime/
│ ├── include/
│ │ ├── tinymldelta.h # Public C API for TinyMLDelta core
│ │ ├── tinymldelta_config.h # Build-time feature flags + firmware guardrail config
│ │ ├── tinymldelta_internal.h # On-wire header / TLV / chunk structures
│ │ └── tinymldelta_ports.h # Platform abstraction: flash, digests, slots, journal, log
│ └── src/
│ └── tinymldelta_core.c # Platform-agnostic patch application engine
│
├── LICENSE # Apache-2.0 license
├── SECURITY.md # Security contact / disclosure policy
└── README.md # Project overview and usage
```
------------------------------------------------------------------------
## Contributing
Contributions welcome:
- New front-ends (Edge Impulse)
- Optimized diff algorithms
- Additional MCU ports (Zephyr, Arduino, STM32, ESP32)
- Metadata TLV extensions
- Docs improvements
- Security signing pipeline
------------------------------------------------------------------------
## License
Apache-2.0\
© 2024--2025 TinyMLDelta Project / Felix Galindo