https://github.com/edycutjong/scarecrow

🔌 Off-grid AI sentry: vision + rules + spoken alerts on a Raspberry Pi ≤4GB
https://github.com/edycutjong/scarecrow
computer-vision dorahacks edge-ai offline-ai qvac raspberry-pi sentry tts
Last synced: 10 days ago
JSON representation
🔌 Off-grid AI sentry: vision + rules + spoken alerts on a Raspberry Pi ≤4GB
Host: GitHub
URL: https://github.com/edycutjong/scarecrow
Owner: edycutjong
License: mit
Created: 2026-06-09T01:48:08.000Z (23 days ago)
Default Branch: main
Last Pushed: 2026-06-12T08:32:58.000Z (19 days ago)
Last Synced: 2026-06-12T09:24:05.937Z (19 days ago)
Topics: computer-vision, dorahacks, edge-ai, offline-ai, qvac, raspberry-pi, sentry, tts
Language: TypeScript
Size: 5.23 MB
Stars: 0
Watchers: 0
Forks: 0
Open Issues: 5
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Security: SECURITY.md
- Agents: AGENTS.md
Awesome Lists containing this project

README

          ## 🧑‍⚖️ For Judges

**The pitch (30s):** A $50 off-grid Raspberry Pi that *understands* a scene and *reasons* about a plain-English rule — "alert if a person is near the shed, ignore the dog" — then speaks the alert and pushes it to a phone. Fully offline, ≤4GB RAM, solar-powered. Not motion detection — multimodal reasoning on the edge.

**Run it in 30 seconds (works on any laptop, no Pi needed):**

```bash

npm install && npm start      # → open http://localhost:8080

```

The dashboard shows the live pipeline, camera HUD, rule editor, and event log. Add a rule or "teach" a known entity right from the UI.

**Where to look:**

| Judging signal | Where | Status |

|---|---|---|

| **Deep QVAC integration** (5 distinct APIs) | `completion` + multimodal, `ragSearch`, `textToSpeech`, `startQVACProvider` — see [Why ONLY QVAC](#-why-only-qvac) | ✅ real `@qvac/sdk` v0.10.2 |

| **Capability unlock** (RAG known-entity recall) | [`src/core/memory.ts`](src/core/memory.ts) — your car/pet won't trip it, a stranger will | ✅ |

| **≤4GB on retail hardware** | `/api/status` reports **real measured RSS**; single-model load→infer→unload | ✅ measured, not faked |

| **True offline** | `python3 scripts/verify_offline.py` — static cloud-import/URL scan + live network probe | ✅ |

| **Production quality** | `npm test` → **128 unit tests**; `npm run ci` (lint + types + coverage) | ✅ |

| **Reproducible classification** | `python3 scripts/bench.py` → real precision/recall of the offline baseline over 233 labeled scenes | ✅ |

**Mandatory constraints — all met:** 100% on-device inference through `@qvac/sdk` (**zero cloud APIs** — the hard disqualifier; every interface declared in [`docs/REMOTE_APIS.md`](docs/REMOTE_APIS.md)), [MIT](LICENSE) licensed and fully public, BYOH consumer hardware (Pi 4/5), reproducible via [`scripts/setup-pi.sh`](scripts/setup-pi.sh).

**Evidence bundle:** remote-API declaration ([`docs/REMOTE_APIS.md`](docs/REMOTE_APIS.md) — zero cloud APIs) · structured inference audit log (`GET /api/audit` — model loads/unloads, TTFT, tokens/sec) · offline scan + live network probe (`verify_offline.py`) · real classification baseline over 233 labeled scenes (`bench.py`) · append-only event log (`data/events.jsonl`) · readiness gate (`check_submission_readiness.py`) · physical demo runbook ([`DEMO.md`](DEMO.md)). *Model latency, peak RAM, and mWh/event are captured **on-device** (see [Benchmarks](#-benchmarks)) — not simulated.*

**Radical honesty:** nothing fake is shown as real. The RAM gauge is measured process RSS (tagged `RSS`); battery/solar are *modelled* and clearly tagged **`SIM`** in the UI until an INA219 sensor is wired (`SCARECROW_BATTERY_PCT`/`SCARECROW_SOLAR_W`). Camera capture uses real `libcamera-still` on the Pi. See [Honest Limitations](#-honest-limitations).

---



  

  
Scarecrow 🔌

  $50 off-grid AI sentry on a Raspberry Pi ≤4GB. Camera → multimodal scene understanding → natural-language rule matching → spoken TTS alerts. Solar/battery powered, fully offline.

  

  


  [![Built for QVAC Hackathon](https://img.shields.io/badge/DoraHacks-QVAC%20Edge%20AI-8b5cf6?style=for-the-badge)](https://dorahacks.io/hackathon/qvac-unleach-edge-ai-i/detail)

  [![Track](https://img.shields.io/badge/Track-Tinkerer%20(Pi%20≤4GB)-f59e0b?style=for-the-badge)](https://dorahacks.io/hackathon/qvac-unleach-edge-ai-i/tracks#tinkerer)

  


  ![Node.js](https://img.shields.io/badge/Node.js-339933?style=flat&logo=node.js&logoColor=white)

  ![TypeScript](https://img.shields.io/badge/TypeScript-3178C6?style=flat&logo=typescript&logoColor=white)

  ![Raspberry Pi](https://img.shields.io/badge/Raspberry_Pi-A22846?style=flat&logo=raspberrypi&logoColor=white)

  ![QVAC](https://img.shields.io/badge/@qvac/sdk-06b6d4?style=flat)

  [![CI](https://github.com/edycutjong/scarecrow/actions/workflows/ci.yml/badge.svg)](https://github.com/edycutjong/scarecrow/actions/workflows/ci.yml)



---

## 👀 See It Work

📺 **[Watch the full demo on YouTube](https://youtu.be/fB8f_LXUss4)**

**What the camera sees** *(test clip)*



A person in dark clothing approaching the shed at dusk.

**What Scarecrow does** *(real dashboard)*



🔴 Reasons about the scene → **THREAT DETECTED** → speaks the alert → logs it.



A dog trotting across the same yard.



✅ Sees the dog (green box, `0.88`) → matches the *ignore* rule → **PERIMETER CLEAR**, no alert.

**Known-entity recall** *(on-device RAG)*



An **unfamiliar** dark van…



…vs. **your** silver truck (taught once).



The van trips a 🔴 **ALERT**; the silver truck is **recognised** and the alert is **suppressed** — *"🧩 Recognised known entity: My silver truck."* Teach it once; a stranger's vehicle still alerts.

> **This is reasoning, not motion detection.** Same yard, same camera — Scarecrow

> alerts on the person and stays silent for the dog because it *understands* the

> scene against your plain-English rules. The scene clips are stimulus (yours /

> licensed stock / AI-generated); the dashboard, spoken alert, and RAM gauge are

> the **real** system on-device. Open it yourself with `npm start`.

---

## 💡 The Problem & Solution

Traditional security cameras need WiFi, cloud subscriptions, and constant power. In farms, construction sites, and remote properties, none of these are available.

**Scarecrow** turns a $50 Raspberry Pi into an intelligent sentry that understands what it sees:

**Key Features:**

- 📸 **Vision AI** — QVAC-Vision-1B analyzes camera frames locally

- 🔒 **NL Rules** — Plain English rules: "alert if person near shed, ignore dogs"

- 🧩 **Known-Entity Recall** — on-device RAG remembers your car/pet/mail carrier and suppresses false alarms

- 🔊 **Spoken Alerts** — Piper TTS announces threats through a speaker

- 📲 **P2P Phone Push** — alerts a paired phone over the LAN, no cloud

- ☀️ **Solar Powered** — PIR-triggered wake with configurable sleep intervals

- 🖥️ **Pi Dashboard** — live web UI + rule editor served from Pi's hotspot

## 🏗️ Architecture & Tech Stack

```mermaid

flowchart TD

    A["⚡ PIR Wake"] --> B["📸 Camera"]

    B --> C["🧠 QVAC-Vision-1B"]

    C --> D["📝 Scene Description"]

    D --> E["🔒 Llama 3.2 1B Rules"]

    E --> F{"Alert?"}

    F -- Yes --> K{"🧩 Known entity? (RAG)"}

    F -- No --> H["💤 Sleep"]

    K -- Known --> H

    K -- Unknown --> G["🔊 Piper TTS + 📲 P2P Push"]

    G --> I["📊 Event Log + Dashboard"]

    I --> H

    H --> A

```

| Layer | Technology |

|---|---|

| **Runtime** | Node.js on Raspberry Pi 4/5 |

| **AI Engine** | @qvac/sdk (completion, multimodal, RAG, TTS, P2P) |

| **Vision** | QVAC-Vision-1B (multimodal) |

| **Rules** | Llama 3.2 1B (NL evaluation) |

| **Known entities** | GTE-Large embeddings (on-device RAG) |

| **TTS** | Piper via @qvac/sdk |

| **Alerts** | Holepunch-backed P2P phone push |

| **Dashboard** | Vanilla HTML/JS (Pi hotspot) |

## 💰 Hardware BOM (~$50)

| Part | Cost |

|---|---|

| Raspberry Pi 4 (2GB/4GB) | ~$35 |

| Pi Camera Module v2 | ~$10 |

| USB Speaker | ~$5 |

| PIR Motion Sensor (HC-SR501) | ~$2 |

| Solar panel + battery (optional) | ~$15 |

### 🔌 Wiring

```

        ☀ Solar ──▶ [ Solar + UPS HAT ] ◀──▶ 🔋 18650 Li-ion

                           │ 5V / 3A (USB-C)

                           ▼

   📷 Camera ──CSI ribbon──▶ ┌────────────────┐ ◀── USB ── 🔊 Speaker

                            │  Raspberry Pi  │            (spoken alerts)

   👁 HC-SR501 PIR ──GPIO17──▶│    4 (≤4GB)    │

                            └────────────────┘

        (VCC→5V · OUT→GPIO17 · GND→GND — see table)

```

| Wire | Pi connection | Phys. pin | BCM |

|---|---|---|---|

| PIR `VCC` | 5V | 2 | — |

| PIR `OUT` | GPIO17 | 11 | **17** *(matches `gpio.ts`)* |

| PIR `GND` | GND | 6 | — |

| Camera | CSI camera port | ribbon | — |

| Speaker | USB-A | — | — |

| Power in | USB-C 5V/3A | — | — |

> The PIR is the **power-aware wake source** — the Pi sleeps until motion, then

> wakes → captures → infers → sleeps. Change the pin via

> `startPIRSentry({ pin })`. Solar/battery is optional; any 5V/3A USB-C supply

> works for a wall-powered build.

## 🏆 Why ONLY QVAC?

| QVAC SDK Method | Scarecrow Usage | Cloud Alternative You'd Need |

|---|---|---|

| `loadModel(QVAC_VISION_1B)` | Multimodal scene analysis from camera frames | Google Cloud Vision API |

| `completion()` + `images` | Describe what the camera sees in natural language | GPT-4 Vision |

| `completion()` (Llama 1B) | NL rule evaluation: "alert if person near shed" | OpenAI API |

| `ragIngest()` / `ragSearch()` | Recognize known-friendly entities, suppress false alarms | Pinecone + embeddings API |

| `textToSpeech()` (Piper) | Spoken alerts when rules match | Amazon Polly |

| `startQVACProvider()` (P2P) | Push alerts to a paired phone over the LAN | Firebase Cloud Messaging |

| `unloadModel()` | Critical — free RAM between model swaps on 4GB Pi | N/A |

**Take QVAC out and you'd need 5+ cloud services** (Google Vision + OpenAI + Pinecone + Amazon Polly + Firebase), a WiFi router, and a power outlet — destroying the entire $50 off-grid premise.

## 🚀 Getting Started

```bash

git clone https://github.com/edycutjong/scarecrow.git

cd scarecrow

npm install

python3 scripts/seed.py

# Start the sentry loop AND serve the live dashboard over the Pi hotspot

npm start                 # → http://192.168.4.1:8080 (PORT env to override)

# Sentry loop only, no dashboard (headless)

npm run sentry

```

The dashboard polls a tiny zero-dependency HTTP API (`node:http`) for live data:

| Endpoint | Returns |

|---|---|

| `GET /` | Live dashboard (camera HUD, pipeline, rules, events) |

| `GET /api/status` | Stage, RAM, battery, solar, frames, armed state |

| `GET /api/events` | Event log from the sentry loop |

| `GET /api/rules` · `POST` · `DELETE /api/rules/:id` | List / add / remove natural-language rules |

| `GET /api/entities` · `POST` · `DELETE /api/entities/:id` | List / teach / forget known-friendly entities |

| `GET /api/audit` | Structured inference audit log + summary (model loads/unloads, TTFT, tokens/sec) |

| `GET /api/health` | Liveness probe |

> Opened directly as a file (`open src/web/index.html`) the dashboard runs in a

> self-contained **simulation**; pointed at the API it switches to **LIVE** mode

> automatically — and the rule/entity editors become live. On an alert, a P2P

> phone push is sent via `@qvac/sdk` (`src/core/p2p.ts`).

### 🧩 Known-entity allow-list (on-device RAG)

Teach Scarecrow what to ignore — *"my silver truck," "the mail carrier," "my dog

Max."* Before raising an alert, `src/core/memory.ts` runs an on-device semantic

search (`ragIngest`/`ragSearch`, GTE-Large embeddings) against the scene; a match

above threshold **suppresses the alert**. This is reasoning, not a blocklist: the

owner's car pulling in won't trip it, but a stranger's will.

### 🔌 Power modes

| Command | Behaviour |

|---|---|

| `npm start` | Dashboard + fixed-interval sentry loop |

| `npm run sentry` | Headless fixed-interval loop |

| PIR mode (`startPIRSentry`, `src/core/gpio.ts`) | Sleep → **motion-triggered wake** → check → sleep (solar-optimal) |

Events are persisted append-only to `data/events.jsonl` (`src/core/storage.ts`) and

restored on boot — a zero-dependency, inspectable offline evidence trail.

**Honest telemetry.** `/api/status` reports **real measured process RSS** for the

RAM gauge (`process.memoryUsage().rss`, tagged `RSS` in the UI). Battery/solar are

**modelled** and tagged **`SIM`** on the dashboard until a real sensor is wired —

export `SCARECROW_BATTERY_PCT` / `SCARECROW_SOLAR_W` (e.g. from an INA219 daemon)

and the values go live with `powerSimulated: false`.

### 🍓 Provisioning a Pi

```bash

bash scripts/setup-pi.sh        # camera, audio, Node, deps

bash scripts/setup-pi.sh --ap   # also configure the Scarecrow-AP hotspot

```

## 📊 Benchmarks

**Classification quality (real, reproducible anywhere — `python3 scripts/bench.py`).**

Precision/recall of the documented *offline keyword-fallback baseline* over the 233

labeled fixture scenes — the floor the Llama 3.2 1B engine improves on:

| Baseline metric | Value |

|---|---|

| Precision | 0.67 |

| Recall | 0.43 |

| F1 | 0.52 |

| Accuracy | 0.73 (169/233) |

> The modest recall is the point: a keyword baseline misses non-"person" alerts

> (vehicles, etc.), which is exactly why on-device LLM reasoning is used. The

> Llama model's real precision/recall is measured **on-device** (needs the model).

**Latency · RAM · power — measured on the Pi, not simulated.** These depend on the

QVAC models and physical rig, so they are captured on-device (the dashboard

`/api/status` reports **live RSS**; `htop`/`vcgencmd` for memory; INA219 for

mWh/event). Targets for Raspberry Pi 4 (4GB):

| Stage | Target |

|---|---|

| Vision analysis (1 frame) | < 4,000 ms |

| Rule evaluation | < 2,000 ms |

| TTS alert | < 2,000 ms |

| Full pipeline | < 8,000 ms |

| Peak RAM | < 3,800 MB (≤4GB constraint) |

> These are budgets, not measurements — fill them from a real Pi run for the

> submission. The benchmark script never fabricates device numbers.

## 🧪 Testing & CI

**128 unit tests** (vision, rules + live editing + real fallback classification over the 233-scene fixture set, power state machine, telemetry, inference audit log, RAG known-entity recognition, P2P phone alerts, PIR wake, JSONL persistence, dashboard API) + offline verification scan + readiness suite. Run `npm test`. *(Tests mock only the `@qvac/sdk`/hardware boundary; every assertion exercises real project logic — no tautological label-fed tests.)*

**4-stage pipeline:** Quality → Security → Offline Verify → Deploy

```bash

python3 scripts/verify_offline.py

python3 scripts/bench.py

python3 scripts/check_submission_readiness.py

```

| Layer | Tool | Status |

|---|---|---|

| Code Quality | ESLint + TypeScript | ✅ |

| Security (SAST) | CodeQL | ✅ |

| Dependency Audit (SCA) | npm audit + Dependabot | ✅ |

| Secret Scanning | TruffleHog | ✅ |

| Offline Verification | verify_offline.py (cloud-import/URL scan + net probe) | ✅ |

## 📁 Project Structure

```

scarecrow/

├── docs/               # README assets

├── data/fixtures/      # test_rules.json, test_scenes.json

├── scripts/            # setup-pi, seed, bench, verify, readiness

├── src/

│   ├── core/

│   │   ├── qvac.ts     # @qvac/sdk wrapper

│   │   ├── vision.ts   # Camera + multimodal analysis

│   │   ├── rules.ts    # NL rule engine + add/remove

│   │   ├── memory.ts   # RAG known-entity allow-list (ragSearch)

│   │   ├── power.ts    # Sentry loop + live telemetry (getSystemStatus)

│   │   ├── gpio.ts     # PIR motion-wake source

│   │   ├── storage.ts  # Append-only JSONL event persistence

│   │   ├── p2p.ts      # P2P phone push alerts on match

│   │   └── audit.ts    # Structured inference audit log (loads/unloads, TTFT, tok/s)

│   └── web/

│       ├── server.ts   # Dashboard HTTP server + JSON API (node:http)

│       └── index.html  # Live dashboard + rule/entity editor (Pi hotspot)

├── .github/            # CI/CD + CodeQL + Dependabot

└── README.md

```

## ⚠️ Honest Limitations

1. Camera capture simulated in dev (real Pi Camera required)

2. PIR wake uses the `onoff` GPIO binding on the Pi; dev/CI run a simulated motion source

3. Battery/solar are modelled (tagged `SIM` in the UI) until an INA219 sensor feeds `SCARECROW_BATTERY_PCT`/`SCARECROW_SOLAR_W`; the RAM gauge is real measured RSS

4. Event log persists to append-only JSONL; a queryable SQLite store is future work

5. Single-model-at-a-time due to 4GB constraint

6. English rules only

## 📄 License

[MIT](LICENSE) © 2026 Edy Cu

## 🙏 Acknowledgments

Built for **QVAC Hackathon I — Unleash Edge AI** (DoraHacks). Proving that useful AI doesn't need the cloud — just a $50 Pi and QVAC.
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/edycutjong/scarecrow

Awesome Lists containing this project

README

Scarecrow 🔌
