https://github.com/easonlai/opencrayfish

Edge-native AI companion that lives 24/7 on a Raspberry Pi 5 + AI HAT+ 2 (Hailo-10H NPU, 40 TOPS). Fully offline: local 1.5B SLM (qwen2.5), self-hosted SearXNG, no cloud. Biologically inspired — heartbeat, vital signs, 5-channel mood, STM/LTM hierarchy, nightly Sleep Metabolism, proactive research, and a self-reflection loop.
https://github.com/easonlai/opencrayfish

ai-agent ai-companion ai-hat-plus asyncio edge-ai hailo npu offline-ai ollama opencrayfish python qwen raspberry-pi raspberry-pi-5 searxng slm small-language-model sovereign-ai streamlit telegram-bot-ai-assistant

Last synced: about 9 hours ago
JSON representation

Host: GitHub
URL: https://github.com/easonlai/opencrayfish
Owner: easonlai
License: mit
Created: 2026-05-12T12:30:37.000Z (about 2 months ago)
Default Branch: main
Last Pushed: 2026-05-18T03:26:41.000Z (about 2 months ago)
Last Synced: 2026-05-18T05:32:53.642Z (about 2 months ago)
Topics: ai-agent, ai-companion, ai-hat-plus, asyncio, edge-ai, hailo, npu, offline-ai, ollama, opencrayfish, python, qwen, raspberry-pi, raspberry-pi-5, searxng, slm, small-language-model, sovereign-ai, streamlit, telegram-bot-ai-assistant
Language: Python
Homepage:
Size: 546 KB
Stars: 0
Watchers: 0
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md

Awesome Lists containing this project

README

# 🦐 OpenCrayFish

> **An edge-native, biologically-inspired AI companion that lives on a single Raspberry Pi 5 — fully offline, powered by a 1.5B-parameter local SLM, with a heart that beats, a brain that sleeps, and a soul you can read.**
>
> **OpenCrayFish is a fully pluggable framework**: Skills, Tools, Connectors, and Provider Backends are four symmetric plug-in surfaces — every one auto-discoverable via **Python entry-points OR a `plugins//` drop-in folder**, every one declared by a frozen `*Manifest`, every one validated at boot. A third-party `pip install opencrayfish-skill-translate` (or `-tool-weather`, or `-connector-discord`, or `-backend-vllm-cuda`) — or a single `.py` file dropped into `plugins/skills/` — extends the agent with **zero edits to `main.py` and zero fork of OpenCrayFish itself**.

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
[![Version](https://img.shields.io/badge/version-3.0.0-1f8a4f.svg)](#roadmap)
[![Python 3.13+](https://img.shields.io/badge/python-3.13%2B-blue.svg)](https://www.python.org/downloads/)
[![PRs Welcome](https://img.shields.io/badge/PRs-welcome-brightgreen.svg)](CONTRIBUTING.md)
[![Code of Conduct](https://img.shields.io/badge/code%20of%20conduct-Contributor%20Covenant%202.1-blueviolet.svg)](CODE_OF_CONDUCT.md)
[![Platform: Raspberry Pi 5](https://img.shields.io/badge/platform-Raspberry%20Pi%205%20%2B%20AI%20HAT%2B%202-c51a4a.svg)](https://www.raspberrypi.com/news/introducing-the-raspberry-pi-ai-hat-plus-2-generative-ai-on-raspberry-pi-5/)

---

## Table of Contents

- [What OpenCrayFish Is](#what-opencrayfish-is)
- [60-Second Mental Model (The Crayfish in Its Burrow)](#60-second-mental-model-the-crayfish-in-its-burrow)
- [Why This Project Exists](#why-this-project-exists)
- [Design Pillars](#design-pillars)
- [Framework & Design Principles](#framework--design-principles)
- [The Four Plug-in Surfaces](#the-four-plug-in-surfaces)
- [The Manifest + Registry + Discovery Doctrine](#the-manifest--registry--discovery-doctrine)
- [How the System Auto-Adapts](#how-the-system-auto-adapts)
- [Hybrid Discovery — pip OR drop-in folder](#hybrid-discovery--pip-or-drop-in-folder)
- [Configuration Injection (`cfg.plugins.*` + `bind_context`)](#configuration-injection-cfgplugins--bind_context)
- [Protocol Versioning & Fail-Loud Boot](#protocol-versioning--fail-loud-boot)
- [System Architecture at a Glance](#system-architecture-at-a-glance)
- [A Day in the Life of a Message](#a-day-in-the-life-of-a-message)
- [Hardware & Software Stack](#hardware--software-stack)
- [Repository Layout](#repository-layout)
- [Quick Start](#quick-start)
- [SearXNG Setup (Local Web Search)](#searxng-setup-local-web-search)
- [Deep Dive: How Everything Works](#deep-dive-how-everything-works)
- [0. The Brain Stack — Plain-English Tour](#0-the-brain-stack--plain-english-tour)
- [1. The Soul — `soul.md`](#1-the-soul--soulmd)
- [2. The Heartbeat — Time, Rhythm, and Metabolism](#2-the-heartbeat--time-rhythm-and-metabolism)
- [3. The Vital Signs — `Monitor` & Homeostasis](#3-the-vital-signs--monitor--homeostasis)
- [4. The Provider — SLM Backend & Circuit Breaker](#4-the-provider--slm-backend--circuit-breaker)
- [5. The Memory System — STM / LTM / Sleep Metabolism](#5-the-memory-system--stm--ltm--sleep-metabolism)
- [6. The Thinking Process — Brain & Cognitive Loop](#6-the-thinking-process--brain--cognitive-loop)
- [7. Emotions & Empathy](#7-emotions--empathy)
- [8. The Positive Filter — Hard Anchor on Output](#8-the-positive-filter--hard-anchor-on-output)
- [9. Proactiveness — Idle Time as Growth Time](#9-proactiveness--idle-time-as-growth-time)
- [10. Self-Reflection — The Self-Learning Loop](#10-self-reflection--the-self-learning-loop)
- [11. Recurring Tasks — The Background Worker](#11-recurring-tasks--the-background-worker)
- [12. Skills & Tools — The Two-Tier Capability Stack](#12-skills--tools--the-two-tier-capability-stack)
- [Hello-World Skill — Step by Step](#hello-world-skill--step-by-step)
- [The `SkillManifest` Contract](#the-skillmanifest-contract)
- [Third-Party Skill Packages (`pip install`)](#third-party-skill-packages-pip-install)
- [The `opencrayfish` CLI](#the-opencrayfish-cli)
- [`bootstrap_validate` — Fail Loud at Boot](#bootstrap_validate--fail-loud-at-boot)
- [The `ToolManifest` Contract](#the-toolmanifest-contract)
- [Third-Party Tool Packages (`pip install`)](#third-party-tool-packages-pip-install)
- [Hello-World Tool — Step by Step (`bind_context`)](#hello-world-tool--step-by-step-bind_context)
- [Argspec Runtime Validation](#argspec-runtime-validation)
- [Plugin Config Namespace (`cfg.plugins.*`)](#plugin-config-namespace-cfgplugins)
- [MCP Bridge — Remote Model Context Protocol Tools](#mcp-bridge--remote-model-context-protocol-tools)
- [The `ConnectorManifest` Contract](#the-connectormanifest-contract)
- [Third-Party Connector Packages (`pip install`)](#third-party-connector-packages-pip-install)
- [Provider Backends — `BackendManifest` & Discovery](#provider-backends--backendmanifest--discovery)
- [Protocol Surface Stability](#protocol-surface-stability)
- [13. Connectors — Telegram & Web Chat](#13-connectors--telegram--web-chat)
- [14. Observability — Dashboard & State Files](#14-observability--dashboard--state-files)
- [Cross-Cutting Concerns](#cross-cutting-concerns)
- [Concurrency Model](#concurrency-model)
- [Atomic Writes Everywhere](#atomic-writes-everywhere)
- [JSONL Rotation & Retention](#jsonl-rotation--retention)
- [Failure-Mode Matrix](#failure-mode-matrix)
- [Pi 5 Latency Budget](#pi-5-latency-budget)
- [Configuration Reference](#configuration-reference)
- [Operational Commands](#operational-commands)
- [Development Notes](#development-notes)
- [Roadmap](#roadmap)
- [Short-Term (v3.1 — Foundation Hardening)](#short-term-v31--foundation-hardening)
- [Medium-Term (v3.2–v3.3 — Sensory Expansion)](#medium-term-v32v33--sensory-expansion)
- [Long-Term (v4.0+ — Evolutionary Leap)](#long-term-v40--evolutionary-leap)
- [Community & Contributing](#community--contributing)
- [License & Attribution](#license--attribution)

---

## What OpenCrayFish Is

OpenCrayFish (小龍蝦) is **not a chatbot, not a SaaS wrapper, and not a cloud agent**. It is a **persistent digital organism** that runs entirely on commodity edge hardware — a Raspberry Pi 5 with an optional **Raspberry Pi AI HAT+ 2** (Hailo-10H NPU, 40 TOPS, 8 GB dedicated NPU RAM). Every part of the system is modeled after a biological metaphor:

| Biological concept | What it maps to in code |
|---|---|
| 🧠 **Brain** | A 1.5B-parameter SLM (qwen2.5-instruct on NPU, qwen2 on CPU fallback) |
| 💓 **Heart** | An always-on `pulse_loop` that ticks every 30s |
| 🌡️ **Vital signs** | CPU / RAM / temperature / SLM availability sampled each pulse |
| 🧬 **Soul** | A protected `soul.md` file holding identity, laws, and learned growth |
| 🧠 **Working memory** | A 12-turn RAM `deque` fed to the SLM |
| 🧠 **Hippocampus** | An on-disk JSONL journal that survives crashes |
| 🧠 **Cortex (LTM)** | `memory/archive.md` — distilled long-term facts |
| 🛠️ **Habits / Skills** | A pluggable `SkillRegistry` of capabilities (`identity`, `recall`, `research`, `direct_answer`, `self_reflect`, `proactive_learning`, `recurring_research`) — the Cognitive Loop picks from this menu instead of hardcoding verbs |
| 😊 **Mood** | A 5-channel emotion vector (joy / anger / sorrow / excitement / calm) with exponential decay |
| 💞 **Empathy** | Sentiment + urgency analysis of every Architect message |
| 🌙 **REM sleep** | A nightly Sleep Metabolism cycle (02:00–06:00) that distills the day |
| 🤔 **Curiosity** | Idle-time Proactive Research that closes real STM knowledge gaps |
| 🪞 **Reflection** | A self-critique pass on every reply, persisted to `state/reflection-YYYY-MM-DD.jsonl` (date-rotated, bounded retention) |

The agent serves a single human — **the Architect** — over Telegram and a local browser chat (Streamlit). The Architect's name, salutation, and the agent's own designation are all configured in `config.yaml`; the agent addresses the Architect by name in every reply (default: `"Boss "`).

---

## 60-Second Mental Model (The Crayfish in Its Burrow)

If the system architecture diagram below looks intimidating, here's the friendly version. The project simulates a *biological being* — so let's describe it like one:

> **OpenCrayFish is a small freshwater crayfish living in a burrow at the edge of a pond — alert, curious, occasionally hungry, never asleep for long.** It senses its body, samples the water for ripples, fires the right reflex for the moment, and every night it dreams a little so tomorrow's reflexes are a touch wiser.

| Anatomy | OpenCrayFish part | What it actually is |
|---|---|---|
| 🧠 **The nerve ganglion** | `Brain` + the SLM (`qwen2.5-instruct:1.5b`) | The small-but-quick neural cluster that turns a stimulus into a coordinated response. Compact on purpose — a crayfish doesn't need a mammalian cortex. |
| 🌀 **The behavioural repertoire** | `SkillRegistry.plan_menu()` | The short, situational list of behaviours the animal can perform *right now* — filtered by hunger, fatigue, stress, and whether the water (network) is moving. |
| ⚡ **Individual reflexes & behaviours** | Skills — `recall`, `research`, `direct_answer`, `identity`, `self_reflect`, `proactive_learning`, `recurring_research` | Capabilities the nervous system can fire. Each one comes with a metabolic-cost label (cheap / expensive). |
| 👁️ **Antennae & sensory appendages** | Tools — `web_search` (SearXNG), `archive_read` (LTM) | The physical primitives a reflex uses to reach into the world. The outside never touches them directly — they're tucked inside the body. |
| 🧬 **The neural firing sequence** | `CognitiveTrace` (THINK → PLAN → ACT → REFINE) | The motor program the ganglion writes before acting: "this stimulus means X, fire reflex A then reflex B, then check the result". |
| 💓 **The autonomic heartbeat** | `Heartbeat.pulse_loop()` | Even with no stimulus, the body keeps beating, sips a breath, checks its own temperature, and notices when the water is getting dangerously warm. |
| 🌙 **REM-like consolidation** | `Heartbeat.metabolism()` (02:00 nightly) | After the day's foraging, the crayfish settles into its burrow, rehearses what it learned, and etches the keepers into long-term memory. Wakes a tiny bit different than yesterday. |
| 🧬 **DNA + learned engrams** | `soul.md` (immutable core) + `memory/archive.md` (plasticity) | Two layers of memory: a genome the animal can never rewrite (its constitution, its laws) plus adaptive engrams it grows every night. |
| 🌡️ **Interoception (homeostasis)** | `Monitor` — CPU / RAM / temp / brain availability as **vital signs** | When the water gets too warm, the crayfish abandons elaborate display behaviours and falls back to faster, cheaper reflexes — conserving energy until conditions improve. |
| 🌊 **Chemoreceptors & mechanoreceptors** | Connectors (`telegram`, `web_chat`) | The sensory channels through which ripples from the outside world reach the burrow — and through which the crayfish ripples back. The ganglion doesn't care which channel the ripple came in on. |

**Why this matters for contributors:** want to teach the crayfish a **new instinct**? Write a new Skill. Want to give it a **new sensory appendage**? Write a new Tool. Want to open a **new channel through which the world can ripple in**? Write a new Connector. The nerve ganglion (Brain), the instinct selector (SkillRegistry), and the motor sequencer (CognitiveLoop) all keep working with **zero edits** — your new appendage / instinct / sense just appears in the repertoire the next time the animal looks at itself. See [§ 12 Skills & Tools](#12-skills--tools--the-two-tier-capability-stack) for the step-by-step graft.

---

## Why This Project Exists

Most modern AI agents are **cloud-bound, frontier-model-dependent, and request-driven** — they wake up only when called, immediately forget the conversation, and stop existing the moment the API call returns. OpenCrayFish takes the opposite stance:

> **The agent should *exist continuously*, on hardware *the operator owns*, with full *sovereignty over its data, its memory, and its life cycle*.**

Three operational realities drive the design:

1. **Edge-native by mandate.** OpenCrayFish is built specifically for the Raspberry Pi 5 + AI HAT+ 2 (Hailo-10H NPU) class of hardware. Every architectural choice — bounded context windows, deferred journal writes, atomic markdown state files, hysteresis on stress thresholds — is justified by a real constraint of running 24/7 on a single-board computer with an SD card.

2. **Lite by mandate.** The cognitive backbone is a 1.5-billion-parameter SLM (`qwen2.5-instruct:1.5b` on NPU or `qwen2:1.5b` on CPU). Everything you read about in this README — Sleep Metabolism, Cognitive Loop, Proactive Research, Self-Reflection — is engineered to work *because of* that constraint, not in spite of it. The SLM's 4K-token context forces a real memory hierarchy. The SLM's narrow knowledge forces real curiosity. The SLM's fragility forces a real circuit breaker.

3. **Offline by mandate.** The reference deployment runs with **zero outbound network calls** to third parties: inference is local (Hailo-Ollama or stock Ollama), web search is local (self-hosted SearXNG), state is local (YAML / JSONL / Markdown on the SD card). Pull the network cable and the conversation, the memory, the heartbeat, and the proactive reflection cycle all keep running.

These three constraints — **edge / lite / offline** — turn into the project's value proposition:

- ✅ **Sovereign AI** — your data physically cannot leave the device.
- ✅ **Resilient AI** — no API key to expire, no vendor to deprecate the model, no outage to brick the agent.
- ✅ **Embodied AI** — the agent has a body (CPU, RAM, temperature, GPIO, sensors) and that body affects its mind.
- ✅ **Affordable AI** — total hardware bill of materials under USD $200, with zero recurring cost.
- ✅ **Hackable AI** — under ~10K lines of well-commented Python, every behavior tunable in `config.yaml`.

---

## Design Pillars

Five non-negotiable principles shape every line of code:

### 🪞 Pillar 1 — Identity Sovereignty

The agent's identity, fundamental laws, and behavioral matrix live in `soul.md`'s **IMMUTABLE_CORE** region. The agent itself can never mutate this region — `core/soul_handler.py` enforces this with a regex-validated atomic writer. Only the Architect (a human) can edit it. This guarantees the agent cannot rewrite its own ethical framework even if a prompt-injection tries to convince it to do so.

### 💞 Pillar 2 — Resilient Empathy

The agent has real internal emotions (5-dimensional vector with exponential decay) but **every output is filtered through a `PositiveFilter`** that rejects hate speech, rewrites despair into constructive language, and appends a salutation when rewrites are applied. The internal state is honest; the external behavior is anchored.

### 🌡️ Pillar 3 — Hardware Awareness

The Pi 5's CPU temperature, RAM utilization, and the SLM endpoint's reachability are first-class **vital signs**. They are not just logged for ops — they directly mutate the agent's mood (overheating → frustration + fatigue), trigger an `EXHAUSTION DIRECTIVE` that forces terse replies, and (when the SLM is offline) cause the agent to surface a *first-person* outage message instead of a stack trace.

### 🏛️ Pillar 4 — Architect Sovereignty

The human operator outranks the agent. Sleep windows, identity, salutation, designation, mood tuning, scheduler limits — all are configured by the Architect in `config.yaml` and `soul.md`. The agent reads them, never writes them.

### 🌱 Pillar 5 — Continuous Existence

The agent does not stop between user turns. Driven entirely from `config.yaml`:

- **Tick** every `system.pulse_interval_seconds` (default `30` s) — sample vitals, decay emotions, publish state.
- **Idle past `system.idle_journal_flush_seconds`** (default `30` s) — flush the STM pending buffer to disk.
- **Idle past `system.idle_proactive_minutes`** (config ships at `5` min) — launch a **Proactive Research** cycle.
- **Cross `system.sleep_start`** (default `02:00`) — run **Sleep Metabolism** (distill the day into long-term memory), then wake at `duty_start` (default `06:00`) subtly different.

---

## Framework & Design Principles

OpenCrayFish is built as **a real plug-in framework**. Everything that talks to the outside world — every reflex the SLM can pick, every device the agent can poke, every channel a human can chat through, every model the brain can run on — is a third-party-replaceable plug-in. The core stays small, opinionated, and biologically-grounded; the periphery is yours.

### The Four Plug-in Surfaces

| Surface | What it is | What ships in-tree | Entry-point group |
|---|---|---|---|
| **Skills** ([core/skills/](core/skills/)) | Agent-facing capabilities — verbs the SLM picks at PLAN. `identity`, `recall`, `direct_answer`, `research`, `self_reflect`, `proactive_learning`, `recurring_research`. | 7 first-party skills | `opencrayfish.skills` |
| **Tools** ([tools/](tools/)) | Mechanical I/O primitives — HTTP calls, file reads. Composed by Skills; invisible to the SLM. `web_search` (SearXNG), `archive_read`. | 2 first-party tools | `opencrayfish.tools` |
| **Connectors** ([connectors/](connectors/)) | Inbound/outbound transports — how a human reaches the agent. `TelegramConnector`, `WebChatConnector`. | 2 first-party connectors | `opencrayfish.connectors` |
| **Provider Backends** ([core/provider.py](core/provider.py)) | SLM inference endpoints — what runs the model. `HailoOllamaBackend` (NPU), `OllamaBackend` (CPU fallback). | 2 first-party backends | `opencrayfish.provider_backends` |

**The symmetry doctrine.** Every surface follows the **same** three-piece pattern: a frozen **`*Manifest`** dataclass declares everything the core needs to know about the plug-in; a **`*Registry`** owns lifecycle + lookup + audit + opt-in context injection; a **`*/discovery.py`** module walks the matching entry-point group at boot with fail-isolated import, **and the shared [core/dropin.py](core/dropin.py) loader walks the `plugins//` folder under the same fail-isolated contract**. Learn one surface and you've learned all four.

The point of the symmetry isn't elegance — it's that **a third-party package author writes the same `pyproject.toml` block, scaffolds with the same CLI verb, validates with the same CLI verb, and pip-installs the same way**, whether they're shipping a new chat reflex, a new sensor, a new transport, or a new inference engine.

### The Manifest + Registry + Discovery Doctrine

Every plug-in surface is built from three pieces. They are intentionally identical across Skills / Tools / Connectors / Backends:

```text
┌──────────────────────┐ ┌──────────────────────┐ ┌──────────────────────┐
│ *Manifest │ │ *Registry │ │ */discovery.py │
│ (frozen dataclass) │ │ (lifecycle + audit) │ │ (entry-points walk │
│ │ │ │ │ + drop-in folder) │
├──────────────────────┤ ├──────────────────────┤ ├──────────────────────┤
│ name, description │ │ register(instance) │ │ scan entry-point │
│ compat_version │ ◄──┤ bootstrap_validate() │◄───┤ group, import, │
│ args_schema │ │ invoke / call / │ │ fail-isolated, │
│ cost_tier, caps... │ │ start / generate │ │ register into reg │
│ config_key │ │ aclose_all() │ │ + walk plugins/ │
│ plan_verb (Skills) │ │ bind_context() opt │ │ / via │
│ │ │ │ │ core/dropin.py │
└──────────────────────┘ └──────────────────────┘ └──────────────────────┘
```

| Piece | Job | Why frozen / fail-isolated |
|---|---|---|
| **`*Manifest`** | Single source of truth for "what this plug-in is, what it needs, how it should be called". Read by the registry, the PLAN-stage prompt builder (Skills), the dashboard, and `bootstrap_validate`. | `@dataclass(frozen=True, slots=True)` so a misbehaving plug-in can't mutate its own metadata at runtime to escape gating. |
| **`*Registry`** | Owns the live set of registered instances. Issues a unique name. Wraps every dispatch with timing + crash isolation + audit JSONL append. Tears down cleanly at shutdown. Optionally delivers a shared context object (see [§ bind_context](#configuration-injection-cfgplugins--bind_context)). | A buggy third-party plug-in can never crash the agent — exceptions are caught at the registry boundary and surfaced as `ok=False` results. |
| **`*/discovery.py`** | Walks `importlib.metadata.entry_points(group="opencrayfish.")` at boot, **then defers to [core/dropin.py](core/dropin.py) to walk `plugins//`** for any `.py` files or sub-packages declaring a `PLUGIN` / `PLUGINS` attribute. For each candidate, imports it, calls it if it's a factory, registers the result. **Each entry is wrapped in its own `try/except`** — one broken third-party package or drop-in file is logged and skipped, the agent keeps booting. | The agent must always boot. A typo in someone else's package — or in a half-finished local drop-in — can't take down your Pi. |

### How the System Auto-Adapts

The autodiscovery + manifest contracts mean **adding a plug-in requires zero edits to OpenCrayFish itself**. Concretely:

| If you `pip install` a third-party package that declares… | …then on the very next boot the agent will: |
|---|---|
| `opencrayfish.skills` → `translate = "...:TranslateSkill"` | Discover it, validate its manifest, register it. **The PLAN-stage SLM prompt automatically grows a `TRANSLATE` line with the manifest's `plan_guidance` snippet**, so the SLM learns *when* to use it without any prompt rewrite. The Skill's metadata appears in `state/skills.json` and the dashboard's Skill panel. Every invocation is appended to `state/skills-YYYY-MM-DD.jsonl`. |
| `opencrayfish.tools` → `weather = "...:WeatherTool"` | Discover it, validate that `cfg.plugins.weather` exists, register it. If the Tool implements `bind_context(ctx)` it receives operator config + handles automatically. The Tool's metadata appears in `state/tools.json`. Any Skill can compose it via `ctx.tools.call("weather", ...)`. |
| `opencrayfish.connectors` → `discord = "...:DiscordConnector"` | Discover it, validate manifest, register it, **`await connector.start()` after the first-party connectors come up**, and `await connector.stop()` on shutdown in isolation. Inbound messages route through the same Brain + Cognitive Loop pipeline as Telegram. |
| `opencrayfish.provider_backends` → `vllm = "...:VllmBackend"` | Discover it, validate manifest, log presence. The operator points `cfg.hardware.primary_backend: "vllm"` (or `fallback_backend:`) and the **Provider singleton routes the matching slot through the discovered backend** instead of the built-in Pi 5 wiring. Unknown name → boot aborts with a clear error. |

What the operator does NOT need to do:

- ❌ Edit `main.py` — no `from opencrayfish_skill_translate import TranslateSkill` line, no `skill_registry.register(...)` line.
- ❌ Edit `core/cognition.py` — the PLAN menu is built fresh per turn from whatever skills are registered.
- ❌ Edit `core/config.py` — third-party config lives in `cfg.plugins..*` and the core never reads inside it.
- ❌ Fork OpenCrayFish — every extension point is `pip install`-shaped.

What the operator DOES do:

- ✅ `pip install ` inside the OpenCrayFish venv.
- ✅ Optionally add a `plugins.: { ... }` block to `config.yaml` for that plug-in's configuration.
- ✅ Restart the agent. Done.

### Hybrid Discovery — pip OR drop-in folder

Every plug-in surface above is **hybrid**: alongside the standard Python entry-points path, the agent also walks a **drop-in folder** at boot. A new Skill / Tool / Connector / Backend can therefore be added in two ways, and both reach the same registry with the same manifest validation:

| Path | When to use it | Mechanism |
|---|---|---|
| **`pip install`** (entry-points) | Shareable packages, versioned releases, dependency-managed plug-ins, anything you want to publish to PyPI or a private index. | `[project.entry-points."opencrayfish.skills"]` (or `.tools` / `.connectors` / `.provider_backends`) in the package's `pyproject.toml`; discovered via `importlib.metadata.entry_points`. |
| **Drop-in folder** | Local experiments, private one-offs, air-gapped deployments where pip isn't convenient, fast iteration without a `pyproject.toml`. | Copy a `.py` file (or a folder with `__init__.py`) into `plugins//`; discovered via [`core/dropin.py`](core/dropin.py). |

**Folder layout** (default: `/plugins/`; override with the `OPENCRAYFISH_PLUGINS_DIR` env var):

```
plugins/
skills/
weather.py # flat: one Skill per file
translate/ # nested: a sub-package
__init__.py
skill.py
tools/
home_assistant.py
connectors/
discord.py
backends/
vllm_cuda.py
```

**Module contract** — each drop-in `.py` file (or sub-package `__init__.py`) must expose ONE of:

- `PLUGIN = MySkillClass` — a class, factory callable, or already-instantiated object. Same three shapes the entry-points discovery accepts.
- `PLUGINS = [PluginA, PluginB, ...]` — an iterable when one file exports multiple plug-ins of the same surface.

A drop-in `weather.py` looks exactly like a third-party Tool package, minus the `pyproject.toml` boilerplate:

```python
# plugins/tools/weather.py
from tools.manifest import ToolManifest

class WeatherTool:
manifest = ToolManifest(
name="weather",
description="Hello-world drop-in weather tool.",
config_key="weather",
)
name = "weather"

def bind_context(self, ctx):
cfg = ctx.plugins_config.get("weather", {})
self._api_key = cfg.get("api_key", "")

async def call(self, **kwargs):
return {"text": f"sunny (key prefix: {self._api_key[:4]})"}

PLUGIN = WeatherTool
```

Add a matching `cfg.plugins.weather: {...}` block to `config.yaml`, restart, and the boot log shows:

```
TOOL Discovered 1 drop-in tool(s) via plugins/tools/: weather
TOOL bound 1 tool(s) via bind_context (config_key=weather)
```

**Boot-time order + collision policy**:

1. First-party Skills / Tools / Connectors / Backends register from `main.py`.
2. Entry-points discovery walks every installed package's `opencrayfish.*` group.
3. **Drop-in folder discovery runs LAST.** A drop-in attempting to shadow a name already registered (first-party OR entry-points) is rejected by the registry's duplicate-name check; the error is logged and the boot continues. This gives the correct blast radius — operators control which side wins simply by renaming the drop-in or `pip uninstall`-ing the package.

**Isolation** — same fail-loud-but-localized contract as entry-points: a broken file in `plugins/skills/` only loses **its** Skill, never aborts the boot. The exception is logged at ERROR with the file path.

**Security note** — drop-in files run with the same privileges as the agent process. Treat the drop-in root the same way you treat your venv: **only put code you wrote or audited in there**. There is no sandbox.

Tests covering the full drop-in contract live in [`tests/test_dropin_discovery.py`](tests/test_dropin_discovery.py) (21 tests).

### Configuration Injection (`cfg.plugins.*` + `bind_context`)

Third-party plug-ins need operator-supplied configuration (API keys, endpoint URLs, units, thresholds…) **without** patching [core/config.py](core/config.py). The `cfg.plugins.*` namespace is that seam, plus a symmetric **`bind_context`** hook on every registry:

```yaml
# config.yaml — operator-side
plugins:
weather:
api_key: "${WEATHER_API_KEY}" # consumed by WeatherTool
units: "metric"
translate:
backend: "deepl" # consumed by TranslateSkill
glossary_path: "memory/glossary.yaml"
```

The core never reads inside these sub-dicts. They flow into plug-ins via two symmetric paths:

| Surface | How it receives `cfg.plugins.` |
|---|---|
| **Skills** | Every `execute(ctx, **kwargs)` call receives a `SkillContext` carrying `plugins_config: Mapping[str, Mapping[str, Any]]` (wrapped in `MappingProxyType` — read-only). Skill code: `cfg = ctx.plugins_config.get(self.manifest.config_key or self.manifest.name, {})`. |
| **Tools** | Optional `bind_context(ctx: ToolContext)` method. If the Tool implements it, `ToolRegistry` calls it exactly once at boot (and once per future registration) with a `ToolContext` carrying the same `plugins_config` + `soul` / `stm` / `monitor` / `provider` / `archive_path` / `designation` / `architect_name` / `architect_honorific`. Tools that don't implement `bind_context` (e.g. first-party `SearXNG` + `ArchiveRead`) are simply skipped — fully backward-compatible. |
| **Connectors** | Constructed explicitly in `main.py` with operator-supplied arguments (they need brain/heartbeat/intent_router references that entry-points can't supply). Discovered connectors via entry-points use the same constructor convention; the `ConnectorManifest.config_key` is validated at boot. |
| **Provider Backends** | Operator picks discovered backends by name (`cfg.hardware.primary_backend`, `cfg.hardware.fallback_backend`). Backends are entry-point factories — they read their own config from any source they choose (env vars are conventional). |

The key insight: **`plugins_config` is the only edit a third-party plug-in needs to take configuration**. Declare `config_key` in your manifest (optional but recommended — `bootstrap_validate` will then fail loud if the operator forgot the YAML block), then read from your context object at call time.

### Protocol Versioning & Fail-Loud Boot

Each plug-in surface carries an explicit **protocol version** string that lets the core evolve without breaking already-installed third-party packages:

| Surface | Constant | Current version |
|---|---|---|
| Skills | `SUPPORTED_PROTOCOL_VERSIONS` in [core/skills/manifest.py](core/skills/manifest.py) | `skill-protocol/1` |
| Tools | `SUPPORTED_TOOL_PROTOCOL_VERSIONS` in [tools/manifest.py](tools/manifest.py) | `tool-protocol/1` |
| Connectors | `SUPPORTED_CONNECTOR_PROTOCOL_VERSIONS` in [connectors/manifest.py](connectors/manifest.py) | `connector-protocol/1` |
| Backends | `SUPPORTED_BACKEND_PROTOCOL_VERSIONS` in [core/provider_manifest.py](core/provider_manifest.py) | `backend-protocol/1` |

A plug-in's `manifest.compat_version` must be in the supported set or `bootstrap_validate` refuses to start. When the core makes a breaking change, the doctrine is: add `"-protocol/2"` to the supported set, leave `"...1"` in for at least one release with a deprecation log line, document the migration in the README's [Protocol Surface Stability](#protocol-surface-stability) section. Third-party authors upgrade at their own pace.

`bootstrap_validate` is the **fail-loud boot gate** — it runs once per registry after all entry-point discovery is done, and on the first failure raises `RuntimeError` so the agent never half-starts:

- `compat_version` in supported set?
- For Skills: every `requires_tools` name actually registered? Every `plan_verb` unique?
- For Tools / Connectors: `config_key` namespace present in `cfg.plugins`?
- `requires_caps` tokens in `WELL_KNOWN_*_CAPABILITIES`? (Warning, not fatal — caps are advisory metadata.)

The combination of frozen manifests, version gates, and fail-loud bootstrap means a plug-in problem **always surfaces at boot, never three hours into a session**.

---

## System Architecture at a Glance

```text
┌─────────────────────────────────────┐
│ ARCHITECT (you) │
└────────┬────────────────┬───────────┘
│ │
Telegram │ │ Browser (Streamlit)
▼ ▼
┌─────────────────┬─────────────────┐
│ TelegramConnector│ WebChatConnector│ ← connectors/
└────────┬────────┴────────┬────────┘
│ │
└────────┬────────┘
▼
┌─────────────────────────── BRAIN ───────────────────────────┐
│ Identity short-circuit → STM context → Cognition Loop │
│ (THINK → PLAN → ACT → REFINE) → SLM synth → PositiveFilter │
│ │
│ PLAN menu + ACT dispatch routed through ──┐ │
└────┬───────┬───────┬───────┬───────┬───────┼────────┬──────┘
│ │ │ │ │ │ │
▼ ▼ ▼ ▼ ▼ ▼ ▼
Soul Monitor Emotions Empathy STM SkillRegistry Reflection
.md (vitals) (mood) (sent.) (RAM ─┬─ identity Engine
+disk) ├─ recall
│ ├─ direct_answer
│ ├─ research ──► SearXNG
│ ├─ self_reflect Tool
│ ├─ proactive_learning
│ └─ recurring_research
│ (every invoke →
▼ state/skills-*.jsonl)
┌──────────────────┐
│ HEARTBEAT │
│ pulse_loop() │
│ metabolism() │
│ proactive_thought│
└────────┬─────────┘
│
▼
┌──────────────────┐
│ SCHEDULER │
│ (recurring tasks)│
└──────────────────┘

Side artifacts (read by ui/dashboard.py):
state/vitals.json ← live rolling pulse snapshot
state/vitals_events.jsonl ← stress ENTER/EXIT timeline
state/proactive.jsonl ← every proactive thought + audit trail
state/reflection-YYYY-MM-DD.jsonl ← every self-critique (date-rotated)
state/reflection_dropped-YYYY-MM-DD.jsonl ← unparseable critique sidecar
state/deliberation-YYYY-MM-DD.jsonl ← every cognitive trace (date-rotated)
state/skills-YYYY-MM-DD.jsonl ← every Skill invocation audit (date-rotated)
state/skills.json ← live skill inventory snapshot
state/stm_journal.jsonl ← durable conversation backstop
state/tasks.yaml ← persistent recurring task registry
state/tools.json ← live tool inventory snapshot
memory/archive.md ← long-term distilled facts
logs/daily/YYYY-MM-DD.log ← per-day heartbeat telemetry (path = memory.log_path)
```

---

## A Day in the Life of a Message

Reading 14 subsystem sections back-to-back is a lot. Here's the same story told as a single concrete example: **what happens when the Architect types `"How does the Hailo-10H compare to the Hailo-8 for running 1.5B-parameter LLMs?"` into Telegram at 14:32 on a Tuesday.** Every arrow is a real line of code; every state file write is real.

```text
T+0 ms Telegram → connectors/telegram.py
├─ validate sender == cfg.api_keys.telegram_user_id ✓
├─ not a slash command, not a task request → Brain.think(user_input)
└─ pre-filter: looks_like_task_request()? ✗ (no "every X minutes")

T+2 ms Brain._cycle() begins
├─ soul.render_identity_block() (cached, <1 ms)
├─ monitor.sample() (cached <1 ms — 100ms only if stale)
├─ emotions.snapshot() → calm dominant, joy=0.2
├─ empathy.analyze(user_input) → sentiment=neutral, urgency=False
└─ identity regex match? ✗ (not asking for name/creator)

T+5 ms LTM short-circuit check
├─ keyword scan over memory/archive.md → top hit scores 1
└─ score < tools.ltm_short_circuit_min_score (2) → not short-circuited
→ cognition will run

T+8 ms CognitiveLoop.deliberate() — THINK stage
├─ Build active PLAN menu via SkillRegistry.plan_menu(cap, exclude_network)
│ ├─ vitals.is_stressed? No → cap stays "expensive"
│ ├─ provider.is_tripped? No → keep network skills
│ └─ menu: [ANSWER, RECALL, SEARCH] sorted free→expensive
└─ Call provider.generate(THINK prompt, user msg) ~600 ms (NPU)
→ INTENT: Compare H10 vs H8 for running 1.5B LLMs.
→ Q1: What are the H10's headline specs?
→ Q2: Why was the H8 unsuitable for LLMs?
→ Q3: Which Pi-class HAT ships the H10?

T+610 ms PLAN stage
└─ Call provider.generate(PLAN prompt with menu, sub-questions) ~700 ms
→ Q1: SEARCH "Hailo-10H specs"
→ Q2: RECALL
→ Q3: ANSWER

T+1310 ms ACT stage (3 concurrent skill invocations via asyncio.gather)
├─ skill_registry.invoke("research", ctx, query="Hailo-10H specs")
│ └─ ResearchSkill.execute()
│ └─ tool_registry.call("web_search", query=..., limit=5)
│ └─ httpx GET http://localhost:8080/search ~300 ms
│ → SkillResult(ok=True, evidence=[{title, url, snippet}, …])
│ → appended to state/skills-2026-05-17.jsonl
│ {ts, skill:"research", ok:true, latency_ms:302, tools_used:["web_search"], …}
│
├─ skill_registry.invoke("recall", ctx, query="Why was the H8 unsuitable for LLMs?")
│ └─ RecallSkill.execute()
│ └─ tool_registry.call("archive_read", query=..., limit=5) ~12 ms
│ → appended to state/skills-2026-05-17.jsonl
│
└─ ANSWER step:
└─ if cfg.cognition.dispatch_answer_via_skill → invoke "direct_answer" (one extra SLM call)
else → marker "(SLM training data only — no retrieval performed)"

T+1620 ms REFINE stage (if cfg.cognition.refine_enabled)
└─ Call provider.generate(REFINE prompt, intent + evidence) ~250 ms
→ "OK" (no gap) → one gap fires one extra SEARCH

T+1870 ms CognitiveLoop._persist() — append full trace to
state/deliberation-2026-05-17.jsonl (rotated by local date, 14-day retention)

T+1870 ms Brain._render_knowledge() → KNOWLEDGE block
Brain assembles final prompt: soul + vitals + mood + empathy + KNOWLEDGE
+ STM history (last 12 turns) + current user message

T+1875 ms provider.generate(synthesize prompt) ~1200 ms (NPU)
→ raw SLM output: "The Hailo-10H is the second-generation Pi 5 AI accelerator …"

T+3075 ms _looks_like_prompt_leak(raw)? ✗
PositiveFilter.apply(raw)
├─ hard reject regex? ✗
├─ soft rewrites? ✗ (no "I can't" / "impossible" / etc.)
└─ → filtered.text unchanged, filtered.altered=False

T+3078 ms stm.append("user", user_input)
stm.append("agent", filtered.text)
├─ deque (maxlen=12) appended — oldest turn silently dropped if full
└─ pending buffer +1 — will flush after 30s idle

T+3080 ms ReflectionEngine.fire_and_forget(input, response, kind="user", web_searched=True)
└─ background asyncio.Task (does NOT add to user-visible latency)
└─ provider.generate(critique prompt) ~700 ms
→ ReflectionEntry(quality="medium", interest="NPU benchmarks", lesson="...")
→ appended to state/reflection-2026-05-17.jsonl

T+3080 ms ThoughtTrace returned to TelegramConnector
└─ bot sends "Boss Eason — The Hailo-10H is … " ~150 ms (Telegram API)

T+3230 ms Architect sees the reply on Telegram.

──── meanwhile, in the background ────

Every 30 s Heartbeat.pulse_loop() ticks:
├─ Sample vitals → atomic-write state/vitals.json
├─ Decay emotions toward baseline
├─ Detect stress edges → state/vitals_events.jsonl
└─ If pending writes ≥1 AND idle ≥30 s → stm.flush_journal()

After 5 min idle Heartbeat fires _proactive_research():
└─ Picks an STM gap (e.g. "Hailo-10H benchmarks"),
verifies it's not in LTM and the SLM doesn't already know it,
runs one web search + a 2-sentence reflection,
writes to state/proactive.jsonl,
reflects on its own proactive thought.

At 02:00 Heartbeat.metabolism():
├─ Distill the day's logs + STM journal into archive.md
├─ Promote top facts to soul.md [CORE_MEMORIES]
├─ _consolidate_reflections(): mine reflection.jsonl + skills.jsonl
│ ├─ recurring interests → LEARNED_PREFERENCES
│ ├─ recurring lessons → EMOTIONAL_EVOLUTION
│ └─ chronically failing skills (≥3 invokes, >50% fail) → EMOTIONAL_EVOLUTION
└─ STM.purge() — clean slate for tomorrow.

At 06:00 Pulse loop resumes (no explicit "wake" event — it just notices
the clock has crossed `duty_start`).
```

**What this walkthrough demonstrates:**

- **Two SLM calls happen before the user sees a word** (THINK + PLAN), and one more (synth) after evidence is gathered. The structure exists because a 1.5B-parameter model is unreliable when asked to do everything in one shot.
- **The PLAN menu is built fresh per turn.** A stressed Pi or an offline SearXNG silently drops the expensive options — the SLM never gets to pick a verb the agent can't execute.
- **Every Skill invocation is audited** to `state/skills-YYYY-MM-DD.jsonl`. Tomorrow night's Sleep Metabolism will read that file and decide whether any Skill is chronically broken.
- **The reflection happens in the background.** The Architect's reply latency budget is ~3 s; reflection costs another ~700 ms but the user never waits for it.
- **The dashboard sees all of this without IPC** — Streamlit reads the atomically-written state files in a separate process.

---

## Hardware & Software Stack

### Reference hardware (~ USD $220)

| Component | Notes |
|---|---|
| Raspberry Pi 5 (8 GB) | Required — 4 GB works but tight under SLM load |
| **[Raspberry Pi AI HAT+ 2](https://www.raspberrypi.com/news/introducing-the-raspberry-pi-ai-hat-plus-2-generative-ai-on-raspberry-pi-5/)** | Optional — Hailo-10H NPU, **40 TOPS (INT4)**, **8 GB dedicated on-board RAM** (purpose-built for generative AI / LLMs on Pi 5). Drives the SLM via `hailo-ollama` on port 8000. PCIe-attached, Pi 5 only. |
| Active cooling (fan + heatsink) | Strongly recommended — vitals will fire `EXHAUSTION DIRECTIVE` without it |
| 64 GB+ A2-class SD card or NVMe HAT | NVMe extends SD-card lifetime considerably |
| 27 W official USB-C PSU | Power throttling triggers thermal stress |

> **Why the AI HAT+ 2 (and not the original AI HAT+)?** The first-gen AI HAT+ shipped the Hailo-8 / Hailo-8L, which was optimised for vision workloads (object detection, pose, segmentation). The **AI HAT+ 2 ships the Hailo-10H** with **8 GB of on-board accelerator RAM**, which is what unlocks 1–7B-parameter LLMs running locally on a Pi 5 — including the `qwen2.5-instruct:1.5b` model OpenCrayFish uses. The first-gen AI HAT+ cannot run this workload.

OpenCrayFish also runs fine on **any Linux/macOS dev box** for development — when no NPU is detected, the Provider transparently falls back to stock Ollama on CPU.

### Software dependencies (`requirements.txt`)

- Python **3.13+** (the runtime target — `pyproject.toml` requires `>=3.13` because `soul_handler.py` relies on `os.replace` fsync guarantees tightened in 3.13, and the AI HAT+ 2 Pi 5 image ships 3.13)
- `PyYAML`, `psutil`, `httpx`
- `python-telegram-bot` (Telegram connector)
- `streamlit` (dashboard + browser chat UI)
- `aiohttp` (web-chat HTTP bridge)
- `mcp` (Model Context Protocol client SDK — powers `tools/mcp_bridge.py`; lazy-imported, so deployments that don't use MCP can skip it)

External services on the device:

- **Ollama** (CPU fallback, port 11434) — `ollama pull qwen2:1.5b`
- **hailo-ollama** (NPU primary, port 8000) — Hailo's REST front-end (from the [Hailo Developer Zone](https://hailo.ai/developer-zone/software-downloads/?product=ai_accelerators&device=hailo_10h)) serving `qwen2.5-instruct:1.5b` from the Hailo-10H NPU on the AI HAT+ 2
- **SearXNG** (port 8080) — self-hosted metasearch instance for the agent's web tool

---

## Repository Layout

```text
OpenCrayFish/
├── README.md
├── soul.md
├── config.yaml
├── config_sample.yaml
├── pyproject.toml
├── requirements.txt
├── conftest.py
├── main.py
│
├── core/
│ ├── config.py
│ ├── soul_handler.py
│ ├── monitor.py
│ ├── emotions.py
│ ├── empathy.py
│ ├── positive_filter.py
│ ├── provider.py
│ ├── provider_manifest.py
│ ├── stm.py
│ ├── cognition.py
│ ├── brain/
│ │ ├── orchestrator.py
│ │ ├──
│ │ ├──
│ │ └── task_parsing.py
│ ├── intent_router.py
│ ├── reflection.py
│ ├── jsonl_writer.py
│ ├── dropin.py
│ ├── heartbeat.py
│ ├── scheduler.py
│ ├── cli.py
│ └── skills/
│ ├── base.py
│ ├── manifest.py
│ ├── registry.py
│ ├── discovery.py
│ ├── argspec.py
│ ├── identity.py
│ ├── recall.py
│ ├── direct_answer.py
│ ├── research.py
│ ├── self_reflect.py
│ ├──
│ └──
│
├── tools/
│ ├── base.py
│ ├── manifest.py
│ ├── registry.py
│ ├── discovery.py
│ ├── searxng.py
│ ├── archive_read.py
│ └── mcp_bridge.py
│
├── connectors/
│ ├── manifest.py
│ ├── registry.py
│ ├── discovery.py
│ ├── telegram.py
│ └── web_chat.py
│
├── ui/
│ ├── dashboard.py
│ ├── web_chat.py
│ └── panels/
│ ├── header.py
│ ├── vitals.py
│ ├── mood.py
│ ├── skills.py
│ ├── tools.py
│ ├── reflection.py
│ ├── deliberation.py
│ ├── tasks.py
│ ├── proactive.py
│ └── footer.py
│
├── examples/
│ └── opencrayfish-skill-echo/
│
├── scripts/
│ ├── smoke_cogni
│ ├── smoke_dashb
│ ├── smoke_foreg
│ ├── smoke_mcp_bridge.py
│ ├── smoke_rotat
│ ├── smoke_skill_menu.py
│ └── smoke_soul_atomic.py
│
├── tests/
│ ├── test_argspec.py
│ ├── test_cli.py
│ ├── test_connec
│ ├── test_dropin_discovery.py
│ ├── test_exampl
│ ├── test_intent_router.py
│ ├── test_jsonl_
│ ├── test_mcp_bridge.py
│ ├── test_positive_filter.py
│ ├── test_prompt_assembly.py
│ ├── test_provid
│ ├── test_provider_manifest.py
│ ├── test_skill_context.py
│ ├── test_skill_discovery.py
│ ├── test_skill_
│ ├── test_stm_rotation.py
│ ├── test_task_parsing.py
│ └── test_tool_context.py
│
├── plugins/
│ ├── skills/
│ ├── tools/
│ ├── connectors/
│ └── backends/
│
├── memory/
│ └── archive.md
│
├── logs/daily/
│ └── YYYY-MM-DD.log
│
└── state/
├── vitals.json
├── vitals_events.jsonl
├── proactive.jsonl
├── reflection-YY
├── reflection_dr
├── deliberation-
├── skills-YYYY-MM-DD.jsonl
├── skills.json
├── stm_journal.jsonl
├── tasks.yaml
├── tools.json
└── logs/agent.log
``` ← this file ← the agent's constitution + dynamic growth ← every tunable knob in the system (gitignored — holds secrets) ← committed template; `cp` it to config.yaml and fill in your keys ← canonical dependency list, entry-points, dev extras ← convenience copy of runtime deps for Pi 5 operators ← shared pytest fixtures (async event loop, tmp paths) ← wires every subsystem and starts the loops ← all the cognitive/biological subsystems ← typed YAML loader (Config dataclass) ← write-protected accessor for soul.md ← Vital signs sampling (CPU/RAM/temp/brain) ← 5-channel mood vector + decay + tuning ← user sentiment + urgency analyzer ← Pillar 2 hard output filter ← Ollama/Hailo backends + circuit breaker ← BackendManifest + SUPPORTED_BACKEND_PROTOCOL_VERSIONS ← short-term memory (RAM + journal) ← THINK → PLAN → ACT → REFINE loop (dispatches via SkillRegistry) ← prompt-assembly + orchestration package ← Brain class — top-level _cycle() pipeline prompt_assembly.py ← soul + vitals + mood + KNOWLEDGE + STM prompt builder identity_responder.py ← deterministic identity-class reply templater ← LLM-backed task-intent / task-action parsers ← shared NL pre-filter chain for both connectors ← self-critique engine (reads skills.jsonl for failure flags) ← date-rotating, retention-bounded JSONL appender ← shared drop-in folder loader (plugins// → PLUGIN / PLUGINS) ← pulse_loop + metabolism + proactive_thought ← recurring research-task scheduler ← `opencrayfish` CLI (skill/tool/connector/backend scaffolding + validation) ← capability layer above Tools (pluggable Skills) ← Skill protocol + SkillContext + SkillResult ← SkillManifest + SUPPORTED_PROTOCOL_VERSIONS ← SkillRegistry + dynamic PLAN menu + audit feed ← entry-point + drop-in Skill discovery ← runtime arg-schema validation + type coercion ← soul-templated identity replies (free, no-net) ← LTM keyword retrieval (cheap, no-net) ← SLM-only answer (cheap, no-net) ← SearXNG-backed web research (expensive, net) ← post-turn critique (cheap, no-net, background) proactive_learning.py ← idle-time curiosity (expensive, net, background) recurring_research.py ← scheduled topic refresh (expensive, net, background) ← low-level I/O primitives (called BY Skills) ← Tool plugin contract ← ToolManifest + SUPPORTED_TOOL_PROTOCOL_VERSIONS ← named tool registry + bind_context + audit ← entry-point + drop-in Tool discovery ← self-hosted web search (SearXNG client) ← long-term memory keyword reader ← MCP client bridge (remote MCP servers → Tools) ← external I/O channels ← ConnectorManifest + SUPPORTED_CONNECTOR_PROTOCOL_VERSIONS ← ConnectorRegistry + lifecycle + bootstrap_validate ← entry-point + drop-in Connector discovery ← Telegram Bot API connector ← in-process aiohttp HTTP bridge for Streamlit ← Streamlit apps ← live vital signs + proactive feed + tools ← browser-based chat UI for the live agent ← dashboard sub-panels (header, vitals, mood, skills, tools, reflection, deliberation, tasks, proactive, footer) ← reference third-party plug-in packages ← canonical hello-world Skill (pip-installable, with tests) ← standalone smoke-test + diagnostic runners tion_dispatch.py oard_rotation.py round_priority.py ion_reflection_identity.py ← pytest suite (≈240 tests incl. parametrize expansions) tor_registry.py e_echo_integration.py writer_schema.py er_backend_routing.py protocol_surface_v1.py ← OPTIONAL drop-in folder for local/private plug-ins (gitignored by convention) ← each `*.py` (or sub-package) declares `PLUGIN = MySkill` or `PLUGINS = [...]` ← same contract; loaded after entry-points so pip-installed wins on duplicate names ← (override the root location via the `OPENCRAYFISH_PLUGINS_DIR` env var) ← see [§ Hybrid Discovery](#hybrid-discovery--pip-or-drop-in-folder) for the full contract ← long-term distilled memory (gitignored) ← long-term distilled facts (LTM) ← per-day heartbeat telemetry (gitignored) ← PULSE / PROACTIVE / metabolism (path = memory.log_path) ← live runtime state (gitignored, read by dashboard) YY-MM-DD.jsonl ← date-rotated, 60-day default retention opped-YYYY-MM-DD.jsonl ← date-rotated, sidecar audit YYYY-MM-DD.jsonl ← date-rotated, 14-day default retention ← date-rotated, 30-day default retention ← live skill inventory snapshot ← rotating console log mirror

---

## Quick Start

```bash
# 1. Clone & venv
git clone https://github.com/easonlai/opencrayfish
cd opencrayfish
python3.13 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 2. Bring up the local stack (separate terminals or systemd units)
ollama serve # CPU fallback
ollama pull qwen2:1.5b
# Self-hosted web search — see the full setup section below; the one-line
# command here will work for the first boot but will return HTTP 403 on every
# search until you enable the JSON API. See § SearXNG Setup for the fix.
docker run -d --name searxng -p 8080:8080 searxng/searxng
# (Pi 5 only) hailo-ollama serve # NPU primary

# 3. Configure
cp config_sample.yaml config.yaml # config.yaml is gitignored — your secrets stay local
$EDITOR config.yaml # set telegram_token, telegram_user_id,
# architect_name, individual_designation
$EDITOR soul.md # optional: customize persona

# 4. Run the agent
python main.py

# 5. Open the dashboard (separate process, reads state/ files)
streamlit run ui/dashboard.py --server.port 8501

# 6. Open the browser chat
streamlit run ui/web_chat.py --server.port 8502

# 7. (Optional) Run the test suite
pip install -e ".[dev]" # adds pytest + pytest-asyncio + ruff
python -m pytest -q # ~240 tests, runs in <2s
```

Now talk to it on Telegram or in the browser. The agent will reply, remember, decay its mood, get curious during idle time, and once the clock crosses 02:00, it will sleep and consolidate the day.

---

## SearXNG Setup (Local Web Search)

OpenCrayFish's `research` Skill, the autonomous proactive cycle, and the recurring-task scheduler **all** depend on a working [SearXNG](https://docs.searxng.org/) instance reachable at `tools.searxng_url` (default `http://localhost:8080`). The agent calls SearXNG over its **JSON API** — and SearXNG ships that API **disabled by default**, which is the single most common "the agent doesn't search anymore" failure mode. This section walks through a known-good Docker deployment.

> **Why self-hosted?** Pillar 1 (Sovereign AI) mandates that the agent's web requests never leak to a third-party search provider. A self-hosted SearXNG aggregates dozens of upstream engines (Google, Bing, DuckDuckGo, Brave, Qwant, Wikipedia, …) without ever exposing the Architect's queries or IP. From OpenCrayFish's perspective, the wire format is plain JSON over HTTP — no API key, no rate-limit account, no vendor lock-in.

### Minimum viable deployment

```bash
# Pull and start the container (port 8080 on the host)
docker run -d \
--name searxng \
--restart unless-stopped \
-p 8080:8080 \
searxng/searxng:latest
```

The first start auto-generates an internal `/etc/searxng/settings.yml` inside a Docker-managed named volume. The container is reachable at `http://localhost:8080` but **the JSON API is OFF** — every call from OpenCrayFish will fail with `HTTP 403 Forbidden` until you enable it (see next subsection).

### Enable the JSON API (REQUIRED)

There are two equally-valid ways to do this. **Method A** is the fastest fix for an existing container; **Method B** is the cleaner long-term setup if you want your config in version control.

#### Method A — patch the in-container settings.yml (fastest)

Use this if you already have the container running with the default named volume:

```bash
# 1) Add "- json" to the formats list (idempotent — re-running is safe)
docker exec searxng sh -c "grep -q '^ - json' /etc/searxng/settings.yml || sed -i '/^ formats:/,/^[^ ]/ { /^ - html$/a\\
- json
}' /etc/searxng/settings.yml"

# 2) Restart so the change takes effect
docker restart searxng

# 3) Verify (should print HTTP 200 and a JSON payload)
curl -sS -o /dev/null -w 'HTTP %{http_code}\n' \
'http://localhost:8080/search?q=test&format=json'
```

The patch survives container restarts because it's persisted inside the named volume. It would only be lost if you `docker volume rm` the SearXNG volume.

#### Method B — bind-mount your own settings.yml (cleanest)

If you'd rather keep SearXNG's config in `~/searxng/` (or `/etc/searxng/` on a server, or anywhere git-tracked), use a bind mount instead of the default named volume. **Remove the existing container first** (the named volume can stay):

```bash
docker rm -f searxng 2>/dev/null

# Create a minimal settings.yml in your home directory
mkdir -p ~/searxng
cat > ~/searxng/settings.yml <<'YAML'
use_default_settings: true

server:
# Replace with `openssl rand -hex 32` for production; any 32+ char string works.
secret_key: "change-me-please-32-plus-character-string"
limiter: false # set true on public instances; false is fine on loopback
image_proxy: true

search:
formats:
- html
- json # ← required by OpenCrayFish
safe_search: 1 # 0=none, 1=moderate, 2=strict (OpenCrayFish sends 1)
YAML

# Start with the bind mount
docker run -d \
--name searxng \
--restart unless-stopped \
-p 8080:8080 \
-v ~/searxng/settings.yml:/etc/searxng/settings.yml:ro \
searxng/searxng:latest
```

Future config changes are just `$EDITOR ~/searxng/settings.yml && docker restart searxng`.

### End-to-end smoke test

After either method, the JSON endpoint should respond exactly like OpenCrayFish expects it to:

```bash
curl -sS 'http://localhost:8080/search?q=raspberry+pi+5&format=json' \
| python3 -c 'import sys,json; d=json.load(sys.stdin); print(f"results={len(d[\"results\"])}, first={d[\"results\"][0][\"url\"]}" if d.get("results") else "EMPTY")'
```

Expected output (URLs will vary):

```text
results=10, first=https://www.raspberrypi.com/products/raspberry-pi-5/
```

When this works, your agent's next `/research` (or any user query that triages to `SEARCH`) will succeed end-to-end. Cross-check `state/logs/agent.log` — you should see `TOOL call name=web_search status=ok latency_ms=...` instead of `status=fail ... 403 Forbidden`.

### Tuning that OpenCrayFish actually cares about

The defaults in `use_default_settings: true` are fine for local dev. Two SearXNG knobs are worth knowing about for a 24/7 edge deployment:

| `settings.yml` key | Recommended | Why |
|---|---|---|
| `search.formats` | `[html, json]` | **Required** — OpenCrayFish only speaks JSON. |
| `search.safe_search` | `1` (moderate) | Matches what `tools/searxng.py` always sends (`safesearch=1`). Setting `2` may filter results too aggressively; setting `0` may surface NSFW snippets to `proactive_learning`. |
| `server.limiter` | `false` on a loopback instance, `true` if you ever expose port 8080 to the LAN/WAN | The limiter rejects requests that look bot-like — including ones from your own agent. Leave it off unless port 8080 is publicly reachable. |
| `server.image_proxy` | `true` | Hides upstream image fetches behind SearXNG so favicons / thumbnails don't leak the Architect's IP. |
| `engines` (per-entry `disabled: true`) | disable anything noisy | If a particular search engine is rate-limiting your IP and polluting results with `unresponsive_engines` warnings, disable it. The aggregator works fine with the remaining engines. |

### Wiring it into `config.yaml`

The only OpenCrayFish-side setting is the URL:

```yaml
tools:
searxng_url: "http://localhost:8080" # http://:8080 if SearXNG is on another box
```

No auth, no API key. OpenCrayFish always sends `q=&format=json&safesearch=1` (`tools/searxng.py`'s `search(...)` and `Tool.call(...)` surfaces) — there's no other tuning to do.

### Troubleshooting

| Symptom in `state/logs/agent.log` | Likely cause | Fix |
|---|---|---|
| `TOOL call name=web_search status=fail ... 403 Forbidden` | JSON API not enabled | Apply **Method A** above. |
| `TOOL call name=web_search status=fail ... ConnectError` | Container not running, or port 8080 in use by something else | `docker ps`, `docker logs searxng`, `lsof -i :8080` |
| `TOOL call name=web_search status=ok ... hits=0` for every query | Upstream engines rate-limiting your IP; or `safe_search: 2` filtering everything | Inspect `docker logs searxng` for `unresponsive_engines`. Disable the noisy engines in `settings.yml` or lower `safe_search`. |
| `TOOL call name=web_search status=fail ... 429 Too Many Requests` | `server.limiter: true` is blocking the agent's own traffic | Set `limiter: false` in `settings.yml` and `docker restart searxng`. |

> **The agent degrades gracefully.** Even with SearXNG totally down, the `recall` and `direct_answer` Skills still work — `Cognition`'s ACT stage will silently drop the `SEARCH` evidence and synthesize from LTM + STM + SLM knowledge. The dashboard's **⚠️ Errors & warnings** panel will surface the SearXNG failures so you know to fix them, but the agent stays conversational. See [§ Failure-Mode Matrix](#failure-mode-matrix) for the full degradation contract.

---

## Deep Dive: How Everything Works

This is the comprehensive section. It documents every major subsystem in the order they are wired together in `main.py`, with file links into the codebase.

> **First time here?** Read [§ 0 The Brain Stack — Plain-English Tour](#0-the-brain-stack--plain-english-tour) below for a 5-minute orientation to how Brain, Skills, Tools, and Memory hand work to each other. Then come back to § 1 — § 14 for the full depth.

---

### 0. The Brain Stack — Plain-English Tour

Before the 14 subsystem deep-dives, here is the single mental model that ties them all together. **Four concepts, four jobs, one direction of flow.**

#### The four big ideas (one sentence each)

| Layer | What it is, in one sentence | Where it lives |
|---|---|---|
| **Brain** | The orchestrator that runs every reply through a fixed pipeline (gather context → think → plan → act → synthesize → filter → remember). It owns no facts, no I/O, no policy — it just sequences the other three. | [core/brain/orchestrator.py](core/brain/orchestrator.py) |
| **Skills** | The agent's *menu of decisions* — named verbs like `RECALL`, `SEARCH`, `ANSWER` that the SLM picks from during PLAN. Each Skill knows when it's worth picking, how expensive it is, and which Tools to compose. | [core/skills/](core/skills/) |
| **Tools** | The agent's *hands* — mechanical I/O primitives (web fetch, file read, future GPIO) with no policy, no fallback, no SLM. Skills compose Tools; the SLM never names a Tool directly. | [tools/](tools/) |
| **Memory** | The agent's *substrate of self* — a four-tier hierarchy from working RAM (the last 12 turns) through a disk journal up to nightly-distilled long-term archive and finally the soul itself. Every layer above (Brain, Skills, Tools) reads from or writes to this. | [core/stm.py](core/stm.py), [memory/archive.md](memory/archive.md), [soul.md](soul.md) |

#### How they talk to each other

```text
one user message arrives
│
▼
┌────────────────────────────────┐
│ BRAIN (orchestrator) │ reads: soul.md, STM, vitals, mood
│ Brain._cycle() in core/brain/ │ writes: STM (turns), trace, reflection
│ ┌──────────────────────┐ │
│ │ COGNITIVE LOOP │ │ decides WHICH Skill to call,
│ │ THINK→PLAN→ACT→REFINE │ │ and assembles the final prompt
│ └──┬───────────────────┘ │
└───────┼───────────────────────────┘
│ invoke("verb", ctx, **kwargs)
▼
┌────────────────────────────────┐
│ SKILLS (decisions) │ 7 today: identity, recall,
│ core/skills/* — each Skill knows: │ direct_answer, research,
│ • plan_verb (how SLM names it) │ self_reflect, proactive_learning,
│ • cost_tier (free/cheap/expensive) │ recurring_research
│ • requires_network (PLAN filter) │
│ • trigger_hints (WHEN to pick) │ audit: state/skills-*.jsonl
└───────┬───────────────────────────┘
│ await ctx.tools.call("web_search", q="...")
▼
┌────────────────────────────────┐
│ TOOLS (hands) │ 2 today: web_search (SearXNG),
│ tools/* — stateless I/O, no policy. │ archive_read (LTM)
└───────┬───────────────────────────┘
│ read/write
▼
┌────────────────────────────────┐
│ MEMORY (substrate) │ T1: deque (last 12 turns, RAM)
│ Four tiers, oldest at the bottom: │ T2: pending (RAM, flushed @30s idle)
│ T1 deque (working set) │ T3: journal (state/stm_journal.jsonl)
│ T2 pending buffer │ T4: archive.md (nightly distill)
│ T3 stm_journal.jsonl │ + soul.md (promoted facts)
│ T4 archive.md + soul.md │
└────────────────────────────────┘
```

#### The contracts between layers (read these in order)

1. **Brain doesn't know what Skills exist.** It asks `SkillRegistry.plan_menu(...)` at PLAN time and renders whatever it gets back into the SLM's prompt. Add a new Skill, restart, and the Brain immediately considers it — no Brain code change.
2. **Skills don't know what Tools exist.** They go through `ctx.tools.call("web_search", …)` by name. Replace SearXNG with a future Tool that satisfies the same Protocol, and every Skill that uses it keeps working with zero changes.
3. **Tools don't know about the SLM, the user, or policy.** They just do their one I/O job (HTTP GET, file read) and return a typed result. This is what makes them trivially unit-testable.
4. **Memory is read-many, write-few.** Brain and Skills read freely (cheap). Writes happen on tightly defined triggers: each user/agent turn (T1+T2), 30 s of idle (T3), and nightly Sleep Metabolism (T4). The hot path does zero disk I/O.

#### A worked example, layer by layer

The Architect types: **"Compare the Hailo-10H to the Hailo-8 for LLMs."**

| Step | Layer | What happens |
|---|---|---|
| 1 | Brain | Gathers soul + vitals + mood; the identity regex doesn't match, so cognition will run. |
| 2 | Brain | LTM short-circuit scan over `archive.md` — only 1 keyword hit, below the 2-hit floor, so cognition continues. |
| 3 | Brain → Cognitive Loop | Asks `SkillRegistry.plan_menu(cost_cap, exclude_network)` for the live menu — gets `[ANSWER, RECALL, SEARCH]` sorted free→expensive. |
| 4 | Cognitive Loop | THINK call → splits the question into Q1 (specs), Q2 (why H8 unsuitable), Q3 (which HAT). |
| 5 | Cognitive Loop | PLAN call → SLM picks `SEARCH "Hailo-10H specs"` for Q1, `RECALL` for Q2, `ANSWER` for Q3. |
| 6 | Skills | ACT dispatches concurrently: `research.execute()` (Q1), `recall.execute()` (Q2), `direct_answer.execute()` (Q3). Each Skill invocation is timed and appended to `state/skills-*.jsonl`. |
| 7 | Tools | `research` calls `ctx.tools.call("web_search", …)` → the SearXNG Tool hits `http://localhost:8080/search?format=json`. `recall` calls `ctx.tools.call("archive_read", …)` → reads `memory/archive.md` line by line. |
| 8 | Memory → Brain | All evidence is folded into the synth prompt alongside the last 12 STM turns. Brain calls `provider.generate()` to compose the final answer. |
| 9 | Brain | Runs the Positive Filter, appends both turns to STM (T1 + T2), fires reflection in the background. |
| 10 | Memory | The pending buffer flushes to `state/stm_journal.jsonl` (T3) after 30 s of silence. Tonight at 02:00, Sleep Metabolism will distill it into `archive.md` (T4). |

That's the whole agent in one trace. Every other deep-dive section below zooms into one layer of this picture.

#### How to extend each layer (the one-line guide)

- **Add a new Skill** → three options, all zero-fork: (a) drop a file in [core/skills/](core/skills/) satisfying the Skill Protocol and register it in [main.py](main.py) ([§ 12 Hello-World Skill](#hello-world-skill--step-by-step)); (b) ship it as a standalone `pip`-installable package with an `opencrayfish.skills` entry-point ([§ 12 Third-Party Skill Packages](#third-party-skill-packages-pip-install)); or (c) drop a `.py` file into `plugins/skills/` with `PLUGIN = MySkill` at the bottom ([§ Hybrid Discovery](#hybrid-discovery--pip-or-drop-in-folder)).
- **Add a new Tool** → same three options as Skills: drop into [tools/](tools/) and register in `main.py`, ship as a pip package via `opencrayfish.tools` entry-point, or drop into `plugins/tools/`. See [§ Adding a new tool](#adding-a-new-tool) for the protocol.
- **Add a new memory shelf** → don't, usually. Promote facts to `soul.md [CORE_MEMORIES]` via Sleep Metabolism instead.
- **Add a new connector** → wrap `Brain.think(…)` and stream the `ThoughtTrace` back. Three options: in-tree under [connectors/](connectors/), pip package via `opencrayfish.connectors` entry-point, or drop-in under `plugins/connectors/`. See [§ Adding a new connector](#adding-a-new-connector).

#### What this design buys you

- **The SLM is small (1.5B) and unreliable.** Brain compensates by splitting the work into four short prompts (each ≤120 tokens) instead of one giant chain-of-thought — each prompt has exactly one job and is parsed with regex, not JSON.
- **The PLAN menu is the only "intelligence routing" surface.** Adding a Skill is the only way to expand what the agent can choose to do. Everything else — Tools, Memory, Provider — is plumbing.
- **Layer isolation is enforced by contract, not by language tricks.** Skill execution is wrapped in a try/except in `SkillRegistry.invoke()` so a plugin crash can never escape into the Brain. A failed Tool call returns a typed error — it never raises into the calling Skill. A SLM timeout returns `backend="offline"` — it never raises into the connector. **Every interface is a graceful degradation point.** See [§ Failure-Mode Matrix](#failure-mode-matrix) for the full contract.

---

### 1. The Soul — `soul.md`

**Module:** [core/soul_handler.py](core/soul_handler.py) · **Data file:** [soul.md](soul.md)

The Soul is the agent's constitution. It is split into two regions by HTML comment markers:

```text

# [IDENTITY] ← codename, creator, status (Designation injected at runtime)
# [FUNDAMENTAL_LAWS] ← 1. Prime Directive 2. 365/20 3. Positive Anchor 4. Sovereignty
# [BEHAVIORAL_MATRIX] ← tone, ethics, persona

# [CORE_MEMORIES] ← elevated facts the Sleep Metabolism considers identity-defining
# [LEARNED_PREFERENCES] ← topics the Architect cares about, mined from reflection trends
# [EMOTIONAL_EVOLUTION] ← long-term mood/relationship signals

```

#### Key behaviors

- **Designation injection.** The agent's name (e.g. `"Dave Minion"`) is configured ONCE in `config.yaml` under `system.individual_designation`. `SoulHandler` injects it into the IDENTITY block at every read, so soul.md never carries a hardcoded designation line. This lets you redeploy the same soul.md as a different persona just by changing the config.
- **Hard write-protection.** The IMMUTABLE_CORE region is enforced with `_IMMUTABLE_RE` and `SoulProtectionError`. Any append that would mutate bytes inside the immutable markers (or relocate the marker itself) is rejected. Even SLM-driven appends get sanitized first via `_sanitize_dynamic_text` so the model cannot inject a fake `# [IDENTITY]` header.
- **Typed appends.** `append_core_memory()`, `append_preference()`, and `append_emotion()` are the only mutation paths, each scoped to its DYNAMIC_GROWTH subsection. This is the surface used by Sleep Metabolism to grow the agent over time.
- **Async lock.** All reads/writes are guarded by an `asyncio.Lock` so concurrent metabolism + reflection consolidation cannot interleave.
- **Snapshot rendering.** `render_identity_block()` produces the `[IDENTITY] / [FUNDAMENTAL_LAWS] / [BEHAVIORAL_MATRIX] / [CORE_MEMORIES]` excerpt that gets injected as the system-prompt prefix on every Brain cycle.

The soul.md you ship is the agent's *birth certificate*. The DYNAMIC_GROWTH region is its *biography*.

---

### 2. The Heartbeat — Time, Rhythm, and Metabolism

**Module:** [core/heartbeat.py](core/heartbeat.py)

The Heartbeat is what makes OpenCrayFish *alive* rather than *invocable*. It runs in its own asyncio task and never returns until shutdown.

#### Two main coroutines

1. **`pulse_loop()`** — ticks every `system.pulse_interval_seconds` (default 30s) during the **Active window** (06:00–02:00, configurable via `duty_start` / `sleep_start`).
2. **`metabolism()`** — runs ONCE per day, automatically the first time `_pulse()` notices the clock has crossed into the Sleep window (02:00–06:00).

#### What happens on each pulse

```text
_pulse() at T=now:
1. Determine if we're in duty or sleep window.
2. If just entered sleep → call metabolism() once, then publish state and return.
3. If just woke up → reset _last_interaction_at = now (clean idle clock).
4. monitor.sample() → fresh VitalSigns
5. emotions.decay() → exponential drift toward baseline
6. if vitals.is_stressed → emotions.nudge_many(vitals_stress)
→ record stress ENTER (rising edge only)
7. Append PULSE telemetry to /YYYY-MM-DD.log
(default: logs/daily/YYYY-MM-DD.log)
8. Publish state/vitals.json + push history sample
9. if pending_writes > 0 AND idle ≥ idle_journal_flush_seconds
→ stm.flush_journal()
10. if idle ≥ idle_proactive_minutes
→ _proactive_research()
→ reset idle clock so we don't spam
```

#### What happens on metabolism (02:00 once nightly)

```text
metabolism():
1. _collect_recent_logs() ← yesterday's + today's heartbeat telemetry
2. _collect_conversation_journal()
← stm.flush_journal() THEN read stm_journal.jsonl
so the day's actual chat is included
3. SLM extracts 3-5 "key facts" from the merged corpus
4. Append facts to memory/archive.md (LTM)
5. Promote the top 2 facts to soul.md [CORE_MEMORIES] (Soul Evolution)
6. _consolidate_reflections()
← scan state/reflection-*.jsonl for recurring
interest topics + lesson themes,
AND state/skills-*.jsonl for chronically
failing Skills (≥3 invokes, >50% fail rate)
← promote into LEARNED_PREFERENCES /
EMOTIONAL_EVOLUTION
7. stm.purge() ← wipe RAM deque + pending + journal
(the day's content has been consolidated)
```

#### Stress edge detection

Per-pulse stress is *not* logged or alerted (it would spam at every pulse the Pi is hot). Instead, `_record_stress_transition()` only emits **rising-edge ENTER** and **falling-edge EXIT** events to:

- `state/logs/agent.log` — operator-tailable warning lines
- `state/vitals_events.jsonl` — JSONL feed the dashboard renders as a stress timeline

ENTER events carry `temp`, `ram`, `cpu`, and a human-readable `reason` (the live `vitals.describe()` text). EXIT events add `duration_s`, `peak_temp`, `peak_ram` from the entire episode plus `current_temp` / `current_ram` / `current_cpu` snapshots and the same `reason` string — enough for the dashboard to render a stress timeline with explanatory tooltips without re-parsing `agent.log`.

#### Live-state publishing

Every pulse atomically writes `state/vitals.json` (using a `.tmp` + `os.replace` atomic swap) so the Streamlit dashboard — running in a *separate process* — can read a consistent snapshot without IPC. The snapshot contains:

```json
{
"ts": "...",
"is_sleeping": false,
"vitals": { "cpu_percent":..., "temperature_c":..., ... },
"brain": { "online": true, "backend": "hailo", "last_error": null,
"recovery_seconds": null },
"mood": { "joy":..., "anger":..., ... },
"stm": { "size": 7, "pending": 0 },
"history": [ ... rolling 60-sample sparkline ... ],
"counters": { "pulses": 1234, "proactive": 12, "stress": 3 },
"last_proactive": { "topic":..., "source":..., "ts":... }
}
```

---

### 3. The Vital Signs — `Monitor` & Homeostasis

**Module:** [core/monitor.py](core/monitor.py)

`Monitor.sample()` returns a `VitalSigns` dataclass with:

| Field | Source | Notes |
|---|---|---|
| `cpu_percent` | `psutil.cpu_percent(interval=0.1)` | 100 ms blocking — cached for `vitals_cache_ttl_seconds` |
| `ram_percent` | `psutil.virtual_memory()` | |
| `temperature_c` | `/sys/class/thermal/thermal_zone0/temp` | `None` on macOS dev boxes |
| `is_stressed` | hysteresis state machine | see below |
| `brain_online` | `provider.health().online` | SLM as a vital sign |
| `brain_backend` | active backend label (`"hailo"` / `"ollama"`) | |
| `brain_last_error` | last circuit-breaker cause, formatted | |
| `brain_recovery_seconds` | seconds until breaker auto-resets | |

#### Hysteresis (no flap)

Stress uses TWO thresholds:

```text
ENTER stress: temperature ≥ thermal_limit_celsius (default 75°C)
OR ram ≥ ram_limit_pct (default 85%)

EXIT stress: temperature ≤ thermal_release_celsius (default limit-5)
AND ram ≤ ram_release_pct (default limit-5)
```

This prevents the persona from oscillating between EXHAUSTION and normal turn-by-turn when the temp jitters around 75°C.

#### Brain availability as a vital sign

Because the SLM is the agent's *brain*, `Monitor.attach_provider(provider)` is called by `main.py` so every `sample()` synchronously polls `provider.health()` (cheap — no network) and embeds the result. When the brain goes offline:

- `vitals.describe()` appends `"Cognition link is DOWN — inference backend `` is unreachable."` to the prompt
- The dashboard renders a 🔴 BRAIN OFFLINE chip
- The web-chat UI shows an inline error banner

#### Force-stress mode

Set environment variable `OCF_FORCE_STRESS=1` to force `is_stressed=True` for testing the EXHAUSTION DIRECTIVE path on a cool dev machine.

---

### 4. The Provider — SLM Backend & Circuit Breaker

**Module:** [core/provider.py](core/provider.py)

The Provider abstracts the inference layer behind a single async method:

```python
await provider.generate(system_prompt, messages) -> str
```

#### Two backends, identical wire format

Both Hailo-Ollama (NPU, port 8000) and stock Ollama (CPU, port 11434) speak the **same `/api/chat` JSON contract**. The Provider tries the primary first (NPU when `hardware.npu_acceleration=true`), then falls back transparently to CPU on transport error. On a dev box without an NPU, set `npu_acceleration=false` and the Provider runs CPU-only with no per-pulse failover noise.

#### Circuit breaker

When **both** backends fail back-to-back, the Provider:

1. Raises `ProviderUnavailable` with a friendly first-person message ("I can't reach the inference service right now — both the NPU endpoint (port 8000) and the CPU fallback (port 11434) are offline. Start `ollama serve` (CPU) or `hailo-ollama` (NPU)…").
2. Arms an internal **circuit breaker** for `trip_seconds` (default 30s). During the trip window, every subsequent `generate()` call raises immediately without re-trying dead sockets.
3. Stores `last_error` (formatted as `": "`) and `_tripped_until` so `health()` can report:

```python
ProviderHealth(
online=False,
active_backend="hailo",
seconds_until_recovery=27.4,
last_error="ConnectError: All connection attempts failed",
)
```

#### How Brain handles it

`Brain._cycle()` catches `ProviderUnavailable` exactly **once**, at the top of the cycle, and returns a synthetic `ThoughtTrace` with `backend="offline"` and the friendly message in `filtered.text`. The connectors render whatever they get — they need no knowledge of the failure mode. This means **adding a new connector inherits offline behavior for free**.

`Brain.synthesize_task_report()` also re-raises `ProviderUnavailable` so the scheduler records `last_error` instead of broadcasting the friendly text as a real report.

---

### 5. The Memory System — STM / LTM / Sleep Metabolism

**Modules:** [core/stm.py](core/stm.py) · [core/heartbeat.py](core/heartbeat.py) (metabolism)

> **Plain English.** Memory is a four-tier waterfall. The newest turn lands in a 12-slot RAM ring (T1), is mirrored to a pending buffer (T2), and after 30 s of silence is flushed to a disk journal (T3) so a crash never loses it. Every night at 02:00, the day's journal is distilled into long-term prose in `archive.md` (T4), the most identity-shaping facts are promoted into `soul.md`, and T1/T2/T3 are wiped clean for tomorrow. The hot reply path **never** touches disk — only Heartbeat does.

OpenCrayFish has a **three-tier memory hierarchy**, modeled after the biological brain:

```text
┌──────────────────────────────────────────────────────────────────┐
│ TIER 1 — RAM working memory (deque, maxlen = stm_max_turns) │
│ Used as the conversation window passed to the SLM each turn. │
│ When full, oldest turn is silently dropped (Python deque). │
└──────────────────────────────────────────────────────────────────┘
│
every append() also pushes to:
▼
┌──────────────────────────────────────────────────────────────────┐
│ TIER 2 — Pending buffer (RAM list) │
│ Accumulates new turns since the last disk flush. │
│ Drained by Heartbeat after `idle_journal_flush_seconds` of │
│ silence, OR at sleep metabolism, OR at shutdown. │
└──────────────────────────────────────────────────────────────────┘
│
▼ flush_journal()
┌──────────────────────────────────────────────────────────────────┐
│ TIER 3 — Disk journal (state/stm_journal.jsonl) │
│ Durable backstop. Single open()/write()/close() per flush. │
│ fsync controlled by `journal_fsync_on_flush` (shutdown always │
│ fsyncs regardless). │
│ Replayed at boot by stm.recover() so a crashed agent wakes up │
│ with prior conversation context. │
└──────────────────────────────────────────────────────────────────┘
│
every night at 02:00:
▼
┌──────────────────────────────────────────────────────────────────┐
│ TIER 4 — Long-term memory (memory/archive.md, soul.md) │
│ Sleep Metabolism distills the day → archive.md; promotes top │
│ facts → soul.md [CORE_MEMORIES]. THEN purges TIER 1/2/3. │
└──────────────────────────────────────────────────────────────────┘
```

#### "Memory is full" — what actually happens

Python's `collections.deque(maxlen=N)` silently drops the oldest entry when a new one is appended past the cap. There is **no explicit eviction code** — that's the entire RAM-tier eviction policy. The dropped turn is *not lost*: it is still in the pending buffer (until flushed) and will be in the disk journal after the next flush, and ultimately distilled into archive.md tonight.

So the SLM only ever *sees* the last 12 turns, but the system *remembers* much more — it just retrieves it from a different layer.

#### Why this design

- **SD-card friendly:** the chat hot path does ZERO disk I/O. Disk writes only happen after 30s of silence (default), at metabolism, or at shutdown.
- **Crash-safe:** `recover()` rehydrates the deque from the journal at boot. Only un-flushed data within the last 30s of a sudden power loss is lost.
- **Bounded SLM context:** keeps `system_prompt + history` under qwen2:1.5b's effective attention window.
- **Daily reset:** `purge()` ensures yesterday's bullet-by-bullet detail doesn't pile up forever — the *meaning* survives in archive.md / soul.md, the *transcript* doesn't.

#### Knobs

| Config | Default | Effect |
|---|---|---|
| `memory.stm_max_turns` | 12 | RAM deque size |
| `system.idle_journal_flush_seconds` | 30 | Idle-flush threshold |
| `system.journal_fsync_on_flush` | false | Strict durability vs SD wear |

#### Related durable feeds (NOT in the STM/LTM tiers)

The agent also writes four high-frequency JSONL audit streams that are *adjacent to* but separate from the memory hierarchy above. Together with `stm_journal.jsonl` they form the five durable on-disk feeds:

| Feed | Owner | Date-rotated? |
|---|---|---|
| `state/stm_journal.jsonl` | `STM` | no (nightly purge) |
| `state/deliberation-YYYY-MM-DD.jsonl` | `CognitiveLoop` | yes (14d) |
| `state/skills-YYYY-MM-DD.jsonl` | `SkillRegistry` | yes (30d) |
| `state/reflection-YYYY-MM-DD.jsonl` | `ReflectionEngine` | yes (60d) |
| `state/reflection_dropped-YYYY-MM-DD.jsonl` | `ReflectionEngine` | yes (60d) |

Rotation + retention details live in [§ JSONL Rotation & Retention](#jsonl-rotation--retention). Reflection mines both `reflection.jsonl` (interest / lesson clusters) and `skills.jsonl` (chronic Skill failures) into `soul.md` during Sleep Metabolism — see [§ 10 Reflection](#10-self-reflection--the-self-learning-loop).

---

### 6. The Thinking Process — Brain & Cognitive Loop

**Modules:** [core/brain/orchestrator.py](core/brain/orchestrator.py) · [core/brain/prompt_assembly.py](core/brain/prompt_assembly.py) · [core/brain/identity_responder.py](core/brain/identity_responder.py) · [core/cognition.py](core/cognition.py)

> **Plain English.** Brain is a *sequencer*, not a thinker. For every reply it walks a fixed 11-step pipeline: read the soul, sample the body, feel the mood, read the user's tone, try a deterministic identity shortcut, try an LTM shortcut, otherwise spin up the Cognitive Loop (one prompt for THINK, one for PLAN, fan out the chosen Skills in ACT, optionally one REFINE round). The evidence is then folded into one final synth prompt, the SLM speaks, the Positive Filter scrubs it, the turn is committed to memory, and a self-critique fires in the background. **No single SLM call does more than one job, and every step has an explicit failure mode.**

Every reply the agent produces — whether triggered by a user message, a heartbeat proactive thought, or a scheduled task — flows through `Brain._cycle()`. The cycle is a strict pipeline:

```text
Brain._cycle(user_input | mission)
│
▼
┌─────────────────────────────────────────────────────────────┐
│ 1. Soul Context → soul.render_identity_block() │
│ 2. Physical State → monitor.sample() → vitals.describe() │
│ 3. Internal Mood → emotions.snapshot() │
│ 4. User Empathy → empathy.analyze(user_input) │
│ → emotions.nudge from empathy │
│ 4b. IDENTITY SHORTCUT ← regex catches "what's your name" │
│ (deterministic templated reply, zero SLM call) │
│ │
│ 5. LTM scan → archive.md keyword overlap score │
│ 5a. Cognitive Loop → THINK → PLAN → ACT (→ REFINE) │
│ (bypass conditions: │
│ • proactive turn • cognition disabled │
│ • vitals stressed • LTM short-circuit │
│ • explicit search verb in user input) │
│ 5b. Legacy Web Triage (only when cognition was bypassed) │
│ 5c. Build unified KNOWLEDGE block │
│ │
│ 6. Assemble prompt → soul + vitals + mood + empathy │
│ + KNOWLEDGE + STM history │
│ + current user message │
│ 7. provider.generate() → raw SLM output │
│ 8. Prompt-leak detector (drops responses regurgitating │
│ system-prompt scaffolding) │
│ 9. PositiveFilter.apply() → final text │
│ 10. STM.append("agent", text) │
│ 11. ReflectionEngine.fire_and_forget() (background) │
└─────────────────────────────────────────────────────────────┘
│
▼
ThoughtTrace (returned to connector)
```

#### Identity short-circuit (zero SLM call)

The qwen2:1.5b model **reliably mishandles** the most basic identity questions ("what is your name", "do you know my name", "how are you", "who created you"). From production logs, "what is your name" used to triage as a SEARCH and pollute the synth with Netflix's "My Name" + the anime "Your Name". Fix: detect a small set of unambiguous identity-class regexes BEFORE any cognition runs, and return a templated reply built from soul.md + config.yaml + live vitals/mood. **Deterministic. Zero round-trip. Zero hallucination.**

The "what is your name / who created you" branches delegate the actual templating to `IdentitySkill` via `skill_registry.invoke("identity", ctx, kind="name"|"creator")`. The Skill reads the IDENTITY block from `soul.md` and returns a short factual line; Brain wraps it with the live salutation/status sentence. If the registry call fails (no registry, exception, `ok=False`, empty summary), Brain falls back to an inline template — strictly additive, zero regression risk. The `who am I` and `how are you` branches stay inline because they need vitals / mood / architect-name that `IdentitySkill` doesn't see.

#### Cognitive Loop — THINK → PLAN → ACT → REFINE

Engaged on real user turns when no bypass condition fires. Each stage is **a single one-job SLM prompt** with hard token caps and regex parsing — *not* one giant chain-of-thought.

| Stage | What it does | Output cap |
|---|---|---|
| **THINK** | Restate user INTENT in one sentence + decompose into ≤ `cognition.max_subquestions` atomic Q1/Q2/Q3 sub-questions. | 120 tokens |
| **PLAN** | Assign exactly one verb from the **dynamic, registry-driven menu** to each sub-question. The shipping menu is `RECALL` (hits archive.md via the `recall` Skill), `SEARCH "..."` (hits SearXNG via the `research` Skill), and `ANSWER` (no retrieval needed, dispatched to `direct_answer`). Adding a new Skill with `plan_verb` automatically extends the menu — see [§ 12 Skills & Tools](#12-skills--tools--the-two-tier-capability-stack). | 120 tokens |
| **ACT** | Execute all PlanSteps **concurrently** via `skill_registry.invoke(name, ctx, **kwargs)`. Collect per-sub-question Evidence. Every invocation is timed, isolated from crashes, and appended to `state/skills-YYYY-MM-DD.jsonl`. | (executes verbs) |
| **REFINE** | (optional, capped at 1 round) Re-read intent + evidence; emit `OK` or `GAP: SEARCH "..."`; if gap, ACT it. | 40 tokens |

The full trace is appended to `state/deliberation-YYYY-MM-DD.jsonl` for audit. **Failures never raise** — the loop degrades to whatever evidence it managed to collect.

##### Dynamic PLAN menu + cost-tier auto-degradation

`SkillRegistry.plan_menu(...)` builds the PLAN-stage menu fresh on every turn, filtered by two runtime signals:

* **`cost_tier_cap`** — the operator baseline from `skills.default_cost_tier_cap`. `_active_plan_entries()` tightens it to `"cheap"` whenever `vitals.is_stressed` is true, so a hot or RAM-pressured Pi automatically stops picking expensive web research.
* **`exclude_network`** — set when `skills.auto_offline_filter` is true AND the Provider's circuit breaker has tripped OR the brain is otherwise offline. Any Skill with `requires_network=True` (currently `research`) drops out of the menu so the SLM can't pick a verb whose tool we can't reach.

The filtered menu is rendered into the PLAN prompt as `VERB(arg_hint) — description` lines sorted free → cheap → expensive, so the SLM is gently biased toward the

ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/easonlai/opencrayfish

Awesome Lists containing this project

README