{"id":51469556,"url":"https://github.com/kagenti/serverless-harness","last_synced_at":"2026-07-06T15:30:53.389Z","repository":{"id":368323520,"uuid":"1271781821","full_name":"kagenti/serverless-harness","owner":"kagenti","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-30T03:08:28.000Z","size":611,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-30T05:07:16.278Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kagenti.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-17T02:22:27.000Z","updated_at":"2026-06-30T02:33:22.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/kagenti/serverless-harness","commit_stats":null,"previous_names":["kagenti/serverless-harness"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/kagenti/serverless-harness","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagenti%2Fserverless-harness","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagenti%2Fserverless-harness/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagenti%2Fserverless-harness/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagenti%2Fserverless-harness/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kagenti","download_url":"https://codeload.github.com/kagenti/serverless-harness/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kagenti%2Fserverless-harness/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35197534,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-06T02:00:07.184Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-06T15:30:52.545Z","updated_at":"2026-07-06T15:30:53.374Z","avatar_url":"https://github.com/kagenti.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Serverless Harness\n\n**Run stateful AI coding agents serverless — scale to zero between turns, resume exactly where they left off.**\n\n![status](https://img.shields.io/badge/status-MVP%20(Phase%201)-success)\n![platform](https://img.shields.io/badge/platform-Knative%20%2B%20KEDA-blue)\n![runtime](https://img.shields.io/badge/runtime-Pi%20coding%20agent-informational)\n![node](https://img.shields.io/badge/node-22%2B-green)\n![license](https://img.shields.io/badge/license-Apache--2.0-blue)\n\nServerless Harness turns a long-lived AI agent into a **scale-to-zero workload** on Kubernetes.\nAn agent process normally has to stay resident — holding its conversation, tool state, and working\ndirectory in memory — even while it sits idle waiting for the next turn or for a human to approve a\nstep. That idle time is pure cost. The harness decouples the agent's **state** (durable in Redis)\nand its **tool execution** (an isolated sandbox pod) from the **agent process** itself, so the agent\nruns as a Knative service that drops to zero pods when idle and cold-starts with full session\ncontinuity on the next request.\n\nThe result is a **leaf-session backend**: an external orchestrator dispatches isolated units of agent\nwork (\"leaves\") over a simple HTTP + shared-volume contract, and the harness runs each one\nsync, async (queued), scheduled, or paused-for-approval — all on infrastructure that costs nothing at\nrest.\n\n## Table of Contents\n\n- [Why](#why)\n- [Architecture](#architecture)\n- [Features](#features)\n- [Quick Start](#quick-start)\n- [Deploy on OpenShift](#deploy-on-openshift)\n- [How It Works](#how-it-works)\n- [Dispatch Archetypes](#dispatch-archetypes)\n- [Repository Layout](#repository-layout)\n- [Evidence](#evidence)\n- [Roadmap](#roadmap)\n- [Documentation](#documentation)\n- [Status \u0026 License](#status--license)\n\n---\n\n## Why\n\n| Persistent agent | Serverless Harness |\n|------------------|--------------------|\n| Process stays resident between turns | Scales to **zero** when idle, cold-starts in sub-second |\n| State lives in process memory — lost on crash/evict | State lives in **Redis** — survives eviction, restart, and cold start |\n| Tools execute in the agent process | Tools execute in an **isolated sandbox pod** (brain/hands split) |\n| Idle compute billed continuously | **Only Redis + sandbox** stay resident (2 pods at rest) |\n| One invocation model | **Four**: sync, async fan-out, scheduled, human-gated |\n\nIn an idle-heavy workload [experiment](deploy/knative/EXPERIMENTS.md), the serverless path consumed\nroughly **a quarter** of the pod-seconds of an equivalent always-on agent — because the expensive\npart (the agent process) exists only while a turn is actively running.\n\n---\n\n## Architecture\n\n```mermaid\nflowchart LR\n    O[External Orchestrator] --\u003e|POST /runs| K[Knative Service\u003cbr/\u003escale-to-zero]\n    C[CronJob\u003cbr/\u003ecron-dispatch] --\u003e|schedule| K\n    K --\u003e|sync| R[runLeaf]\n    K --\u003e|async: true| Q[(Redis Streams\u003cbr/\u003equeue + session state)]\n    Q --\u003e W[KEDA ScaledJob\u003cbr/\u003eleaf-worker 0..N]\n    W --\u003e R\n    R --\u003e|kubectl exec| S[sandbox-0\u003cbr/\u003etool / code execution]\n    R \u003c--\u003e|session state| Q\n```\n\n| Component | Role |\n|-----------|------|\n| **Knative Service** | Scale-to-zero HTTP endpoint; runs a turn inline (sync) or enqueues it (async) |\n| **Redis** | Durable session state (resume by `sessionId`), work queue (Streams), gate state |\n| **KEDA ScaledJob** | Autoscales `leaf-worker` pods 0→N on queue depth (`lagCount` + `pendingEntriesCount`) |\n| **sandbox-0** | Persistent pod where all tool/code execution runs; reached via `kubectl exec` |\n| **Shared PVC** | Volume-envelope contract — inputs, results, and markers travel as files |\n| **CronJob** | Scheduled dispatch (`cron-dispatch`) for periodic batch work |\n\n\u003e **Note:** The Knative Service and the `leaf-worker` are the **same container image** with two entry\n\u003e points (`server.ts` vs `leaf-job.ts`). Both converge on `runLeaf()`, which routes execution into\n\u003e `sandbox-0`. The \"brain\" (model inference + session logic) runs in whichever pod called `runLeaf()`;\n\u003e the \"hands\" (actual command/tool execution) always run in the sandbox.\n\n---\n\n## Features\n\n- **Scale-to-zero turns** — Knative drops the agent to zero pods between turns; the activator\n  cold-starts a fresh pod on the next request.\n- **Durable resume** — sessions are append-only logs in Redis; a cold-started pod recalls full\n  conversation and state by `sessionId`, surviving pod eviction.\n- **Brain/hands isolation** — the agent never executes tools in its own process; everything runs in a\n  separate hardened `sandbox-0` pod via a persistent in-pod channel.\n- **Four dispatch modes** — one `/runs` endpoint serves sync, async-queued, cron-scheduled, and\n  human-gated execution (see [Dispatch Archetypes](#dispatch-archetypes)).\n- **Human-in-the-loop gates** — a leaf can pause mid-run, report `awaiting_approval`, and resume on an\n  external approve/reject/abort verdict — scaling to zero while it waits.\n- **Volume-envelope contract** — orchestrators pass inputs and collect results as files on a shared\n  PVC, decoupling result size from HTTP limits.\n- **Hardened by default** — non-root UID, read-only root filesystem, all capabilities dropped,\n  `RuntimeDefault` seccomp, no service-account token automount.\n- **Built on Pi** — wraps a pinned [`kagenti/pi`](https://github.com/kagenti/pi) coding agent through\n  an injectable `SessionStorageBackend` seam; the agent itself is unmodified.\n\n---\n\n## Quick Start\n\nBring up the full stack on a local [Kind](https://kind.sigs.k8s.io/) cluster and drive an agent that\nscales to zero and resumes from cold.\n\n\u003e **Prerequisites:** `kind`, `kubectl`, `docker`, and an Anthropic-compatible model credential.\n\n```bash\n# 1. Clone (the Pi agent is a submodule)\ngit clone --recurse-submodules https://github.com/kagenti/serverless-harness.git\ncd serverless-harness\n\n# 2. Provide a model credential — either a direct key...\nexport ANTHROPIC_API_KEY=sk-...\n#    ...or a Bearer-token gateway (e.g. LiteLLM):\n# export ANTHROPIC_BASE_URL=https://your-gateway\n# export ANTHROPIC_AUTH_TOKEN=...\n\n# 3. One-shot: create cluster, install Knative + Kourier, deploy Redis + sandbox + harness\n./deploy/knative/setup-kind.sh\n```\n\nIn a second terminal, expose the gateway and watch pods:\n\n```bash\nkubectl port-forward -n kourier-system svc/kourier 8080:80   # leave running\nwatch -n5 'kubectl get pods'                                  # in another pane\n```\n\n**Send the first turn** — a pod cold-starts to handle it, then scales back to zero:\n\n```bash\ncurl -s -H \"Host: serverless-harness.default.example.com\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"prompt\":\"Remember the secret word: pineapple. Reply only with OK.\"}' \\\n  http://localhost:8080/turn | jq .\n# =\u003e { \"sessionId\": \"019ed8e8-...\", \"response\": \"OK\" }\n```\n\n**Resume across a cold start** — wait ~90s for the pod to terminate, then ask on the *same* session.\nA fresh pod spins up from zero and still remembers the state from Redis:\n\n```bash\nexport SID=\"\u003csessionId from above\u003e\"\ncurl -s -H \"Host: serverless-harness.default.example.com\" \\\n  -H \"Content-Type: application/json\" \\\n  -d \"{\\\"sessionId\\\":\\\"$SID\\\",\\\"prompt\\\":\\\"What was the secret word?\\\"}\" \\\n  http://localhost:8080/turn | jq .\n# =\u003e response contains \"pineapple\"\n```\n\nSee [`serverless-harness-demo.md`](serverless-harness-demo.md) for the full guided walkthrough\n(including sandbox command execution) and [`deploy/knative/SMOKE.md`](deploy/knative/SMOKE.md) for the\nverified smoke-test claims.\n\n---\n\n## Deploy on OpenShift\n\nFor OpenShift (4.20+), use [`deploy/knative/setup-ocp.sh`](deploy/knative/setup-ocp.sh) — the\nOpenShift-native sibling of `setup-kind.sh`. It installs OpenShift Serverless (Knative + Kourier)\nvia the Red Hat operator, deploys Redis, the sandbox, the PVC and the harness Knative Service, and\nexposes it over an auto-created **OpenShift Route** (no port-forward).\n\n```bash\noc login ...                       # cluster-admin on OpenShift 4.20+\nexport ANTHROPIC_API_KEY=sk-...    # or the gateway vars above\n./deploy/knative/setup-ocp.sh      # add --dry-run to preview, --with-keda for async leaf\n```\n\nThe full guide — prerequisites, flags, storage/SCC notes, smoke testing over the Route, and\ntroubleshooting — is in **[`deploy/knative/README-ocp.md`](deploy/knative/README-ocp.md)**.\n\n---\n\n## How It Works\n\n1. **An orchestrator POSTs a leaf** to `/runs` (or `/turn` for a single interactive turn).\n2. The **Knative Service** wakes from zero, and either runs the leaf inline (`sync`) or pushes the\n   envelope onto **Redis Streams** and returns `202` (`async: true`), then idles back to zero.\n3. For async work, a **KEDA ScaledJob** scales `leaf-worker` pods up on queue depth and drains items.\n4. Both paths call **`runLeaf()`**, which executes all tools inside **`sandbox-0`** via `kubectl exec`.\n5. **Session state streams to Redis** as it goes, so the leaf is resumable by `sessionId` even if its\n   pod dies mid-run.\n6. **Results land as files** on a shared PVC, and a **done-marker** signals completion to the\n   orchestrator.\n\n---\n\n## Dispatch Archetypes\n\nThe same backend serves three orchestration patterns, all validated end-to-end on Kind:\n\n| Archetype | Pattern | Example use case |\n|-----------|---------|------------------|\n| **A — Async fan-out** | `{async:true}` → Redis Streams → KEDA scales workers 0→N → done-markers | \"Research 10 topics concurrently\" |\n| **B — Human gate** | Leaf pauses → `awaiting_approval` → external verdict → resume/terminate | \"Draft a clause, pause for legal sign-off, finalize\" |\n| **C — Scheduled** | CronJob → `cron-dispatch` reads a config list → posts each as async | \"Summarize yesterday's tickets at 02:00 daily\" |\n\n---\n\n## Repository Layout\n\n```text\nserverless-harness/\n├── packages/\n│   ├── session-backend/   # Generic append-only LogStore + Redis Streams impl\n│   ├── k8s-sandbox/       # Routes Pi tool execution to a remote pod (kubectl exec)\n│   ├── knative-server/    # HTTP server (server.ts) + leaf-worker (leaf-job.ts) entry points\n│   └── work-queue/        # Redis Streams work queue (async dispatch)\n├── harness/               # Pi SessionStorageBackend adapter (write-behind) + headless smoke\n├── pi-fork/               # Pinned Pi coding agent (submodule) with the injectable backend seam\n├── deploy/knative/        # Kind setup, manifests, smoke + experiment drivers\n├── experiments/           # @sh/experiments — reproducible cost/behaviour experiments\n└── docs/specs/            # Design specs (per-milestone) + milestone registry\n```\n\n---\n\n## Evidence\n\nBehaviour and economics are backed by reproducible experiments rather than claims:\n\n- **[`deploy/knative/EXPERIMENTS.md`](deploy/knative/EXPERIMENTS.md)** — cluster experiments E1\n  (economics), E3 (mobility), E4 (recovery), run live on Kind.\n- **[`docs/experiment-results.md`](docs/experiment-results.md)** — E2 (reconstruction cost) and E5\n  (budget enforcement) from the `@sh/experiments` workspace.\n- **[`deploy/knative/SMOKE.md`](deploy/knative/SMOKE.md)** — the 6/6 cold-start + resume smoke claims.\n\n---\n\n## Roadmap\n\n**Phase 1 — Decoupled Harness (built):** Redis session backend, remote sandbox client, persistent\nin-pod channel, Knative wrapper, compaction-checkpoint fast path, experiments, and the leaf-session\nbackend with all three dispatch archetypes. See the\n[milestone registry](docs/specs/README.md) for the source-of-truth status of every milestone.\n\n**Phase 2 — Zero-Trust Credential Plane (design complete, deferred):** a credential plane where\n*no component influenced by model output ever holds a raw secret.*\n\n| ID | Adds |\n|----|------|\n| Z1 | Per-session SPIFFE identity (SPIRE) |\n| Z2 | Secret-free, default-deny harness lock-down |\n| Z3 | Inference injector — provider-key chokepoint, mTLS to the LLM gateway |\n| Z4 | MCP code-mode in the sandbox |\n| Z5 | Generalized credentialed egress (sandbox forward proxy) |\n| Z6 | Subagents as isolated child sessions |\n| Z7 | Red-team + formal validation of the credential plane |\n\nToday the harness uses a trust-the-operator model: the model credential is a pre-provisioned\nKubernetes Secret, there is no egress policy, and all leaves share one service-account identity. Those\ngaps are exactly what Phase 2 closes.\n\n---\n\n## Documentation\n\n- [Deploy on OpenShift](deploy/knative/README-ocp.md) — `setup-ocp.sh` install guide (OCP 4.20+)\n- [Executive overview — leaf-session backend](docs/executive-overview-leaf-session.md)\n- [Milestone registry](docs/specs/README.md) — authoritative milestone numbering and status\n- [Design specs](docs/specs/) — one dated design doc per milestone\n- [`harness/README.md`](harness/README.md) — local dev build (Pi workspace build order, headless smoke)\n\n---\n\n## Status \u0026 License\n\nThis is explorative work. It is an MVP — the scale-to-zero, durable-resume, sandbox-isolation, \nand dispatch features above are built and smoke-verified; the zero-trust credential plane is \ndesigned but not yet implemented. Interfaces may change.\n\nLicensed under the [Apache License 2.0](LICENSE).\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkagenti%2Fserverless-harness","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkagenti%2Fserverless-harness","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkagenti%2Fserverless-harness/lists"}