{"id":49377261,"url":"https://github.com/jwekavanagh/agentskeptic","last_synced_at":"2026-05-17T06:04:05.508Z","repository":{"id":349241099,"uuid":"1200094800","full_name":"jwekavanagh/agentskeptic","owner":"jwekavanagh","description":"Catch silent failures in agent workflows by verifying real system state","archived":false,"fork":false,"pushed_at":"2026-05-13T00:53:33.000Z","size":19175,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-13T01:03:15.300Z","etag":null,"topics":["agents","ai","automation","cli","developer-tools","llm","observability","testing","typescript","verification","workflows"],"latest_commit_sha":null,"homepage":"https://agentskeptic.com","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jwekavanagh.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":".github/CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":"docs/governance.md","roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-04-03T03:08:13.000Z","updated_at":"2026-05-13T00:53:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"3edc9b19-b1dc-4047-bff0-2a8c1ff9818c","html_url":"https://github.com/jwekavanagh/agentskeptic","commit_stats":null,"previous_names":["jwekavanagh/project-nod","jwekavanagh/workflow-verifier","jwekavanagh/agentskeptic"],"tags_count":66,"template":false,"template_full_name":null,"purl":"pkg:github/jwekavanagh/agentskeptic","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwekavanagh%2Fagentskeptic","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwekavanagh%2Fagentskeptic/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwekavanagh%2Fagentskeptic/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwekavanagh%2Fagentskeptic/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jwekavanagh","download_url":"https://codeload.github.com/jwekavanagh/agentskeptic/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jwekavanagh%2Fagentskeptic/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33129100,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-16T18:38:32.183Z","status":"online","status_checked_at":"2026-05-17T02:00:05.366Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agents","ai","automation","cli","developer-tools","llm","observability","testing","typescript","verification","workflows"],"created_at":"2026-04-28T03:06:01.619Z","updated_at":"2026-05-17T06:04:05.501Z","avatar_url":"https://github.com/jwekavanagh.png","language":"TypeScript","funding_links":[],"categories":["Applications","Software"],"sub_categories":["Tools","AI \u0026 LLM Testing"],"readme":"\u003c!-- discovery-readme-title:start --\u003e\n# AgentSkeptic — state vs trace\n\u003c!-- discovery-readme-title:end --\u003e\n\n\u003c!-- discovery-acquisition-fold:start --\u003e\n## Trust reality, not traces.\n\nTool effects vs read-only store facts.\n\nTraces can show success while stored data disagrees.\n\nAgentSkeptic re-checks the stores your agent claims to change, then returns a deterministic Outcome Certificate before you ship.\n\n### Bundled terminal proof\n\n```text\n### Success (`wf_complete`) — canonical `agentskeptic check`\n\nstderr (first lines): truth_check_verdict: trusted, then release_critical_truth_check_verdict: trusted\nstdout (Outcome Certificate excerpt): {\"schemaVersion\":3,\"workflowId\":\"wf_complete\",\"runKind\":\"contract_sql\",\"stateRelation\":\"matches_expectations\"}\n\n### Failure (`wf_missing`)\n\nstderr (first line): truth_check_verdict: not_trusted\nHuman report then explains ROW_ABSENT (missing downstream row vs registry expectation).\nstdout (Outcome Certificate excerpt): {\"schemaVersion\":3,\"workflowId\":\"wf_missing\",\"runKind\":\"contract_sql\",\"stateRelation\":\"does_not_match\"}\n\nOptional coverage budgets: when configured (`--coverage-budget` or `agentskeptic/coverage-budget.json` beside `--project`), stderr adds `coverage_budget_verdict:` and `coverage_budget_detail:` after the two truth lines — see docs/integrate.md § Optional coverage budgets.\n```\n\n[How it works](https://agentskeptic.com/database-truth-vs-traces)\n\u003c!-- discovery-acquisition-fold:end --\u003e\n\n\u003c!-- adoption-canonical:start --\u003e\n## Default path: one truth check\n\n**Start here:** **[`docs/first-truth-check.md`](docs/first-truth-check.md)** — canonical first-run steps (command, inputs, stdout/stderr, CI, Cursor, troubleshooting).\n\nCompare recorded tool activity to downstream state (SQL and, in contract mode, HTTP witnesses, object storage, vectors, Mongo per your registry) and get **Outcome Certificate v3** on stdout (**`schemaVersion: 3`**, **`failureSpine`**, **`evidenceCompleteness`**) plus **`truth_check_verdict:`** and **`release_critical_truth_check_verdict:`** on stderr ([**Trust artifact naming glossary**](docs/outcome-certificate-normative.md#trust-artifact-naming-glossary) explains receipts and decision-bundle `exit.json` naming). Optional **coverage budgets** append two more machine lines only when a policy is active — **[`docs/integrate.md`](docs/integrate.md#optional-coverage-budgets)**.\n\n```bash\nnpx agentskeptic check --workflow-id wf_example \\\n  --project ./path/to/your-app \\\n  --db ./path/to/readable.sqlite\n```\n\nWith the conventional layout, **`--registry`** and **`--events`** default to **`./path/to/your-app/agentskeptic/tools.json`** and **`events.ndjson`**. Pass them explicitly when your paths differ. Shortest path: [`docs/first-truth-check.md`](docs/first-truth-check.md). Full integrator SSOT: [`docs/integrate.md`](docs/integrate.md).\n\n**No license required.** The default `agentskeptic check` path needs no `AGENTSKEPTIC_API_KEY` and no license server; it runs stateless contract verification locally. (Stateful **`agentskeptic enforce`** for baselines, drift, and acceptance is a later opt-in commercial path — see below.)\n\n**Reading the result.** stdout is one **Outcome Certificate v3** line (machine JSON as above). On verdict exits, stderr begins with **`truth_check_verdict:`** and **`release_critical_truth_check_verdict:`** (each `trusted`, `not_trusted`, or `unknown`), then the human certificate report. When a coverage budget policy is active, two additional machine lines follow — see **[`docs/integrate.md`](docs/integrate.md#optional-coverage-budgets)**.\n\nThe **`truth_check_verdict`** values mean:\n\n```text\ntruth_check_verdict: trusted\ntruth_check_verdict: not_trusted\ntruth_check_verdict: unknown\n```\n\n| Verdict | Meaning |\n|---------|---------|\n| `trusted` | Checked outcome matched expected downstream state — only this verdict means the workflow can be relied on. |\n| `not_trusted` | Determinate mismatch or required state missing. Do not claim verified; fix the mismatch. |\n| `unknown` | Evidence incomplete or not established. Do not claim verified; collect missing evidence or narrow checked scope. |\n\nFull verdict and stderr contract: [`docs/first-truth-check.md`](docs/first-truth-check.md) (details in [`docs/integrate.md`](docs/integrate.md#first-truth-check)).\n\n**Exportable activation (advanced):** `BootstrapPackInput` v1 + **`agentskeptic activate`** (writes **`proof/`** under **`--out`** on exits 0–2; **`bootstrap`** is legacy — [`docs/bootstrap-pack-normative.md`](docs/bootstrap-pack-normative.md)).\n\n### Lifecycle\n\n1. Keep **`agentskeptic/tools.json`** in version control; update when `toolId` → verification mapping changes.\n2. Emit observations via the canonical SDK emitter, then append emitted rows to the gate buffer. Optionally mirror the same JSON lines to **`agentskeptic/events.ndjson`** for CI replay.\n3. On the code path **before** irreversible work you control (ship, bill, ticket close), call **`await gate.assertSafeForIrreversibleAction()`** so **unsafe** trust (or required emissions that never reached the gate) blocks **that** branch — it is not a substitute for wiring the gate everywhere it matters, and outcomes can still be **`unknown`** when **`highStakesReliance`** is not **`permitted`** (see [`docs/outcome-certificate-normative.md`](docs/outcome-certificate-normative.md)).\n\n### Compare two saved Outcome Certificates (semantic diff)\n\nTo compare **trust posture** between two saved **`schemaVersion: 3`** Outcome Certificate JSON files (no workflow payloads or event streams):\n\n```bash\nnpx agentskeptic compare certificates --before ./prior-outcome.json --after ./current-outcome.json\n```\n\n**Stdout** is **`VerificationDiffCertificateV1`** ([`schemas/verification-diff-certificate-v1.schema.json`](schemas/verification-diff-certificate-v1.schema.json)); **stderr** is human-readable text (not JSON). Multi-run **structural** regression compare still uses **`agentskeptic compare --manifest`** ([`docs/regression-artifact-normative.md`](docs/regression-artifact-normative.md)). Full CLI contract: **[`docs/agentskeptic.md`](docs/agentskeptic.md#verification-diff-outcome-certificate-v3)**.\n\n### Install\n\n```bash\nnpm install agentskeptic\n```\n\n### Network access (OSS CLI)\n\nLocal contract verification (for example **SQLite file** `--db`) runs **offline by default**: anonymous **product-activation telemetry** is **disabled** unless you opt in with **`AGENTSKEPTIC_TELEMETRY=1`** or persist **`{\"telemetry\": true}`** in **`~/.agentskeptic/config.json`**. Use **`AGENTSKEPTIC_TELEMETRY=0`** to force telemetry off. When enabled, the CLI sends best-effort anonymous usage events to AgentSkeptic’s telemetry endpoint (**`POST /api/funnel/product-activation`**); it does **not** send workflow payloads, database contents, credentials, prompts, traces, or verification artifacts. Outbound access can still occur when you explicitly configure **remote databases**, **`--share-report-origin`**, state witnesses (**HTTP / vector / S3 / Mongo**), **commercial** license or enforcement flows (**`AGENTSKEPTIC_API_KEY`**), **`funnel-anon pull`**, or when running the **`website/`** app (Stripe, email, databases, etc.).\n\n### Code\n\n```bash\nnpx agentskeptic init --framework none --database sqlite --yes\n```\n\n```ts\nimport { join } from \"node:path\";\nimport { AgentSkeptic } from \"agentskeptic\";\n\nconst skeptic = new AgentSkeptic({\n  registryPath: join(\"agentskeptic\", \"tools.json\"),\n  databaseUrl: join(process.cwd(), \"demo.db\"),\n});\n\nconst certificate = await skeptic.check({\n  workflowId: \"wf_complete\",\n  observations: [\n    {\n      toolId: \"crm.upsert_contact\",\n      params: { recordId: \"c_ok\", fields: { name: \"Alice\", status: \"active\" } },\n    },\n  ],\n});\n```\n\n### Python / LangGraph / CrewAI (same truth check)\n\nThe default verification contract is unchanged: **`agentskeptic check` semantics**, Outcome Certificate on stdout, and the **`truth_check_verdict:`** / **`release_critical_truth_check_verdict:`** stderr prefix—whether you invoke the published **npm CLI** alongside your stack or use the **Python SDK / extras** documented in **[`docs/integrate.md`](docs/integrate.md)**. Start here: **`pip install`** and framework notes there, plus **[`examples/python-verification/README.md`](examples/python-verification/README.md)**.\n\nSee **[`docs/integrate.md`](docs/integrate.md)** (canonical integrator guide — see title in integrate.md) and [`docs/migrate-2.md`](docs/migrate-2.md) for 1.x → 2.0 renames.\n\n### CI (GitHub Actions)\n\n**Default:** copy **[`examples/github-actions/agentskeptic-check.yml`](examples/github-actions/agentskeptic-check.yml)** — first-party **[composite action](https://docs.github.com/en/actions/how-tos/sharing-actions-and-workflows/recording-deployment-history-and-assignments/creating-composite-actions)** at [`.github/actions/agentskeptic-check`](.github/actions/agentskeptic-check) wrapping **`agentskeptic check`** (default **`mode`**). No `AGENTSKEPTIC_API_KEY` on the OSS path. In another repo, pin upstream with `uses: jwekavanagh/agentskeptic/.github/actions/agentskeptic-check@\u003cref\u003e` (not Marketplace). Advanced CLI flags pass through **`extra-args`**. Transparent manual fallback: invoke `npx agentskeptic check …` inline (same CLI contract). The composite’s default **`agentskeptic@latest`** **`package`** input is **not** a branch-protection posture—pin **`package`** per **[`docs/ambient-ci-distribution.md#composite-package-input-contract-normative`](docs/ambient-ci-distribution.md#composite-package-input-contract-normative)**.\n\nEvery composite run produces three CI surfaces: a **certificate-derived job summary** (failure spine + per-step table + witness kinds), a downloadable artifact named **`agentskeptic-outcome-certificate`** (`outcome-certificate.json`), and **structured composite outputs** (`verdict`, `state-relation`, `trust-decision`, `release-critical-verdict`, `failing-tool-ids`, `primary-reason-codes`, `failing-witness-kinds`, `recommended-action`, `automation-safe`, `certificate-path`, plus the existing `stdout-path` / `stderr-path` / `exit-code`). The OSS example needs only **`permissions: contents: read`** — artifact upload uses `ACTIONS_RUNTIME_TOKEN`, not `GITHUB_TOKEN`, so no `actions: write` scope is required. Full contract: **[`docs/ambient-ci-distribution.md`](docs/ambient-ci-distribution.md)**.\n\n**Opt-in commercial / stateful enforcement** (baseline, drift, acceptance): **[`examples/github-actions/agentskeptic-commercial.yml`](examples/github-actions/agentskeptic-commercial.yml)** requires an API key and license server ([`docs/commercial-enforce-gate-normative.md`](docs/commercial-enforce-gate-normative.md)).\n\u003c!-- adoption-canonical:end --\u003e\n\n## Buy vs build: why not only SQL checks\n\n**The scar (one pattern, over and over):** the trace says the tool succeeded—here **`crm.upsert_contact`** / **`contacts`**—but the row is missing or wrong. The repo demo names it **`wf_missing`** / **`ROW_ABSENT`**; **the same failure shape** applies whenever your registry maps tool activity to SQL state (ledgers, orders, tickets—not only CRM). That is not a logging problem—it is a **money and risk** problem the moment you ship, bill, close, or treat the run as audit evidence.\n\n**Why “we’ll just write SQL checks” stops working**\n\n- **Drift:** Scripts rot when schemas and workflows change; nobody keeps them current.\n- **No ownership:** The author leaves; the checks become folklore.\n- **Not an org contract:** Expectations live in heads and one-off files—not in a shared **`tools.json`** + **NDJSON** contract everyone replays.\n- **CI and audit:** Ad-hoc checks are skipped locally and rarely ship as **repeatable artifacts**; when the issue is cross-team or compliance, scripts do not hold. Use **CI lock** / enforcement when you need pins ([`docs/ci-enforcement.md`](docs/ci-enforcement.md)).\n\n**What you standardize on instead:** when the row backs revenue or customer promises, **you stop betting the business on whoever wrote the last script.** AgentSkeptic is how the org **owns** the check: one verifier, one replayable contract, **Quick → Contract** when stakes go up—explore with **Quick Verify** ([`docs/quick-verify-normative.md`](docs/quick-verify-normative.md)), lock with **contract** mode and a **`tools.json`** registry when “we ran a query” is not evidence ([`docs/agentskeptic.md`](docs/agentskeptic.md)). **That is the responsible default** once the failure mode hurts.\n\n**Core mechanism:** Read-only SQL checks that your database **at verification time** matches **expectations derived from structured tool activity**—not whether a trace step “succeeded.”\n\n\u003c!-- public-product-anchors:start --\u003e\nRead-only checks at verify time—not color.\n\n- **Repository:** https://github.com/jwekavanagh/agentskeptic\n- **npm package:** https://www.npmjs.com/package/agentskeptic\n- **Canonical site:** https://agentskeptic.com\n- **Integrate:** https://agentskeptic.com/integrate\n- **Runtime truth-check (CLI / SDK):** https://github.com/jwekavanagh/agentskeptic/blob/main/docs/first-truth-check.md\n- **OpenAPI (hosted commercial API):** https://agentskeptic.com/openapi-commercial-v1.yaml\n- **Verification Contract Manifest:** https://agentskeptic.com/contract/v1.json\n- **llms.txt (agents, site):** https://agentskeptic.com/llms.txt\n- **llms.txt (repo, raw):** https://raw.githubusercontent.com/jwekavanagh/agentskeptic/refs/heads/main/llms.txt\n- **llms.txt (repo, blob):** https://github.com/jwekavanagh/agentskeptic/blob/main/llms.txt\n\n\u003c!-- public-product-anchors:end --\u003e\n\n## Advanced\n\n**Canonical runnable (same API as README `### Code`):** after `npm run build`, run `node examples/decision-gate-canonical.mjs`.\n\n## Try it (about one minute)\n\nThis is the fastest way to see **`ROW_ABSENT`** versus **verified** on the same screen—the concrete failure mode the section above is about (bundled CRM-style demo, not your production incident yet).\n\n**Prerequisite:** **Node.js ≥ 22.13** (built-in [`node:sqlite`](https://nodejs.org/api/sqlite.html)), or use [Docker](#docker-quickstart-optional) below.\n\n**Fast first run on your own DB:** run the same `agentskeptic check` from the [Default path](#default-path-one-truth-check) above against your inputs — after `npm install` and `npm run build`:\n\n```bash\nagentskeptic check --workflow-id \u003cid\u003e --events \u003cpath\u003e --registry \u003cpath\u003e --db \u003csqlitePath\u003e\n```\n\nstdout is the Outcome Certificate; stderr begins with `truth_check_verdict: trusted|not_trusted|unknown` and `release_critical_truth_check_verdict: trusted|not_trusted|unknown`, then the human report (optional coverage budget lines when configured).\n\n**Local feedback loop with run history (advanced):** `agentskeptic loop` wraps the same `check` contract and adds local run history, prior-run comparison, and a single TRUSTED / NOT TRUSTED / UNKNOWN line for tight inner-loop iteration — normative contract: [`docs/local-feedback-loop.md`](docs/local-feedback-loop.md).\n\n**Advanced compatibility paths:** `agentskeptic quick`, `agentskeptic crossing`, and `agentskeptic verify-integrator-owned` remain supported for specialized workflows and CI parity; they are not the first-run path.\n\n```bash\nnpm install\nnpm start\n```\n\n**What you should see:** `npm start` builds, seeds **`examples/demo.db`**, and runs two workflows from **`examples/events.ndjson`** with **`examples/tools.json`**. The first case ends **`complete` / `verified`**; the second **`inconsistent` / `missing`** with reason **`ROW_ABSENT`**. That contrast is the product on one screen.\n\n`npm install` does not compile TypeScript. To run the CLI without `npm start`, run **`npm run build`** first so **`dist/`** exists.\n\n### Docker quickstart (optional)\n\nUse this when you want the bundled demo without Node **22.13+** on the host. The repo is bind-mounted so **`examples/demo.db`** stays on your machine.\n\n**Bash / macOS / Linux** (repo root):\n\n```bash\ndocker run --rm -it -v \"$PWD:/work\" -w /work node:22-bookworm bash -lc \"npm install \u0026\u0026 npm start\"\n```\n\n**PowerShell** (repo root):\n\n```powershell\ndocker run --rm -it -v \"${PWD}:/work\" -w /work node:22-bookworm bash -lc \"npm install \u0026\u0026 npm start\"\n```\n\n## Minimal model (event → registry → result)\n\n**One structured observation** (NDJSON line; full schema in [Event line schema](docs/agentskeptic.md#event-line-schema)):\n\n```json\n{\"schemaVersion\":1,\"workflowId\":\"wf_complete\",\"seq\":0,\"type\":\"tool_observed\",\"toolId\":\"crm.upsert_contact\",\"params\":{\"recordId\":\"c_ok\",\"fields\":{\"name\":\"Alice\",\"status\":\"active\"}}}\n```\n\n**Registry entry** (excerpt; full file is **`examples/tools.json`**) telling the engine how that `toolId` maps to a row check:\n\n```json\n{\n  \"toolId\": \"crm.upsert_contact\",\n  \"verification\": {\n    \"kind\": \"sql_row\",\n    \"table\": { \"const\": \"contacts\" },\n    \"identityEq\": [{ \"column\": { \"const\": \"id\" }, \"value\": { \"pointer\": \"/recordId\" } }],\n    \"requiredFields\": { \"pointer\": \"/fields\" }\n  }\n}\n```\n\n**When the row matches:** workflow result (excerpt; demo prints full JSON to stdout):\n\n```json\n{\n  \"workflowId\": \"wf_complete\",\n  \"status\": \"complete\",\n  \"steps\": [{ \"seq\": 0, \"toolId\": \"crm.upsert_contact\", \"status\": \"verified\" }]\n}\n```\n\nWhen the row is missing or fields disagree, you get **`inconsistent`** / **`missing`** and reason codes such as **`ROW_ABSENT`**.\n\n## What this is (and is not)\n\nRetries, partial failures, and race conditions mean a success flag in a trace is not proof the intended row exists with the right values. The engine derives **expected** state from your registry and events and compares it to **observed** state with read-only `SELECT`s.\n\n| This **is** | This is **not** |\n|-------------|-----------------|\n| A **SQL ground-truth state check** against expectations from structured tool activity | Generic observability, log search, or arbitrary unstructured logs |\n| A verifier for **persisted state** after agent or automation workflows | A test runner for application code |\n| Proof that **observed DB state matched expectations** at verification time | Proof that a tool **executed**, **wrote**, or **caused** that state |\n\n**This is for you if** you need persisted-row SQL truth after agent or automation runs when the trace looks fine but the DB might not.\n\n**This is not for you if** you need proof a tool executed, log search as verification, or a model where read-only SQL against your app DB is not the right check. Homepage “for you / not for you” copy lives in **`website/src/content/productCopy.ts`** (single source with the site).\n\n**Trust boundary (once):** a green trace does **not** prove the row exists with the right values—only whether **read-only `SELECT`s** matched **expected** rows under your rules, not deep causality.\n\n**Declared → expected → observed** (how reports reason about runs):\n\n1. **Declared** — what the captured tool activity encodes (`toolId`, parameters).\n2. **Expected** — what should hold in SQL under the rules (in **Quick Verify**, inferred; in **contract mode**, registry-driven from events).\n3. **Observed** — what read-only SQL returned at verification time.\n\n## Contract path (registry + events)\n\n**CLI:** the canonical local replay command is **`agentskeptic check`** (see [Default path](#default-path-one-truth-check) above) — the same command CI and the Cursor rule wrap. After **`npm install`** and **`npm run build`**, run it via `agentskeptic check` (or `node dist/cli.js check`). Postgres: **`--postgres-url`** instead of **`--db`** (exactly one).\n\nTypical integration:\n\n1. Emit **one NDJSON line per tool observation** (see [Event line schema](docs/agentskeptic.md#event-line-schema)).\n2. Add a **registry** entry per `toolId` (start from **`examples/templates/`**).\n3. Run the truth check:\n\n```bash\nnpm run build\nagentskeptic check --workflow-id \u003cid\u003e --events \u003cpath\u003e --registry \u003cpath\u003e --db \u003csqlitePath\u003e\n```\n\nReplay the bundled files: **`wf_complete`** / **`examples/events.ndjson`** / **`examples/tools.json`** / **`examples/demo.db`** (same flags as above).\n\n**From source without `agentskeptic` on PATH:** `node dist/cli.js` with the same flags.\n\n**Why SQLite in the demo:** file-backed ground truth with no extra services. The demo (re)creates **`examples/demo.db`**; verification still uses read-only SQL.\n\n## Quick Verify and assurance (optional)\n\n**Quick Verify** (`agentskeptic quick`): inferred checks, **no registry file**; **provisional**, not audit-final—graduate to **contract mode** for explicit per-tool expectations. Full contract: **[`docs/quick-verify-normative.md`](docs/quick-verify-normative.md)**.\n\n**Input contract:** We only accept **structured tool activity**—JSON or NDJSON that describes tool calls and parameters our ingest model can extract—not arbitrary logs, traces, or unstructured observability text.\n**Quick** uses read-only SQL against the database you pass in. **Contract** verification adds registry-backed checks for HTTP witnesses, object storage, vector indexes, and Mongo where configured—see [`docs/verification-state-stores.md`](docs/verification-state-stores.md).\n\n```bash\nnpm run build\nagentskeptic quick --input test/fixtures/quick-verify/pass-line.ndjson --db examples/demo.db --export-registry ./quick-export.json\n```\n\nUse **`--postgres-url`** instead of **`--db`**; **`-`** as **`--input`** reads stdin.\n\n**Assurance** (`assurance run` / `assurance stale`): multi-scenario sweeps and staleness over saved reports; success paths emit one **`AssuranceOutputV1`** JSON line on stdout (embedded **`runReport`**)—**[Assurance subsystem](docs/agentskeptic.md#assurance-subsystem-normative)**, **[`examples/assurance/manifest.json`](examples/assurance/manifest.json)**.\n\n## Sample output (contract demo)\n\nThe **`npm start`** driver prints human report + workflow JSON to **stdout** (one stream for the demo). Normal CLI: machine JSON on **stdout**, human report on **stderr**—[Human truth report](docs/agentskeptic.md#human-truth-report). **Full success/failure transcripts** (same strings as below) are in the [acquisition fold](#your-traces-say-success-your-database-disagrees) at the top of this README.\n\nOperational note: `agentskeptic check` / `agentskeptic quick` persist one verification receipt JSON per run under `artifacts/agentskeptic-receipts/` (write/schema failures are fail-closed with exit 3).\n\n### Success (`wf_complete`)\n\n*Interpretation:* Under the configured rules, **expected** state matched **observed SQL** for this step—**state alignment**, not proof of execution.\n\n### Failure (`wf_missing`)\n\n*Interpretation:* **Expected** state from the tool activity implied a row **observed SQL** did not find—**inconsistent**—a gap traces alone often miss. Still not proof a write was attempted or rolled back.\n\n## How this differs from logs, tests, and observability\n\n| Approach | What it tells you |\n|----------|-------------------|\n| **Logs / traces** | A step ran, duration, errors—not “row X has columns Y.” |\n| **Unit / integration tests** | Code paths in your repo—not production agent runs against live DB state. |\n| **Metrics / APM** | Health and latency—not semantic equality of persisted records. |\n| **Ad-hoc SQL checks / one-off scripts** | Same failure mode as [**Buy vs build**](#buy-vs-build-why-not-only-sql-checks)—drift, weak ownership, not a durable contract. |\n| **agentskeptic** | Whether **observed SQL** matches **expectations from declared tool parameters** (contract mode), via read-only SQL—not proof the tool executed. |\n\n## When to run it\n\nRun **after** a workflow (or CI replay of its log), **before** you treat the outcome as safe for customer-facing or regulated actions.\n\n**Inputs:** NDJSON observations, registry JSON, read-only **SQLite** or **Postgres**. Semantics: [`docs/relational-verification.md`](docs/relational-verification.md).\n\n**Typical uses:** block a release, trigger human review, open an incident, or attach a verification artifact to an audit trail.\n\n**CI with over-time guarantees:** use stateful **`agentskeptic enforce`** baseline/check/accept lifecycle—[`docs/ci-enforcement.md`](docs/ci-enforcement.md).\n\n## Further capabilities (reference)\n\nEverything beyond core contract verification lives in **[`docs/agentskeptic.md`](docs/agentskeptic.md)**—subcommands, hooks, bundles, debug, plan transition, human report layout, exit codes.\n\n## Documentation map\n\n| Doc | Purpose |\n|-----|---------|\n| [`docs/contract.md`](docs/contract.md) | **Verification Contract Manifest** SSOT — names, hashes, and versions the event/registry/registry-export schemas; one URL, one CI gate |\n| [`docs/epistemic-contract.md`](docs/epistemic-contract.md) | **Normative epistemic contract** (grounded output vs funnel; ranking limits; telemetry proxies)—single authored source; other docs link or generate from here |\n| [README — Buy vs build](#buy-vs-build-why-not-only-sql-checks) | Canonical **buy vs build** narrative (failure mode, scripts limits, Quick → Contract) |\n| [`docs/agentskeptic.md`](docs/agentskeptic.md) | Authoritative CLI and behavior reference (SSOT) |\n| [`docs/cursor-integration.md`](docs/cursor-integration.md) | Consumer Cursor rule template for running local `agentskeptic check` truth checks |\n| [`docs/quick-verify-normative.md`](docs/quick-verify-normative.md) | Quick Verify normative contract |\n| [`docs/verification-product.md`](docs/verification-product.md) | Product intent, trust boundary, authority matrix |\n| [`docs/reconciliation-vocabulary.md`](docs/reconciliation-vocabulary.md) | Reconciliation dimension IDs and UI mapping |\n| [`docs/verification-operational-notes.md`](docs/verification-operational-notes.md) | First-run runbooks, TTFV, export vs replay coverage |\n| [`docs/langgraph-reference-boundaries.md`](docs/langgraph-reference-boundaries.md) | LangGraph reference path: emitter/CLI boundaries and test chain |\n| [`docs/langgraph-checkpoint-trust.md`](docs/langgraph-checkpoint-trust.md) | LangGraph checkpoint trust: v3 wire, terminal contract, shared kernel, production gate |\n| [`docs/relational-verification.md`](docs/relational-verification.md) | Relational verification semantics |\n| [`docs/ci-enforcement.md`](docs/ci-enforcement.md) | CI enforcement and lock fixtures |\n| [`docs/correctness-definition-normative.md`](docs/correctness-definition-normative.md) | Correctness and limits (normative) |\n\n## Development and testing\n\n**Why SQLite:** same note as under [Contract path](#contract-path-registry--events) (file-backed demo DB; read-only verification SQL).\n\n**`npm test`** runs **`npm run verification:truth`** (regeneration + contract gate, Postgres distribution, then full journey suite). Requires **`DATABASE_URL`** and **`TELEMETRY_DATABASE_URL`** (see [`website/.env.example`](website/.env.example)). Ordering: **[`docs/testing.md`](docs/testing.md)**.\n\n**Full CI parity** (Postgres + Playwright for Debug Console): set **`POSTGRES_ADMIN_URL`** and **`POSTGRES_VERIFICATION_URL`**, then **`npm run test:ci`**. See **[`docs/testing.md`](docs/testing.md)**, [`.github/workflows/ci.yml`](.github/workflows/ci.yml), and: `docker run -d --name etl-pg -p 5432:5432 -e POSTGRES_PASSWORD=postgres postgres:16`.\n\n## Commercial CLI (npm) vs OSS (this repo)\n\n\u003c!-- codegen:buyer-truth:start:COMMERCIAL_ENTRY --\u003e\n**Commercial metering (published npm)** uses `AGENTSKEPTIC_API_KEY` + `POST /api/v1/usage/reserve` as documented in docs/commercial.md — account-pooled quota per billing month.\n\n**OSS/unmetered CLI** for single-run verification: clone this repo and use the OSS build (`WF_BUILD_PROFILE=oss` / default `npm run build` artifact). State over-time `enforce` needs the commercial CLI and a paid entitlement.\n\u003c!-- codegen:buyer-truth:end:COMMERCIAL_ENTRY --\u003e\n\nCanonical write-up: **[`docs/commercial.md`](docs/commercial.md)** (npm package, Stripe, keys, telemetry, validation, entitlements; operator metrics in **[`docs/funnel-observability.md`](docs/funnel-observability.md)**—OSS CLI product-activation posts are **opt-in** via **`AGENTSKEPTIC_TELEMETRY=1`** or persisted config; **`AGENTSKEPTIC_TELEMETRY=0`** forces them off). OSS builds in this repo run stateless contract checks via **`agentskeptic check`** (and positional compatibility invocation) / **`quick`** without a license server. Stateful **`agentskeptic enforce`** (commercial / stateful / opt-in) and over-time guarantees require a commercial build per **[`docs/commercial-enforce-gate-normative.md`](docs/commercial-enforce-gate-normative.md)**.\n\n**GitHub Actions:** default OSS **truth check** (composite **`agentskeptic check`**) — **[`examples/github-actions/agentskeptic-check.yml`](examples/github-actions/agentskeptic-check.yml)** + [`.github/actions/agentskeptic-check`](.github/actions/agentskeptic-check). Opt-in **enforcement** — **[`examples/github-actions/agentskeptic-commercial.yml`](examples/github-actions/agentskeptic-commercial.yml)**. See **[`docs/ambient-ci-distribution.md`](docs/ambient-ci-distribution.md)**.\n\n## Status, contributing, security\n\n**Maturity:** **0.x** (`package.json`). APIs, CLI flags, and JSON schemas may evolve; rely on tests and docs for current contracts.\n\n**Contributing:** see **[CONTRIBUTING.md](CONTRIBUTING.md)**.\n\n**Security:** see **[SECURITY.md](SECURITY.md)**.\n\n## License\n\nReleased under the **MIT License** — **[LICENSE](LICENSE)**.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjwekavanagh%2Fagentskeptic","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjwekavanagh%2Fagentskeptic","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjwekavanagh%2Fagentskeptic/lists"}