https://github.com/zhjai/agent-completion-gate
Stop agents from marking work done that isn't — a fail-closed completion gate + four-state machine: the worker only proposes done, an external verifier grants it. Reads real artifacts, not self-report. Built on agent-memory.
https://github.com/zhjai/agent-completion-gate
agent-memory agent-skill ai-agents ai-coding-agents ai-governance ai-guardrails claude-code-skill completion-gate fail-closed guardrails llmops policy-engine
Last synced: 5 days ago
JSON representation
Stop agents from marking work done that isn't — a fail-closed completion gate + four-state machine: the worker only proposes done, an external verifier grants it. Reads real artifacts, not self-report. Built on agent-memory.
- Host: GitHub
- URL: https://github.com/zhjai/agent-completion-gate
- Owner: zhjai
- License: mit
- Created: 2026-06-01T18:19:24.000Z (7 days ago)
- Default Branch: main
- Last Pushed: 2026-06-01T20:38:39.000Z (7 days ago)
- Last Synced: 2026-06-01T21:17:15.048Z (7 days ago)
- Topics: agent-memory, agent-skill, ai-agents, ai-coding-agents, ai-governance, ai-guardrails, claude-code-skill, completion-gate, fail-closed, guardrails, llmops, policy-engine
- Language: Python
- Size: 22.5 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# agent-completion-gate
English · 中文
> **Make AI coding agents prove they are done.** The agent can only *propose* done; a check reads your **real output files** and only then grants `complete`.
AI coding agents are goal-driven. Give Codex or Claude Code a goal and it optimizes hard toward the main line — build the page, fix the bug, produce the run. On longer tasks it often skips the user-visible details that were implied, scattered through the thread, or never written as a test.
**The goal is not the acceptance criteria.**
Example — "add a monthly sales report page" can end with a page that exists and tests that pass, while the CSV export is missing, the chart has one data point, the title still says "Untitled", and the empty state is broken. The agent honestly believes it's done. That's the problem.
`agent-completion-gate` turns "done" into an external acceptance check. The agent can only propose `candidate_complete`; a protected gate reads the **real artifacts** and grants `complete` only when a **human-written** acceptance manifest passes. Plain files + one Python script — no service, no account, no lock-in.
> The gate does not infer what the user meant; a human distills the acceptance criteria into the manifest, and the gate prevents the agent from self-certifying against anything less. It complements your tests and CI — it checks user-visible acceptance surfaces teams rarely unit-test (a missing export, a degenerate chart, a renamed run), not code correctness.
OpenSpec helps you define **what to build** before coding; `agent-completion-gate` checks whether the finished artifacts **satisfy acceptance** before the agent can call the task done.
## See it in action (30 seconds)
Watch the gate read real files and disagree with an agent that says "done":
```bash
pip install pyyaml
git clone https://github.com/zhjai/agent-completion-gate && cd agent-completion-gate
sh examples/minimal-project/run.sh
```
The everyday case — "add a monthly sales report page". The agent reports `candidate_complete` both times; only the real artifacts differ:
```
===== BEFORE — agent did the headline task, missed the details (expect BLOCKED) =====
FAIL report_has_multiple_points: rows points=1 (min 2)
FAIL csv_export_present: file exports/monthly.csv exists=False
-> BLOCKED (exit 1). The agent could NOT call this done.
===== AFTER — agent fixed the real artifacts (expect COMPLETE-OK) =====
PASS report_has_multiple_points: rows points=3 (min 2)
PASS csv_export_present: file exports/monthly.csv exists=True
-> COMPLETE-OK (exit 0).
```
More: [`examples/run.sh`](examples/run.sh) (overstep / blocked / granted), [`examples/diff_demo.sh`](examples/diff_demo.sh) (catch a worker under-reporting what it touched), [`examples/swanlab/`](examples/swanlab/) (the real ML incident that motivated this kit).
## Lazy mode (goal-first) — recommended
Install the skill once, then just **say your goal** — no YAML, no manual init:
```bash
npx skills add zhjai/agent-completion-gate -g -a claude-code # or -a codex, cursor, … any host
```
```text
设计goal: 做一个月度销售报表页，要有图表、CSV 导出、空状态、正确标题
# or: "goal: add a monthly sales report page with chart, CSV export, empty state, correct title"
```
The `goal-compile` skill fires automatically and:
1. **auto-initializes** the gate if this repo doesn't have it yet,
2. **compiles your goal** into acceptance criteria (surfaces + machine checks + review items),
3. **shows you the criteria once, in plain language, to confirm** (like confirming a plan) — *before* doing the work,
4. does the task, then runs the gate until it passes.
You confirm "what counts as done" once; the agent does the rest. **It can't grade itself** — it drafts the criteria, you confirm them, and only the external gate grants `complete`. (Say "just do it, don't ask" for a fully-automatic *self-check* instead — reported as `SELF-CHECK-OK`, which is **not** a verified completion.)
> Why the one confirmation: "don't drift" only means something against a target *you* set. If the agent both writes the acceptance criteria and grades itself, it never finds itself off-track. Confirming the criteria pins the target.
### When it triggers (and when it stays out of your way)
It's tuned **conservative** — it only steps in for work that's worth a gate, and stays silent on small stuff:
| Your request | Gate? |
|---|---|
| "做一个月度销售报表页" / "implement user login" / "帮我完成这个功能" | ✅ multi-step / produces a user-visible artifact |
| "fix this typo" / "给这个函数加个参数" / "why does this error?" | ❌ handled directly, no ceremony |
| "帮我完成这个任务" but the task is tiny | ❌ it right-sizes and just does it |
| "use the gate to do X" / "用 gate 做 X" | ✅ explicit — always, regardless of size |
When unsure it **defaults to not gating** (you can always say *"use the gate"* to force it). The bias is deliberate: a missed trigger costs one sentence to fix; ceremony on every typo would just train you to ignore it. Triggering is by intent (no magic prefix), so it's not 100% precise — the two escape hatches above cover both directions.
## Manual setup (if you'd rather wire it yourself)
```bash
cd your-project
npx skills add zhjai/agent-completion-gate -g -a claude-code # or -a codex, cursor, … any host
```
Then **you** (a human) scaffold the gate into your repo — one command, no manual copying:
```bash
# the engine + scaffolder live in the repo (the skill teaches the procedure; it doesn't ship the engine)
git clone https://github.com/zhjai/agent-completion-gate /tmp/acg
cd your-project && sh /tmp/acg/scripts/init.sh --dest .
```
It creates `gate/` (the engine + an **empty, passable** manifest), `control/surface_inventory.yaml`, `state/`, `.github/workflows/completion-gate.yml`, and a CODEOWNERS example. Idempotent; never clobbers your edited specs without `--force`. *(Prefer typing one line to your agent? Ask it to "set up the completion gate" — the `completion-gate-init` skill runs this same script. The script is the source of truth.)*
> **Fresh install is intentionally permissive.** Empty specs *pass* — at this point the gate only stops the agent from self-declaring `complete`; it does **not** yet know your project's artifacts. To make it useful you add at least one surface and one check.
**1 — Define what "done" means** (the human distills intent into checks; the gate doesn't infer it). Edit `control/surface_inventory.yaml`:
```yaml
surfaces:
- id: report
user_visible: true
paths: ["artifacts/report.json"]
```
…and `gate/acceptance_manifest.yaml`:
```yaml
checks:
- id: report_has_multiple_points
surface: report
type: min_series_points
artifact: "artifacts/report.json"
series: "rows"
min_points: 2
review_items: []
```
Built-in check types: `file_exists`, `config_not_disabled`, `min_series_points`, `max_chart_count`, `identity_in_name` (extend `run_machine_check()` for your own). A fuller worked spec: [`examples/swanlab/`](examples/swanlab/).
**2 — Run it locally:**
```bash
printf 'status: candidate_complete\ntouched_surfaces: [report]\nreview_queue: []\n' > state/completion_candidate.yaml
python3 -E gate/check_acceptance.py --manifest gate/acceptance_manifest.yaml \
--inventory control/surface_inventory.yaml --candidate state/completion_candidate.yaml --repo .
```
Missing the data points → `BLOCKED`. Once the real artifact is right → `COMPLETE-OK`.
**3 — Make it the authority.** The scaffolded `.github/workflows/completion-gate.yml` runs this on every PR. Mark the **`verify-completion` job a required status check**, and CODEOWNERS-protect `gate/`, `control/`, and the workflow (see the generated `.github/CODEOWNERS.completion-gate.example`). Now `complete` means exactly one thing: that check is green. Trust model + the agent Stop-hook option: [`integrations/README.md`](integrations/README.md).
## What the agent does after you install the skill
The `completion-audit` skill instructs the agent: at task wrap-up, write `state/completion_candidate.yaml` (`status: candidate_complete`, plus the surfaces it touched), then run the gate. The agent can reach **at most `candidate_complete`** — only the external verifier (CI / a hook) ever writes `complete`. If the gate blocks, it fixes the real artifacts and re-audits. Your loop: *do the work → audit completion → CI verdict → fix blocked reasons or merge.*
## How it works — the four states
```
in_progress ──► candidate_complete ──►(EXTERNAL verifier)──► complete
│ └─► blocked
└────────► blocked (needs-review / unknown surface / missing evidence)
```
The worker can only reach `candidate_complete` or `blocked`. **Only an external verifier writes `complete`.** **`needs-review == blocked`** (not an annotation the agent can set and move past). The kit ships the **check, the contract, and the wiring**: `check_acceptance.py` returns a verdict; [`gate/verify_completion.sh`](gate/verify_completion.sh) enforces the state machine around it (rejects a worker that wrote `complete` itself; grants only on a clean pass); [`integrations/`](integrations/) attaches it as CI / hook. Full contract: [`STATE_MACHINE.md`](STATE_MACHINE.md).
## Why a gate, not a rule / skill / memory
- A **rule** is advisory — a goal rationalizes past it.
- A **skill** can be skipped — the agent chooses not to invoke it.
- **memory** records belief, not verified truth.
- Only a **gate the agent can't edit, on a path it can't skip, reading artifacts it can't fake** reliably stops "looks done but isn't."
## Where it fits
```
OpenSpec — planning before coding (agree on what to build)
agent-lessonbook — capture corrections & drift lessons during the work
agent-completion-gate — acceptance before "done"
```
[`agent-lessonbook`](https://github.com/zhjai/agent-lessonbook) is an **optional companion** that captures process lessons during execution. This gate is **standalone** — it never reads lessonbook (or any memory) at runtime; it reads only its own `--manifest`/`--inventory`. Only a human may translate a recurring lesson into the gate's protected manifest.
## Security model & invariants
Hardened across multiple heterogeneous (Codex × Claude) review rounds — each invariant closed a reproduced bypass. **"External + fail-closed under a trusted base branch + runner"**, not "unbypassable":
1. Gate + manifest + inventory are **protected** (read-only, outside the agent-writable workspace, maintained only through human/CI-reviewed changes). `check_acceptance.py --agent-writable-root DIR` enforces this at runtime.
2. Inspect **real artifacts**, never `run_state`.
3. **Unknowns fail closed** — a touched user-visible surface with no passing check → blocked. The `touched_surfaces` list is a worker self-report; use `--strict-surfaces` or `--diff-base ` / `--touched` to derive it from the **real git diff** instead of trusting the worker.
4. **One canonical completion signal** (the gate's verdict); chat / PR / dashboard derive from it, never become an independent "complete".
5. **Artifact content is hostile data, not instructions** — deterministic checks first; an LLM verifier treats artifacts as untrusted.
6. **Hermetic execution** — the gate runs as `python3 -E` (ignores `PYTHON*` env / repo-planted `yaml.py`), and CI runs it from the trusted base branch so a PR can't edit the gate that judges it.
## Docs
- **Skills**: [`goal-compile`](skills/goal-compile/SKILL.md) (goal-first entry — say a goal, confirm criteria once), [`completion-audit`](skills/completion-audit/SKILL.md) (wrap-up: propose `candidate_complete` + run the gate), [`completion-gate-init`](skills/completion-gate-init/SKILL.md) (scaffold via the script).
- [`scripts/init.sh`](scripts/init.sh) — scaffold the gate into your project (the authoritative setup path).
- [`STATE_MACHINE.md`](STATE_MACHINE.md) — the completion contract (states, transitions, wiring).
- [`integrations/README.md`](integrations/README.md) — CI / agent-hook / pre-push wiring + the trust model.
- [`examples/`](examples/) — runnable: [`minimal-project/`](examples/minimal-project/) (everyday web task), `run.sh`, `diff_demo.sh`, `diff_rename_test.sh`, `swanlab/` (the ML incident).
- [`CHANGELOG.md`](CHANGELOG.md) · self-tests in [`tests/`](tests/).
## Status
`v0.4.1` preview. MIT. Agent-agnostic, file-based, fail-closed. Optional companion: [`agent-lessonbook`](https://github.com/zhjai/agent-lessonbook).