{"id":51151296,"url":"https://github.com/humanbased-ai/verifyflow","last_synced_at":"2026-06-26T06:04:17.390Z","repository":{"id":362247064,"uuid":"1256230126","full_name":"humanbased-ai/verifyflow","owner":"humanbased-ai","description":null,"archived":false,"fork":false,"pushed_at":"2026-06-03T10:00:51.000Z","size":164,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-03T10:17:43.508Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/humanbased-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-01T15:24:43.000Z","updated_at":"2026-06-03T10:01:04.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/humanbased-ai/verifyflow","commit_stats":null,"previous_names":["humanbased-ai/verifyflow"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/humanbased-ai/verifyflow","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanbased-ai%2Fverifyflow","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanbased-ai%2Fverifyflow/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanbased-ai%2Fverifyflow/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanbased-ai%2Fverifyflow/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/humanbased-ai","download_url":"https://codeload.github.com/humanbased-ai/verifyflow/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/humanbased-ai%2Fverifyflow/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34805109,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-26T02:00:06.560Z","response_time":106,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-26T06:04:16.546Z","updated_at":"2026-06-26T06:04:17.351Z","avatar_url":"https://github.com/humanbased-ai.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# VerifyFlow\n\nEvidence-backed delivery verification for Linear-driven pull requests.\n\nVerifyFlow answers one question:\n\n\u003e Did this PR actually deliver what the linked ticket asked for?\n\nIt is not code review. Crosscheck reviews code and merge readiness; VerifyFlow runs after that,\nexecutes the ticket acceptance criteria, captures evidence, and reports the delivery verdict.\n\n```text\nLinear issue + GitHub PR -\u003e execution evidence -\u003e delivery verdict\n```\n\nThe CLI is `verifyflow`, with the short alias `vf`.\n\n## Quickstart\n\nThe fastest way to see a full run — no credentials, no checkout, fully offline:\n\n```bash\nnpx github:humanbased-ai/verifyflow demo\n```\n\n`vf demo` runs bundled fixtures through the whole pipeline and writes a report you can read.\n\nThen check your environment and (optionally) get a guided setup:\n\n```bash\nvf doctor      # are gh / claude / LINEAR_API_KEY / Playwright / a sandbox runtime ready?\nvf onboard     # guided first-run setup; prints the exact fix command for anything missing\n```\n\n`vf onboard` can save your Linear key to `~/.verifyflow/credentials.json` (mode 0600). At runtime\n`LINEAR_API_KEY` is resolved from the environment first, then that credentials file — so you never\nhave to export it again.\n\n## Install\n\nAfter the first npm release:\n\n```bash\nnpm install -g @humanbased/verifyflow\nvf doctor\n```\n\nBefore npm publish, run straight from GitHub:\n\n```bash\nnpx github:humanbased-ai/verifyflow doctor\nnpx github:humanbased-ai/verifyflow run \\\n  --fixtures fixtures/example-cli \\\n  --linear EX-1 \\\n  --pr example/greet#7 \\\n  --level functional\n```\n\n## Verify a PR — `vf run`\n\n```bash\nvf run \\\n  --linear IN-123 \\\n  --pr humanbased-ai/monorepo#456 \\\n  --level auto \\\n  --checkout \\\n  --comment\n```\n\n- `--linear` is optional: if omitted, VerifyFlow derives the issue from the PR body's Linear link\n  or the branch name.\n- `--checkout` clones the repo and checks out the PR head for real execution. Or point at an\n  existing checkout with `--workdir \u003cdir\u003e`, or run offline with `--fixtures \u003cdir\u003e`.\n- `--comment` posts the markdown report as a PR comment (idempotent — updates in place).\n- `--linear-writeback` also posts the delivery verdict back to the linked Linear issue.\n\nPreview what VerifyFlow would do without checking out or executing anything (exits 0):\n\n```bash\nvf run --linear IN-123 --pr humanbased-ai/monorepo#456 --level auto --dry-run\n```\n\nVerify a PR that has no resolvable ticket, against its own description (verdict capped at\n`manual_review_required`):\n\n```bash\nvf run --pr humanbased-ai/monorepo#456 --allow-no-ticket\n```\n\n### Levels — `--level`\n\n| Level | What it does | Needs |\n| --- | --- | --- |\n| `functional` | Command/test probes against a checkout | checkout/workdir |\n| `ui` | AI-driven browser checks via Playwright | Playwright + a running app |\n| `journey` | Multi-step end-to-end (backend + browser) | checkout + Playwright |\n| `auto` | Picks the level from the ticket; downgrades to `functional` if no browser is available, and says why | — |\n\nFor `ui` / `journey`, point at the app with `--base-url \u003curl\u003e` (otherwise VerifyFlow tries to find a\ndeployment preview from the PR's checks), and supply an authenticated session with\n`--ui-auth \u003cstorageState.json\u003e` (create it with `playwright codegen --save-storage=auth.json`).\n\n### Merge policy — `--policy`\n\n| Policy | Behavior |\n| --- | --- |\n| `advisory` | Default. Reports only, never blocks. |\n| `merge_gate` | Exits non-zero on `needs_fix`, so a failing verdict blocks merge. |\n| `strict` | Also blocks on `manual_review_required` / `accept_with_risks`. |\n\n## Other commands\n\n| Command | What it does |\n| --- | --- |\n| `vf run` | Verify one PR against its Linear ticket (see above). |\n| `vf step` | Orchestrator-facing step (Symphony/Jazzband): advisory-only, auto-resolves the issue, checks out + executes + comments, and prints one machine-readable JSON line to stdout. Never blocks. |\n| `vf watch` | Independent daemon: watch a repo's Crosscheck-approved PRs, verify delivery, and (with `--auto-merge`) squash-merge on a clean `accept`. |\n| `vf report` | Aggregate accumulated runs into quality metrics; `--trend`, `--since`, `--repo`, `--level`, `--json` filters. |\n| `vf replay \u003crunId\u003e` | Re-run the verdict engine against a past run's stored evidence — no probes/tests re-execute. |\n| `vf show \u003crunId\u003e` | Re-render a past run's `report.md` (or `report.json`). |\n| `vf signal \u003crunId\u003e` | Pretty-print a past run's improvement-signal. |\n| `vf memory` | Inspect reusable test-point memory: `vf memory ls`, `vf memory show \u003ckey\u003e`, `vf memory clear [--repo \u003co/r\u003e] [--yes]`. |\n| `vf init` | Scaffold a `verifyflow.config.json` in the target repo (auto-detects npm / uv / go / cargo / make). |\n| `vf doctor` | Check that the tools/env VerifyFlow relies on are ready. |\n| `vf onboard` | Guided first-run setup; `--non-interactive` to skip prompts. |\n| `vf demo` | Offline demo with bundled fixtures; `--open` to open the report. |\n\nFull flag-by-flag reference: **[`docs/commands.html`](docs/commands.html)**. Run `vf --help` for the\nsame usage in the terminal.\n\nWatch a repo's Crosscheck-approved PRs:\n\n```bash\nvf watch --repo humanbased-ai/monorepo --interval 120              # monitor + verify + comment\nvf watch --repo humanbased-ai/monorepo --auto-merge --interval 120 # also squash-merge on accept\n```\n\n## What works today\n\n- One Linear issue against one GitHub PR.\n- Acceptance-criteria extraction from the Linear issue (or the PR description with `--allow-no-ticket`).\n- `functional`, `ui`, `journey`, and `auto` levels.\n- Real command/test execution against a checkout or workdir.\n- Browser-backed UI and journey checks through Playwright.\n- Markdown and JSON reports, screenshots, traces, command logs, and reusable test memory.\n- Idempotent PR comments and optional Linear writeback.\n- `advisory` / `merge_gate` / `strict` merge policies.\n- Standalone CLI, orchestration step, and Crosscheck-approved watcher modes.\n- Colorized terminal output in an interactive shell (auto-disabled when piped or under `NO_COLOR`).\n\nVerifyFlow is conservative: uncertainty becomes `blocked` or `not_evaluable`, not a fake product\nfailure.\n\n## Requirements\n\nVerifyFlow stores no secrets. It reuses local tools and environment variables.\n\n| Tool | Needed for | Required? |\n| --- | --- | --- |\n| Node.js \u003e= 20 | CLI runtime | required |\n| `gh` authenticated | GitHub PR context and comments | required |\n| `LINEAR_API_KEY` (env or `~/.verifyflow/credentials.json`) | Linear issue reads | required (or use `--fixtures`) |\n| `claude` authenticated | LLM planning and judging; otherwise rules-only fallback | optional |\n| `docker` or `podman` | Sandbox isolation for executing PR code (IN-555); without it, probes run on the host | optional |\n| Playwright | `ui` and browser-backed `journey` runs | optional |\n| `npm`, `uv`, etc. | Target repo setup and tests | as the target needs |\n\n## Roadmap\n\nNext major layer: project-level verification.\n\n```text\nLinear project -\u003e ticket/PR matrix -\u003e leveled runs -\u003e evidence bundle -\u003e project report\n```\n\nPlanned work:\n\n- Read a Linear project and map tickets to PRs.\n- Generate a coverage matrix across tickets, criteria, PRs, SHAs, and evidence.\n- Run per-ticket functional/UI/journey checks.\n- Produce one project-level implementation gap report.\n- Keep stronger sandbox isolation for untrusted PR execution.\n- Add opt-in Linear status transitions and follow-up issue filing.\n\n## Boundaries\n\nVerifyFlow does not:\n\n- review code or decide merge readiness; that is Crosscheck\n- decompose tickets or dispatch coding agents; that is Symphony today and Jazzband next\n- move money, send production email, or perform irreversible side effects\n- replace CI\n- provide native mobile evaluation yet\n\n## Toolchain fit\n\n```text\nSymphony / Jazzband -\u003e Crosscheck -\u003e VerifyFlow\norchestration          code review    delivery verification\n```\n\nSymphony is the current Python orchestration layer. Jazzband is the planned open TypeScript/npm\nsuccessor. VerifyFlow works with both, and also runs alone, by using public artifacts: GitHub PR\nmetadata, Linear issue links, SHA-bound comments, CLI JSON, and evidence files.\n\nMore detail:\n\n- [Command reference](docs/commands.html)\n- [End-to-end auto evaluation](docs/end-to-end-auto-evaluation.md)\n- [Coordination contract](docs/coordination-contract.md)\n- [Publishing](docs/publishing.md)\n- [UI level](docs/ui-level.md)\n\n## Development\n\n```bash\nnpm install\nnpm run typecheck\nnpm test\nnpm run build\nnpm pack --dry-run\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumanbased-ai%2Fverifyflow","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhumanbased-ai%2Fverifyflow","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhumanbased-ai%2Fverifyflow/lists"}