{"id":50609824,"url":"https://github.com/vitron-ai/alethia-anvil","last_synced_at":"2026-06-06T02:30:46.671Z","repository":{"id":351864688,"uuid":"1212677913","full_name":"vitron-ai/alethia-anvil","owner":"vitron-ai","description":"Alethia's reference app + Playwright/PW-MCP benchmark. Fork to evaluate Alethia against your own agent or reproduce the numbers.","archived":false,"fork":false,"pushed_at":"2026-04-29T01:22:57.000Z","size":156,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-29T02:35:26.230Z","etag":null,"topics":["agent-testing","alethia","benchmark","e2e-testing","mcp","playwright"],"latest_commit_sha":null,"homepage":"https://vitron.ai","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vitron-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-16T16:08:13.000Z","updated_at":"2026-04-29T01:23:01.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/vitron-ai/alethia-anvil","commit_stats":null,"previous_names":["vitron-ai/alethia-starter","vitron-ai/alethia-anvil"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/vitron-ai/alethia-anvil","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vitron-ai%2Falethia-anvil","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vitron-ai%2Falethia-anvil/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vitron-ai%2Falethia-anvil/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vitron-ai%2Falethia-anvil/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vitron-ai","download_url":"https://codeload.github.com/vitron-ai/alethia-anvil/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vitron-ai%2Falethia-anvil/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33967639,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-06T02:00:07.033Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-testing","alethia","benchmark","e2e-testing","mcp","playwright"],"created_at":"2026-06-06T02:30:45.999Z","updated_at":"2026-06-06T02:30:46.659Z","avatar_url":"https://github.com/vitron-ai.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# alethia-anvil\n\n[![alethia](https://github.com/vitron-ai/alethia-anvil/actions/workflows/alethia.yml/badge.svg?branch=main)](https://github.com/vitron-ai/alethia-anvil/actions/workflows/alethia.yml?query=branch%3Amain)\n[![alethia verified](https://img.shields.io/badge/alethia-verified-22c55e?logo=data:image/svg%2bxml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyNCAyNCIgZmlsbD0iI2ZmZiI+PGNpcmNsZSBjeD0iMTIiIGN5PSIxMiIgcj0iMTAiLz48L3N2Zz4=)](https://github.com/vitron-ai/alethia)\n[![npm](https://img.shields.io/npm/v/%40vitronai%2Falethia?label=%40vitronai%2Falethia\u0026color=3b82f6)](https://www.npmjs.com/package/@vitronai/alethia)\n[![license](https://img.shields.io/badge/license-MIT-blue)](./LICENSE)\n\nReference application + complete E2E test suite for [Alethia](https://www.npmjs.com/package/@vitronai/alethia). Fork **Anvil** to learn the agent-native testing pattern, adapt the 14 plain-English `.alethia` specs to your own app, and run them in CI. Also hosts the reproducible head-to-head Playwright / Playwright-MCP benchmark cited on [vitron.ai](https://vitron.ai).\n\nNo framework. No build step. Static HTML + 14 NLP test files, runs in any static server.\n\n---\n\n## Try it in 30 seconds\n\n```bash\ngit clone https://github.com/vitron-ai/alethia-anvil.git\ncd alethia-anvil\npython3 -m http.server 5173\n```\n\nOpen [http://127.0.0.1:5173](http://127.0.0.1:5173) — you should see **Anvil**, a tiny operations console. Sign in with anything (e.g. `alice@company.com` / `platform`).\n\n---\n\n## Run the tests with your agent\n\nInstall Alethia in your MCP client's config:\n\n```json\n{\n  \"mcpServers\": {\n    \"alethia\": { \"command\": \"alethia-mcp\" }\n  }\n}\n```\n\nThen tell your agent:\n\n\u003e Run the Alethia tests in `./__alethia__/` against the app at http://127.0.0.1:5173.\n\nThe agent will call `alethia_tell` once per test file and report pass/fail. Files are in `__alethia__/`:\n\n| File | What it tests |\n|---|---|\n| `smoke.alethia` | Sign-in screen renders |\n| `signin-flow.alethia` | Happy path — sign in, land on dashboard, nav is present |\n| `crud-flow.alethia` | Task list — add, assert it appears |\n| `edit-flow.alethia` | Inline editing — open modal, change title, save |\n| `search-flow.alethia` | Live-filter search box narrows the task list |\n| `priority-flow.alethia` | Priority dropdown updates a task + confirms via toast |\n| `toast-flow.alethia` | Transient toast appears, then auto-dismisses |\n| `count-flow.alethia` | Text-based counters stay consistent after add |\n| `tabs-flow.alethia` | Tab filter (All/Active/Done) narrows the list |\n| `bulk-flow.alethia` | Select-all + bulk complete across the visible tasks |\n| `export-flow.alethia` | Settings → Export Tasks emits JSON + success toast |\n| `shortcuts-flow.alethia` | Keyboard-shortcut legend renders in Settings |\n| `toast-stack-flow.alethia` | Multiple toasts stack newest-first without replacement |\n| `safety.alethia` | **EA1 policy verification** — `expect block:` on Delete Account |\n\n---\n\n## Running the tests in CI\n\nThis repo ships a GitHub Actions workflow at [`.github/workflows/alethia.yml`](./.github/workflows/alethia.yml) that:\n\n1. Starts Anvil on a static server\n2. Installs xvfb + Electron's Linux deps\n3. Runs [`__alethia__/ci-runner.mjs`](./__alethia__/ci-runner.mjs) — a tiny stdio MCP client that pipes every `.alethia` file through `@vitronai/alethia` and fails the build on the first red step\n\nTo run the same flow locally:\n\n```bash\npython3 -m http.server 5173 \u0026\nnode __alethia__/ci-runner.mjs\n```\n\nThe runner respects `ALETHIA_TARGET` if your app runs on a different port.\n\n---\n\n## Verify the \"faster than CDP-based tools\" claim yourself\n\nThe [`benchmark/`](./benchmark) directory is a reproducibility kit. Same Anvil app. Same three flows. Same machine. Numbers you can check.\n\n```bash\nnpm install --prefix benchmark\nnpm run playwright:install --prefix benchmark\nnpm install -g @vitronai/alethia@latest\n\npython3 -m http.server 5173 \u0026\nALETHIA_HEADLESS=1 node benchmark/compare.mjs --iterations 10\n```\n\nPrints a comparison table; writes `benchmark/results.json`. Takes ~3–5 min for N=10. Committed baseline numbers from a reference machine live in `benchmark/baseline.json`. Full methodology, flag list, and honest caveats in [`benchmark/README.md`](./benchmark/README.md).\n\nIf your numbers disagree with the baseline on comparable hardware, open an issue — we want the claim to be defensible, not marketing-absolute.\n\n### Reporter options\n\n```bash\nnode __alethia__/ci-runner.mjs --reporter pretty   # default — full summary, EA1 activity, slowest tests\nnode __alethia__/ci-runner.mjs --reporter plain    # one-line-per-test minimal output (good for logs)\nnode __alethia__/ci-runner.mjs --reporter json     # structured output for agent consumption\nnode __alethia__/ci-runner.mjs --only safety       # run a single test by name\n```\n\nSet `NO_COLOR=1` to disable ANSI colors. Status badges: `[PASS]` green, `[FAIL]` red, `[BLOCK]` blue (policy-verified via `expect block:`).\n\n---\n\n## What the tests exercise\n\nThe Anvil app is deliberately small but real enough to surface bugs that a static HTML demo wouldn't:\n\n- **Async operations** with loading spinners (sign-in, add-task, toggle, delete — each is a `setTimeout`-simulated round trip)\n- **Client-side form validation** with inline error messages\n- **SPA routing** via hash-based nav (back/forward, direct links to `/settings`)\n- **Destructive-action gating** — `Delete Account` is a write-high action that Alethia's EA1 policy gate should refuse by default\n\nIf any of those surfaces break under a future Alethia release, the tests in `__alethia__/` will catch it.\n\n---\n\n## `expect block:` — the verifiable-safety primitive\n\nThe `safety.alethia` file ends with:\n\n```\nexpect block: click Delete Account\n```\n\nThat line **passes** when the EA1 gate blocks the action, **fails** when the gate lets it through. It's how you regression-test your app's safety boundary itself — not just \"does my UI work\" but \"does the policy gate still catch the destructive action I wired to this button?\"\n\nNo other E2E framework can express this cleanly, because none have a policy gate to assert against. This is the v0.2 primitive.\n\n---\n\n## Adapting this to your own app\n\n1. Replace `index.html` with your app (or point the tests at your existing dev server).\n2. Update the NLP in `__alethia__/` to reference your own headings, buttons, and flows.\n3. Run the tests. Iterate.\n\nThat's it. No config, no test runner, no `.spec` files. Your agent reads the NLP, calls Alethia, reads back structured `nearMatches` / `suggestedFix` / `pageContext` fields, and self-corrects.\n\n---\n\n## Links\n\n- Alethia runtime: [github.com/vitron-ai/alethia](https://github.com/vitron-ai/alethia)\n- MCP bridge (npm): [npmjs.com/package/@vitronai/alethia](https://www.npmjs.com/package/@vitronai/alethia)\n- Tessl listing: [tessl.io/registry/vitron-ai/alethia](https://tessl.io/registry/vitron-ai/alethia)\n- Website: [vitron.ai](https://vitron.ai)\n\n## Security posture — local-only by architecture\n\nAlethia is a **local-only** runtime by design. The signed binary refuses to navigate to any origin outside the local allowlist (`file://`, `localhost`, `127.0.0.1`, `.local`, RFC1918 ranges). This is a compile-time constant enforced in four places inside the runtime — it is **not** a CLI flag, env var, MCP argument, profile, or UI toggle.\n\nWhy: Alethia's zero-IPC execution would make it an ideal credential-stuffing / account-takeover / abuse-automation tool if turned against the open web. It will not become that tool. For partner-specific production-origin access, contact **team@vitron.ai** — we issue custom-signed builds; we do not ship configurability.\n\nFull posture at [vitron.ai/safety](https://vitron.ai/safety).\n\n## License\n\nThe starter code (this repo) is MIT. The Alethia runtime it's testing is patent-pending and governed by a separate license — see [github.com/vitron-ai/alethia](https://github.com/vitron-ai/alethia).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvitron-ai%2Falethia-anvil","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvitron-ai%2Falethia-anvil","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvitron-ai%2Falethia-anvil/lists"}