{"id":35019169,"url":"https://github.com/rul1an/assay","last_synced_at":"2026-05-26T00:01:23.277Z","repository":{"id":330131340,"uuid":"1120300697","full_name":"Rul1an/assay","owner":"Rul1an","description":"CI-native evidence compiler for agent systems: MCP policy enforcement, evidence receipts, Trust Basis claims, and reviewable artifacts.","archived":false,"fork":false,"pushed_at":"2026-05-23T20:52:32.000Z","size":220168,"stargazers_count":2,"open_issues_count":1,"forks_count":2,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-23T21:25:28.552Z","etag":null,"topics":["agent-security","ai-agents","ai-security","ci","cyclonedx","evidence","evidence-bundles","evidence-receipts","github-actions","mcp","mcp-server","openfeature","policy-as-code","policy-enforcement","promptfoo","provenance","rust","sbom","supply-chain-security","trust-basis"],"latest_commit_sha":null,"homepage":"https://getassay.dev","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Rul1an.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":"docs/ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-20T22:56:24.000Z","updated_at":"2026-05-23T20:51:23.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/Rul1an/assay","commit_stats":null,"previous_names":["rul1an/verdict"],"tags_count":116,"template":false,"template_full_name":null,"purl":"pkg:github/Rul1an/assay","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rul1an%2Fassay","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rul1an%2Fassay/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rul1an%2Fassay/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rul1an%2Fassay/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Rul1an","download_url":"https://codeload.github.com/Rul1an/assay/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Rul1an%2Fassay/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33497930,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-25T14:31:05.219Z","status":"ssl_error","status_checked_at":"2026-05-25T14:31:02.878Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-security","ai-agents","ai-security","ci","cyclonedx","evidence","evidence-bundles","evidence-receipts","github-actions","mcp","mcp-server","openfeature","policy-as-code","policy-enforcement","promptfoo","provenance","rust","sbom","supply-chain-security","trust-basis"],"created_at":"2025-12-27T05:40:14.325Z","updated_at":"2026-05-26T00:01:23.231Z","avatar_url":"https://github.com/Rul1an.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003ch1 align=\"center\"\u003eAssay\u003c/h1\u003e\n  \u003cp align=\"center\"\u003e\n    \u003cstrong\u003eEvidence compiler for agent review artifacts\u003c/strong\u003e\u003cbr /\u003e\n    \u003cspan\u003ePortable evidence receipts, verifiable bundles, and bounded Trust Basis claims for agent systems.\u003c/span\u003e\n  \u003c/p\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"https://crates.io/crates/assay-cli\"\u003e\u003cimg src=\"https://img.shields.io/crates/v/assay-cli.svg\" alt=\"Crates.io\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/Rul1an/assay/actions/workflows/ci.yml\"\u003e\u003cimg src=\"https://github.com/Rul1an/assay/actions/workflows/ci.yml/badge.svg\" alt=\"CI\"\u003e\u003c/a\u003e\n    \u003ca href=\"https://github.com/Rul1an/assay/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/crates/l/assay-core.svg\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003c/p\u003e\n  \u003cp align=\"center\"\u003e\n    \u003ca href=\"#see-it-work\"\u003eSee It Work\u003c/a\u003e ·\n    \u003ca href=\"docs/use-cases/evidence-receipts-from-promptfoo-jsonl.md\"\u003ePromptfoo JSONL\u003c/a\u003e ·\n    \u003ca href=\"docs/use-cases/openfeature-evaluationdetails-to-ci-review-artifact.md\"\u003eOpenFeature\u003c/a\u003e ·\n    \u003ca href=\"docs/use-cases/cyclonedx-mlbom-model-to-inventory-receipt.md\"\u003eCycloneDX ML-BOM\u003c/a\u003e ·\n    \u003ca href=\"examples/mcp-quickstart/\"\u003eQuick Start\u003c/a\u003e ·\n    \u003ca href=\"docs/guides/github-action.md\"\u003eCI Guide\u003c/a\u003e ·\n    \u003ca href=\"https://github.com/Rul1an/assay/discussions\"\u003eDiscussions\u003c/a\u003e\n  \u003c/p\u003e\n\u003c/p\u003e\n\n---\n\nUse Assay if you already have machine-readable AI outcomes or agent tool-call tests and want a small reviewable artifact boundary in CI.\n\nStart with the path that matches what you already have:\n\n| You have | Use this when | What you get | Next click |\n|---|---|---|---|\n| Promptfoo JSONL from CI evals | You want smaller PR evidence than a full eval export | Eval outcome receipts, verified bundle, Trust Basis diff | [Promptfoo JSONL](docs/use-cases/evidence-receipts-from-promptfoo-jsonl.md) |\n| OpenFeature boolean `EvaluationDetails` | You want CI evidence for a runtime flag decision boundary | Decision receipt, verified bundle, Trust Basis diff | [OpenFeature EvaluationDetails](docs/use-cases/openfeature-evaluationdetails-to-ci-review-artifact.md) |\n| CycloneDX ML-BOM model component | You want CI evidence for the model inventory/provenance boundary that existed | Inventory receipt, verified bundle, Trust Basis diff | [CycloneDX ML-BOM](docs/use-cases/cyclonedx-mlbom-model-to-inventory-receipt.md) |\n| MCP tool calls | You are ready to put a policy file around tool execution | Allow/deny audit trail and evidence for observed tool behavior | [MCP Quick Start](examples/mcp-quickstart/) |\n| A GitHub PR gate | You want CI to block regressions from checked artifacts | Trust Basis diff, gate status, SARIF/JUnit-ready output | [CI Guide](docs/guides/github-action.md) |\n\nThe core workflow is intentionally small: import or record a bounded outcome, bundle and verify it, compile `trust-basis.json`, then gate the Trust Basis diff. Assay does not make the upstream tool the source of truth; it makes the evidence boundary inspectable.\n\n```text\nTrust Basis Gate\nStatus: OK\nBundles verified: 1\nRegressed claims: 0\n```\n\nAssay is not a trust-score engine, a generic eval dashboard, or a hosted observability product. See [What Assay is and is not](docs/concepts/scope.md) for the boundary.\n\n## Is This For Me?\n\n**Yes, if you:**\n- already have eval output, runtime decisions, inventory artifacts, or MCP tool-call tests\n- want a CI review artifact instead of a dashboard-only result\n- need bounded auditability, not a scalar trust badge\n\n**Not yet, if you:**\n- need Assay to judge model correctness or policy quality for you\n- want a hosted dashboard as the primary product\n- want a compliance claim instead of a bounded evidence boundary\n\n## Install\n\n```bash\ncargo install assay-cli\n```\n\nCI: [GitHub Action](https://github.com/marketplace/actions/assay-ai-agent-security). Python SDK: `pip install assay-it`.\n\nNo hosted backend. No API keys for core flows. Deterministic: same input, same decision.\n\n\u003cdetails\u003e\n\u003csummary\u003eEvidence levels and non-goals\u003c/summary\u003e\n\nTrust claims use explicit **epistemology**, not a single “safety score”:\n\n| Level | Meaning |\n|-------|---------|\n| `verified` | Backed by direct evidence or offline verification in the bundle/path |\n| `self_reported` | Emitted by the system without stronger independent corroboration |\n| `inferred` | Derived from bounded, documented rules |\n| `absent` | No trustworthy evidence supports the claim |\n\nAssay does **not** ship a primary aggregate trust score or a `safe/unsafe` badge as the main output. See [ADR-033](docs/architecture/ADR-033-OTel-Trust-Compiler-Positioning.md).\n\n\u003c/details\u003e\n\n## What ships today\n\n| Output | Role |\n|--------|------|\n| **Policy gate** | MCP `wrap` — deterministic allow/deny before tools run (see CLI note below the diagram). |\n| **Evidence bundle** | Offline-verifiable, tamper-evident archive for audit and replay. |\n| **External receipts** | Selected eval outcomes, runtime decision details, and inventory/provenance surfaces as bounded evidence receipts with JSON Schema contracts. |\n| **Trust Basis** | Canonical `trust-basis.json` — bounded claim classification from verified bundles. |\n| **Trust Card** | `trustcard.json` / `trustcard.md` / `trustcard.html` — same claims, review-friendly artifacts. |\n| **SARIF / CI** | GitHub Action, Security tab integration, policy gates on PRs. |\n\n\u003e **Repository truth:** release notes and [CHANGELOG.md](CHANGELOG.md) remain the authority for what is actually public. `main` may carry release-prep commits before a tag is cut; crates.io publication is separate from repository merge state.\n\n```\n  Agent ──► Assay ──► MCP Server\n              │\n              ├─ ✅ ALLOW / ❌ DENY  (policy)\n              ├─► 📋 Evidence bundle (verifiable)\n              └─► 📊 Trust Basis → Trust Card → SARIF / CI\n```\n\n\u003e **CLI:** The `mcp` command group is **hidden** from top-level `assay --help` while the surface stabilizes; it is supported. Use `assay mcp --help`, `assay mcp wrap …`, or follow the [MCP Quickstart](examples/mcp-quickstart/).\n\n\u003e **Wedge, not category.** “MCP firewall” describes the control plane; **trust compilation** describes the outcome: reviewable claims backed by evidence. See [ADR-033](docs/architecture/ADR-033-OTel-Trust-Compiler-Positioning.md) and [RFC-005](docs/architecture/RFC-005-trust-compiler-mvp-2026q2.md).\n\n## See It Work\n\n[![SafeSkill 72/100](https://img.shields.io/badge/SafeSkill-72%2F100_Passes%20with%20Notes-yellow)](https://safeskill.dev/scan/rul1an-assay)\n\n```bash\ncargo install assay-cli\n\nmkdir -p /tmp/assay-demo \u0026\u0026 echo \"safe content\" \u003e /tmp/assay-demo/safe.txt\n\nassay mcp wrap --policy examples/mcp-quickstart/policy.yaml \\\n  -- npx @modelcontextprotocol/server-filesystem /tmp/assay-demo\n```\n\n```\n✅ ALLOW  read_file  path=/tmp/assay-demo/safe.txt  reason=policy_allow\n✅ ALLOW  list_dir   path=/tmp/assay-demo/           reason=policy_allow\n❌ DENY   read_file  path=/tmp/outside-demo.txt      reason=path_constraint_violation\n❌ DENY   exec       cmd=ls                          reason=tool_denied\n```\n\nInspect the audit artifact:\n\n```bash\nassay evidence show demo/fixtures/bundle.tar.gz\n```\n\n![Evidence Bundle Inspector](demo/output/screenshots/evidence-bundle-inspector.svg)\n\nThe bundle is tamper-evident and cryptographically verifiable. Signed mandate events can include an Ed25519-backed authorization trail for high-risk actions.\n\n### Trust artifacts from a verified bundle\n\nAfter a bundle verifies, compile the claim artifact:\n\n```bash\n# Machine-readable claim basis (deterministic, claim-first)\nassay trust-basis generate demo/fixtures/bundle.tar.gz \u003e trust-basis.json\n```\n\n`trust-basis.json` is the canonical output for CI and review. Claim `id` values are stable across runs; consumers should key by `id`, not row count or order. It is not a scalar trust score.\n\nThe current claim-visible receipt families are Promptfoo assertion-component results, OpenFeature boolean `EvaluationDetails`, and CycloneDX ML-BOM model components. See the [receipt-family matrix](docs/reference/receipt-family-matrix.json), the [three-family note](docs/notes/EVIDENCE-RECEIPTS-FOR-AI-OUTCOMES-RUNTIME-DECISIONS-MODEL-INVENTORY.md), and [Evidence Receipts in Action](docs/notes/EVIDENCE-RECEIPTS-IN-ACTION.md).\n\n\u003cdetails\u003e\n\u003csummary\u003eTrust Card details\u003c/summary\u003e\n\n```bash\nassay trustcard generate demo/fixtures/bundle.tar.gz --out-dir ./trust-out\n# -\u003e trust-out/trustcard.json , trust-out/trustcard.md , trust-out/trustcard.html\n```\n\nThe Trust Card is a deterministic render of the same claim rows plus frozen non-goals; `trustcard.json` is canonical, while Markdown and static HTML are reviewer projections. Contract versions, pack floors, and release checklist: [MIGRATION — Trust Compiler 3.2](docs/architecture/MIGRATION-TRUST-COMPILER-3.2.md), [receipt-family matrix](docs/reference/receipt-family-matrix.json). Release history belongs in [CHANGELOG.md](CHANGELOG.md).\n\n\u003c/details\u003e\n\n## Add to Cursor in 30 Seconds\n\nAssay ships a helper that finds your local Cursor MCP config path and prints a ready-to-paste entry:\n\n```bash\nassay mcp config-path cursor\n```\n\nIt generates JSON like:\n\n```json\n{\n  \"filesystem-secure\": {\n    \"command\": \"assay\",\n    \"args\": [\n      \"mcp\",\n      \"wrap\",\n      \"--policy\",\n      \"/path/to/policy.yaml\",\n      \"--\",\n      \"npx\",\n      \"-y\",\n      \"@modelcontextprotocol/server-filesystem\",\n      \"/Users/you\"\n    ]\n  }\n}\n```\n\nThe same wrapped command works in other MCP clients — see [MCP Quick Start](docs/mcp/quickstart.md).\n\n## Policy Is Simple\n\n```yaml\nversion: \"2.0\"\nname: \"my-policy\"\n\ntools:\n  allow: [\"read_file\", \"list_dir\"]\n  deny: [\"exec\", \"shell\", \"write_file\"]\n\nschemas:\n  read_file:\n    type: object\n    additionalProperties: false\n    properties:\n      path:\n        type: string\n        pattern: \"^/app/.*\"\n        minLength: 1\n    required: [\"path\"]\n```\n\nLegacy `constraints:` policies still work. Use `assay policy migrate` for the v2 JSON Schema form, or `assay init --from-trace trace.jsonl` to generate from observed behavior.\n\nSee [Policy Files](docs/reference/config/policies.md).\n\n\u003cdetails\u003e\n\u003csummary\u003eOther import paths and protocol adapters\u003c/summary\u003e\n\n### OpenTelemetry in, canonical evidence out\n\nAssay ingests OpenTelemetry JSONL, builds replayable traces, and exports **canonical evidence** — OTel is a bridge, not the sole semantic authority.\n\n```bash\nassay trace ingest-otel \\\n  --input otel-export.jsonl \\\n  --db .eval/eval.db \\\n  --out-trace traces/otel.v2.jsonl\n```\n\nSee [OpenTelemetry \u0026 Langfuse](docs/guides/otel-langfuse.md).\n\n### Protocol adapters\n\nAssay ships adapters that map protocol events into **canonical evidence**:\n\n| Protocol | Adapter | What it maps |\n|----------|---------|--------------|\n| **ACP** (OpenAI/Stripe) | `assay-adapter-acp` | Checkout events, payment intents, tool calls |\n| **A2A** (Google) | `assay-adapter-a2a` | Agent capabilities, task delegation, artifacts |\n| **UCP** (Google/Shopify) | `assay-adapter-ucp` | Discover/buy/post-purchase state transitions |\n\nAdapter crates are workspace / binary-driven, not published as separate `crates.io` packages.\n\n\u003c/details\u003e\n\n## Add to CI\n\n```yaml\n# .github/workflows/assay.yml\nname: Assay Gate\non: [push, pull_request]\npermissions:\n  contents: read\n  security-events: write\njobs:\n  assay:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n      - uses: Rul1an/assay-action@v2\n```\n\nPRs that violate policy get blocked; SARIF can surface in the Security tab.\n\n## Why Assay\n\n| | |\n|---|---|\n| **Canonical evidence** | Assay’s evidence model is the stable contract; OTel and adapters map into it. |\n| **Deterministic** | Same input, same decision — not probabilistic. |\n| **Portable artifacts** | Bundles, Trust Basis, Trust Card, SARIF — for CI, review, audit. |\n| **Bounded claims** | Explicit about what is **verified** vs **visible** vs **absent** — no score-first UX. |\n| **MCP-native wedge** | `assay mcp wrap` is the fast path (the `mcp` group is hidden from `assay --help`; use `assay mcp --help`). Adapters extend the same engine. |\n| **Offline-first** | No backend required for core enforcement and bundle verification. |\n\n\u003cdetails\u003e\n\u003csummary\u003eMeasured latency\u003c/summary\u003e\n\nOn the M1 Pro/macOS fragmented-IPI harness, protected tool-decision path:\n\n- Main protection run: `0.771ms` p50 / `1.913ms` p95\n- Fast-path scenario: `0.345ms` p50 / `1.145ms` p95\n\nThese are tool-decision timings, not end-to-end model latency. (See [Research \u0026 experiments](#research-mappings-experiments) for methodology context.)\n\n\u003c/details\u003e\n\n## Learn More\n\n- [Promptfoo JSONL to Evidence Receipts](docs/use-cases/evidence-receipts-from-promptfoo-jsonl.md) — smallest adoption path for existing eval artifacts\n- [OpenFeature EvaluationDetails to CI Review Artifact](docs/use-cases/openfeature-evaluationdetails-to-ci-review-artifact.md) — runtime decision receipt path\n- [CycloneDX ML-BOM Model to Inventory Receipt](docs/use-cases/cyclonedx-mlbom-model-to-inventory-receipt.md) — model inventory/provenance receipt path\n- [MCP Quickstart](examples/mcp-quickstart/) — filesystem server walkthrough\n- [Policy Files](docs/reference/config/policies.md) — YAML schema for `assay mcp wrap`\n- [OpenTelemetry \u0026 Langfuse](docs/guides/otel-langfuse.md) — traces → replay and evidence\n- [CI Guide](docs/guides/github-action.md) — GitHub Action\n- [Evidence Store](docs/guides/evidence-store-aws-s3.md) — S3, B2, MinIO\n- [ADR-033: Trust compiler positioning](docs/architecture/ADR-033-OTel-Trust-Compiler-Positioning.md)\n- [RFC-005: Trust compiler MVP \u0026 Trust Card](docs/architecture/RFC-005-trust-compiler-mvp-2026q2.md)\n\n## Internal: Assay-Runner\n\nAssay-Runner is an internal measured-run subsystem used by Assay's delegated Linux/eBPF acceptance path. It is **not a standalone product**. As of Phase 2D, the runner candidate is split into extraction-ready Rust crates (`assay-runner-schema`, `assay-runner-core`, `assay-runner-linux`) — all `publish = false` — plus the `runner-fixtures/` package tree (Node fixture marked `\"private\": true`; Python fixture has no distribution surface). Everything stays inside this repository.\n\n- [Assay-Runner reference index](docs/reference/runner/index.md) — internal contracts, boundary map, slice history\n- [Measured-run proof-bundle walkthrough](docs/reference/runner/examples/measured-run-proof-bundle.md) — read-only walkthrough for maintainers evaluating standalone use cases\n- [Phase 2D consolidation audit](docs/reference/runner/phase-2d-consolidation-audit.md) — current burn-in criteria; the extraction question is closed until the criteria are observed and at least one concrete external use case appears\n\nNo release commitment. No timeline. No external demand has been measured.\n\n## Research, mappings \u0026 experiments\n\n**Bounded context:** numbers below support **mapping and experiments**, not a product “security score.”\n\n- [OWASP MCP Top 10 Mapping](docs/security/OWASP-MCP-TOP10-MAPPING.md) — how Assay relates to each risk category (coverage is **not** a scalar guarantee).\n- Third-party survey: popular MCP servers often show weak defaults — Assay adds policy + evidence; see discussion in the mapping doc.\n- [Security experiments](docs/architecture/SYNTHESIS-TRUST-CHAIN-TRIFECTA-2026q2.md) — attack vectors and harness notes (methodology matters more than headline counts).\n\n## Contributing\n\n```bash\ncargo test --workspace\ncargo clippy --workspace --all-targets -- -D warnings\n```\n\nSee [CONTRIBUTING.md](CONTRIBUTING.md). **Discussions:** [GitHub Discussions](https://github.com/Rul1an/assay/discussions) — seed topics for pinned threads live in [docs/community/DISCUSSIONS.md](docs/community/DISCUSSIONS.md).\n\n## License\n\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frul1an%2Fassay","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frul1an%2Fassay","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frul1an%2Fassay/lists"}