{"id":50755833,"url":"https://github.com/blasrodri/truth","last_synced_at":"2026-06-11T05:00:50.332Z","repository":{"id":363252278,"uuid":"1262511060","full_name":"blasrodri/truth","owner":"blasrodri","description":"Deterministic fact-checker for AI coding agents — verify an agent's claims about its own work against the real code, git diff \u0026 logs. Local MCP server, cited verdicts, refuses to bluff.","archived":false,"fork":false,"pushed_at":"2026-06-08T04:12:40.000Z","size":271,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-08T06:13:59.592Z","etag":null,"topics":["agent-tools","ai-agents","ai-coding-assistant","claude","claude-code","coding-agent","developer-tools","fact-checking","hallucination","mcp","mcp-server","model-context-protocol","rust","static-analysis"],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/blasrodri.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-MIT","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-08T03:59:28.000Z","updated_at":"2026-06-08T04:12:30.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/blasrodri/truth","commit_stats":null,"previous_names":["blasrodri/truth"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/blasrodri/truth","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blasrodri%2Ftruth","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blasrodri%2Ftruth/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blasrodri%2Ftruth/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blasrodri%2Ftruth/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/blasrodri","download_url":"https://codeload.github.com/blasrodri/truth/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/blasrodri%2Ftruth/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34183109,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-11T02:00:06.485Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent-tools","ai-agents","ai-coding-assistant","claude","claude-code","coding-agent","developer-tools","fact-checking","hallucination","mcp","mcp-server","model-context-protocol","rust","static-analysis"],"created_at":"2026-06-11T05:00:27.438Z","updated_at":"2026-06-11T05:00:50.326Z","avatar_url":"https://github.com/blasrodri.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# truth\n\n**`truth` is a deterministic fact-checker for the claims an AI coding agent makes about its own work.**\n\nWhen an agent says *\"I added the `/v1/refund` route, set `MAX_RETRIES` to 5, only touched the parser, and tests pass,\"* `truth` checks each claim against the **real code, the working-tree git diff, recorded command runs, and logs**, and answers **Supported / Contradicted / Refused** — every verdict cited.\n\nIt is not a chatbot and it does not decide truth with a model. A language model only parses the agent's sentence into a structured claim; a fixed-rule engine decides the verdict from retrieved evidence. **The agent cannot talk it into a different answer.**\n\n```\n$ truth verify-turn \"I added the /v1/refund endpoint, set MAX_RETRIES to 3, I only changed src/api.rs, and tests pass\"\n\n  ✓ Supported     I added the /v1/refund endpoint  (src/api.rs)\n  ✗ Contradicted  set MAX_RETRIES to 3             (src/config.rs:1)\n  ✗ Contradicted  I only changed src/api.rs        (src/api.rs)\n  ✗ Contradicted  tests pass                       (recorded 2026-06-11 09:14 UTC)\n\n  1 supported · 3 contradicted · 0 refused\n\n  ⚠ The agent's message contradicts the evidence above.\n```\n\n## Why\n\nCoding agents over-claim. They report success inferred from clean logs, invent\nterminal output, and confidently describe changes they didn't make. `truth`\ncatches the **checkable** subset — wrong config values, routes/functions\nclaimed added or removed, files claimed edited, \"I only changed X\" scope\nclaims, renames, and *\"tests pass\"* (against recorded runs) — and **refuses**\nthe rest (*\"this is cleaner\"*) instead of guessing. A refusal is honest, not a\ngap: a verifier that bluffs is worse than none.\n\nEverything runs **locally**. The store is a single SQLite file in `.truth/`;\nraw logs are never persisted (only redacted aggregates); your code never leaves\nthe machine. No LLM, network, or account is required.\n\n## Install\n\n**Prebuilt binaries** (no Rust toolchain needed) — grab the tarball for your\nplatform from the [latest release](https://github.com/blasrodri/truth/releases/latest)\n(macOS arm64/x64, Linux x64/arm64), then:\n\n```bash\ntar -xzf truth-*.tar.gz\ninstall truth-*/truth truth-*/truth-mcp /usr/local/bin/\n```\n\n**Or build from source:**\n\n```bash\ncargo build --release --workspace\ninstall target/release/truth target/release/truth-mcp /usr/local/bin/\n# binaries: truth (CLI) and truth-mcp (the MCP server)\n```\n\n## Use it from your coding agent (MCP)\n\n`truth-mcp` is a local [Model Context Protocol](https://modelcontextprotocol.io)\nserver (stdio JSON-RPC). It exposes one tool, **`verify_turn`**, that an agent\ncalls on **its own** message before telling you a change is done — so the agent\ncatches and corrects its own lies first.\n\nThe repo ships a [`server.json`](server.json) (MCP registry manifest) and a\n[`.mcp.json`](.mcp.json), so cloning it auto-registers the server in MCP-aware\nclients (subject to their approval prompt).\n\n### Claude Code\n\n```bash\nclaude mcp add --scope user truth -- /usr/local/bin/truth-mcp\nclaude mcp list          # verify it connected\n```\n\n### Cursor, or any MCP client (generic config)\n\nAdd to `~/.cursor/mcp.json` (Cursor) or your client's `mcpServers` block:\n\n```json\n{\n  \"mcpServers\": {\n    \"truth\": {\n      \"type\": \"stdio\",\n      \"command\": \"/usr/local/bin/truth-mcp\"\n    }\n  }\n}\n```\n\nThat's the whole config — **no per-repo setup in the MCP file**. The agent\npasses the repo path with each call (the `repo` argument below), so one\nregistration works across all your projects.\n\n### The `verify_turn` tool\n\n| Argument | Required | Meaning |\n|---|---|---|\n| `message` | yes | The agent's raw prose about its work. truth scans it as a backstop, so claims you forget to list in `claims` still get checked. |\n| `claims` | recommended | An array of the individual factual claims, each a short self-contained sentence (`[\"I set MAX_RETRIES to 5\", \"I added the /v1/refund route\"]`). The **calling model extracts these from its own message** — it's free (the agent is already mid-turn) and far more reliable than truth re-parsing prose. truth still decides each verdict from real evidence, not from the wording. |\n| `repo` | recommended | Absolute path to the repo root. `truth` opens `\u003crepo\u003e/.truth` and diffs that working tree. Omit it and the server falls back to its own working directory, which may be wrong. |\n| `local_log` | no | Path to a local log file for usage/error claims. |\n\n\u003e **Why `claims` is the elegant path:** truth keeps the hybrid architecture —\n\u003e the LLM *parses*, the deterministic engine *decides*. By having the agent\n\u003e (already an LLM, already mid-turn) extract its own claims, truth sidesteps its\n\u003e regex parser entirely for ~tens of tokens, while still never letting a model\n\u003e decide truth. Omissions are caught by the `message` backstop.\n\n**Guaranteeing the agent uses `claims`.** The tool description asks for it, but\nto make it a habit in your own repos, add a line to your agent's project\ninstructions (e.g. `CLAUDE.md`):\n\n```md\nBefore telling me a code change is done, call the `verify_turn` tool: extract\nyour concrete factual claims (files edited, values set, routes/functions added\nor removed, renames, \"only changed X\", \"tests pass\") into the `claims` array\nand pass the repo path. Run tests through `truth run -- \u003ccmd\u003e` so \"tests pass\"\nis checkable. Fix anything it marks `contradicted` before reporting done.\n```\n\nOr skip the honor system entirely: `truth hook install` (next section) makes\nverification run on every turn whether the agent calls the tool or not.\n\nIt returns the verdict table as text plus `structuredContent` (the JSON below),\nincluding an `index` block reporting whether the index is empty or stale — so a\n\"clean\" result is never trusted blindly.\n\n\u003e **One-time per repo:** run `truth init` once to create the `.truth/` store.\n\u003e After that, `verify_turn` **auto-refreshes the index** on every call —\n\u003e incrementally, skipping unchanged files (~10–50 ms), so code-existence /\n\u003e usage / config claims always reflect the current working tree. You never have\n\u003e to re-run `truth index` by hand. Claims about the **working-tree diff**\n\u003e (\"I added/removed X\") need no index at all. If the index still ends up empty\n\u003e (e.g. nothing indexable), `verify_turn` says so loudly instead of passing.\n\n## Make it a gate the agent can't skip (hooks)\n\nThe MCP tool relies on the agent *choosing* to verify itself. Hooks remove the\nchoice:\n\n```bash\ntruth hook install          # project .claude/settings.json (--user for global)\n```\n\nThis registers two Claude Code hooks:\n\n- **Stop** — when the agent finishes its turn, its final message is\n  fact-checked against the repo, the working-tree diff, and recorded runs.\n  Contradictions **block the stop** and feed the cited verdict back, so the\n  agent corrects itself before you ever read the claim.\n- **PostToolUse (Bash)** — test/build/lint commands the agent runs are\n  recorded as command receipts automatically, which is what makes its later\n  *\"tests pass\"* checkable. (Receipts are only recorded when the hook payload\n  carries a real exit code — never guessed.)\n\nBoth hooks are fail-open: if truth errors, the session is never wedged.\n\nFor pull requests, [`examples/github/pr-factcheck.yml`](examples/github/pr-factcheck.yml)\nis a drop-in GitHub workflow that fact-checks a PR's **description against its\nactual diff** and posts the verdict table as a comment — agent-written PRs get\nchecked before any human reads them.\n\n## The lie ledger\n\nEvery check is already stored as an audit trail; `truth stats` reads it back:\n\n```\n$ truth stats --window 7d\n\n  claims checked      142\n  supported            96 (68%)\n  contradicted          9 (6%)\n  refused              37 (26%)\n  runs recorded        12 (10 green, 2 failing)\n\n  contradictions by claim type:\n    config_value   4\n    file_changed   3\n```\n\n`truth stats --all` aggregates across every repo you've run `truth init` in\n(via `~/.truth/registry.json` — the stores themselves stay per-repo).\n\n## Use it yourself (CLI)\n\n```bash\ncd your-repo\ntruth init                      # writes truth.toml + .truth/, runs migrations\ntruth index .                   # index code/docs/config (re-run after big changes)\n\ntruth verify-turn \"I added /v1/refund, set MAX_RETRIES to 5, removed /v1/checkout\"\ntruth verify-turn \"\u003cagent message\u003e\" --repo /path/to/repo --json\n\ntruth run -- cargo test         # run + record a receipt; \"tests pass\" becomes checkable\ntruth stats                     # the lie ledger\n```\n\n`--repo` opens that repo's `.truth` store and diffs that tree. Without it,\ntruth walks up from the current directory to the nearest `truth.toml`/`.truth`\nroot (like git does), so every command works from any subdirectory. `--json`\nemits stable machine-readable output.\n\n## What it can and cannot check\n\n**Checks (state claims):** route added/removed/exists, **function/symbol\nadded/removed/exists**, config value, named constant (\"changed X from 3 to 5\"\nchecks the *5*), retry count, timeout value, env var present, dependency used,\nversion required, usage count, error-still-happening, job-last-success,\nfeature-flag enabled — across **Rust / TypeScript / Python / Go** — against\n**code + git diff + logs**, with the diff outranking a possibly-stale index\nfor \"I just changed X\" claims.\n\n**Checks (diff claims — what THIS turn changed):** *\"I edited/created/deleted\n`src/auth.rs`\"* (the diff's file list decides), *\"I **only** changed the\nparser\"* (catches collateral edits — every changed path must match), *\"renamed\n`parse_legacy` to `parse_v2`\"* (old name must be gone AND the new one added),\n*\"updated all 4 call sites of X\"* (changed-line count). An empty diff reports\n**unknown (already committed?)** — never a free pass.\n\n**Checks (command receipts):** *\"tests pass\"*, *\"it compiles\"*, *\"clippy is\nclean\"* — verified against runs recorded by `truth run -- cargo test` (or the\nClaude Code hook below). Supported **only** when a matching run exited 0\n**after** your last working-tree edit; a failing receipt contradicts the\nclaim; a green-but-stale receipt proves nothing and is refused.\n\n**Refuses (by design):** action claims with no receipt (*\"I ran the tests\"* —\nrecord runs and it becomes checkable), and judgment claims (*\"this is cleaner\n/ faster\"* — no measurable subject). Refused ≠ confirmed.\n\n## Configuration\n\nBehavior lives in `truth.toml` (written by `truth init`). Tweak it without\nhand-editing — `truth settings` validates and preserves the rest of the file,\nso a user *or an agent* can change knobs programmatically:\n\n```bash\ntruth settings list                              # every knob, current value, help\ntruth settings set indexer.extractor mixed       # turn on AST precision (symbols/routes)\ntruth settings set repo.include src,lib,app      # what to index\ntruth settings set llm.enabled true              # use an LLM to parse claims (engine still decides)\ntruth settings get indexer.extractor --json\n```\n\nThe highest-value knob is **`indexer.extractor`**: `mixed` (the default —\nAST-precise function/struct/route definitions for Rust, TypeScript/JavaScript,\nPython, and Go, so a symbol named only in a comment isn't mistaken for a real\ndefinition; regex fills in everywhere else) · `ast` · `regex` (fastest,\nlanguage-agnostic, noisier). Re-run `truth index .` after changing it.\n\n## How it works\n\n```\nagent message\n  → segment into candidate claims (sentences, clauses)\n  → claim extraction (regex by default; optional LLM, never decides truth)\n  → structured claim  (unverifiable → Refused, never guessed)\n  → query plan (safe templates only — the LLM never writes LogQL/SQL)\n  → evidence: repo index + working-tree git diff (changed lines, file list,\n    untracked files) + command receipts (`truth run`) + log queries\n  → deterministic verdict engine (fixed rules, source-authority order,\n    diff \u003e stale index, receipts must postdate the last edit)\n  → cited verdict: Supported / Contradicted / Refused\n```\n\nEvery check is stored as an audit trail (the claim, the queries run, the\nverdict) in SQLite. Log samples are redacted (emails, JWTs, UUIDs, IPs, tokens)\nbefore being stored or shown.\n\n## Other commands\n\n`truth` is built on a general claim/evidence engine; `verify-turn` is the agent\nfront door. The engine is also usable directly:\n\n```\ncheck     Check a single natural-language engineering claim\nrun       Run a command and record a receipt (makes \"tests pass\" verifiable)\nrecord-run  Record a receipt for a command that already ran (hooks, CI)\nstats     The lie ledger: claims checked, contradictions caught, runs recorded\nhook      Install agent-harness hooks (verification the agent can't skip)\nusage     Observed usage of a route/event/pattern (deterministic)\nerrors    Error occurrences (deterministic)\nconfig    Search indexed config/code definitions\nowners    Who has worked on the code behind a subject\nuses      Find code references to a symbol/route/dependency\ndocs      Is a subject documented, and consistent with code?\ninspect   Show exactly what was indexed (trust the evidence)\ndoctor    Validate local setup and explain readiness\nclaims/report/ci/eval/diff   claim files, reports, CI gates, regression diffs\n```\n\nRun `truth \u003ccommand\u003e --help` for details. `truth serve` (Slack/HTTP) is an\ninformational placeholder and intentionally not built — the local verifier is\nthe product.\n\n## Tests\n\n```bash\ncargo test --workspace\n```\n\nCovers extractors (Rust/TS/Python/Go routes, constants, env vars, deps, file/\nscope/rename/count/command claims), the git-diff adapter (changed lines, file\nstatuses, untracked files), the verdict rules and golden fixtures (including\nreceipt freshness: green-but-stale runs never pass), claim segmentation, hook\nsettings merging, index-freshness warnings, JSON output, and end-to-end checks\nover the sample repo. `truth eval fixtures/eval/agent_claims.yaml` is the\nagent-fact-checking quality harness.\n\n### Measuring extraction quality\n\n`fixtures/eval/extractor_corpus.yaml` is a **diagnostic corpus**, not a gate: the\nsame ground-truth facts phrased many ways, including hard `H*` edge cases the\nregex extractor is *expected* to miss. Run it to measure where claim extraction\nstands and what a better extractor (agent-supplied `claims`, a local LLM, or\nAST) would improve:\n\n```bash\ntruth eval fixtures/eval/extractor_corpus.yaml\n```\n\nA `T*`/`H*` case that returns `inconclusive` is a **recall gap** (extractor too\nweak); an `F*` case that returns `supported` is a dangerous **false pass**; an\n`R*` case that returns a verdict is a **hallucination**. The bands make all\nthree visible, so changes can be measured instead of guessed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblasrodri%2Ftruth","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fblasrodri%2Ftruth","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fblasrodri%2Ftruth/lists"}