https://github.com/yzhao062/auditable
Audit any agent decision across its past, present, and future, on one typed graph.
https://github.com/yzhao062/auditable
agent-reliability agentic-ai ai-agents ai-risk ai-safety anomaly-detection llm llm-agents llmops observability python trustworthy-ai
Last synced: 29 days ago
JSON representation
Audit any agent decision across its past, present, and future, on one typed graph.
- Host: GitHub
- URL: https://github.com/yzhao062/auditable
- Owner: yzhao062
- License: apache-2.0
- Created: 2026-06-16T06:18:51.000Z (about 1 month ago)
- Default Branch: main
- Last Pushed: 2026-06-24T04:52:13.000Z (about 1 month ago)
- Last Synced: 2026-06-24T06:26:47.616Z (about 1 month ago)
- Topics: agent-reliability, agentic-ai, ai-agents, ai-risk, ai-safety, anomaly-detection, llm, llm-agents, llmops, observability, python, trustworthy-ai
- Language: Python
- Homepage: https://auditable-ai.readthedocs.io/
- Size: 1.58 MB
- Stars: 5
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
- awesome-auditable-ai - auditable
README
# auditable
**Audit any agent decision across its past, present, and future, on one typed graph.**
*Your logs show what the agent did. `auditable` shows what it relied on, replays it under live state, and rolls it back when it no longer holds.*
[](https://pypi.org/project/auditable/)
[](https://pypi.org/project/auditable/)
[](LICENSE)
[](https://github.com/yzhao062/auditable)
[Compare](#auditable-vs-the-tools-you-already-use) · [Flagship Demo](#the-flagship-moment) · [Plug In Your Agent](#plug-in-your-agent) · [Lifecycle](#the-lifecycle) · [Install](#install) · [Docs](https://auditable-ai.readthedocs.io/) · [Roadmap](#roadmap)
### auditable vs. the Tools You Already Use
| Capability | **auditable** | Tracing and Observability
[LangSmith](#ref-1), [Langfuse](#ref-2), [Phoenix](#ref-3) | Eval Harnesses
[DeepEval](#ref-5), [Ragas](#ref-6), [promptfoo](#ref-7) | Guardrails
[NeMo](#ref-9), [Guardrails AI](#ref-10), [Lakera](#ref-11) |
|---|:---:|:---:|:---:|:---:|
| Captures the dependency state a decision relied on | ✅ | 🟡 | ❌ | ❌ |
| Re-decides under live state (replay) | ✅ | 🟡 | ❌ | ❌ |
| Reverses a committed action (rollback) | ✅ | ❌ | ❌ | ❌ |
| One graph across plan, run, and review | ✅ | 🟡 | ❌ | ❌ |
| Framework-agnostic, dependency-free core | ✅ | 🟡 | 🟡 | 🟡 |
✅ yes · 🟡 partial · ❌ no, per each tool's public docs (2026). These categories are complementary to `auditable`, not competitors: tracing captures execution, evals score quality, guardrails block at runtime. `auditable` is the audit-and-recovery layer that plugs into them. See [References](#references).
**`auditable` is the only column with the full set.** What makes it new:
- **A unified graph representation for agentic AI.** Every agent run becomes one typed graph that PRE, LIVE, and POST all read. One representation from plan to live operation to review, instead of three disconnected tools.
- **Recover, do not just observe.** `auditable` captures the dependency state a decision relied on, replays it under the state that is live now, and reverses the committed action through a compensating rail when it no longer holds. Logging tells you what broke; `auditable` undoes it.
- **One decision, three spans, judged together.** Data, model, and harness are bound in a single signed, hash-chained record, so a decision is audited as one unit, not three disconnected logs.
### Proven on Public Agent Benchmarks
The graph is not just structure. Across six public agent corpora (GRADE, [arXiv:2606.22741](https://arxiv.org/abs/2606.22741)), the dependency layer predicts which runs fail at **ROC-AUC 0.805** where run length carries no signal, and the execution layer localizes the faulting step at **Top-3 0.614**. See [the benchmark detail](docs/post-analysis.md) for the full numbers and corpora.
Figures from GRADE (arXiv:2606.22741).
Agents act on dependency state that quietly drifts. A budget read minutes ago can fall below the amount already committed; a price pinned at plan time can move before the action lands. Most tools log what happened, yet they cannot re-decide under the state that is live now, so a stale decision stands until a human notices. `auditable` closes that recovery gap across the lifecycle on one graph: it captures the decision, replays it against live state, and reverses the committed action when it no longer holds.
## The Flagship Moment
One payment, walked through the whole lifecycle in 18 `auditable` calls. The agent approves a $2,083.20 vendor payment against a budget snapshot that covered it. Six days later the live budget has dropped below the amount. `replay` re-decides on the live state, and the gate reverses the committed payment. This is recovery, not a log line.
```bash
pip install "auditable[graph]"
python examples/example_end_to_end.py
```
The run prints a single audit report: a REVIEW verdict, the keystone with its coverage reason, six findings with severity tags and recommended actions, and the LIVE recovery that rolled the payment back. Paste it into a pull request or an issue.
## Plug In Your Agent
`auditable` works with the agent you already run. Wrap a LangGraph `StateGraph` of plain sync or async function nodes over TypedDict state, and every node's reads and writes over the state channels become **observed** dependency edges, matched across the superstep barrier, with no change to your node logic:
```python
from langgraph.graph import StateGraph
from auditable import analyze_run
from auditable.integrations.langgraph import instrument
builder = instrument(StateGraph(State)) # 1) wrap once, then build / compile / invoke as usual
...
graph = builder.compile()
graph.invoke(initial_state)
report = analyze_run(builder, adapter=builder) # 2) the observed dependency graph for that run
print(report.keystone) # the step the rest of the run rests on
```
```bash
pip install "auditable[langgraph]"
python examples/example_langgraph_capture.py # a real LangGraph run -> observed=100%, keystone named
```
The same wrapped builder also yields **replayable** records, so the LIVE pillar runs on the captured run, not a hand-built dict. `builder.to_records(decisions={"approve": "vendor_payment"})` lowers a marked decision node into a `DecisionRecord` carrying the state it relied on, and `replay(record, live_state=...)` re-decides it under state that is live now and routes a rollback. See [`example_langgraph_live_replay.py`](examples/example_langgraph_live_replay.py): one real LangGraph run, captured, ranked (POST), then replayed and reversed under a drifted budget (LIVE).
Want a real model in the loop? [`example_langgraph_llm_agent.py`](examples/example_langgraph_llm_agent.py) runs the same capture with live LLM nodes against any OpenAI-compatible endpoint (a plain OpenAI key, a gateway, or a local vLLM or Ollama server). The captured edges are identical, because the capture sees the state channels a node read and wrote, not the model call inside it.
Not on LangGraph? The framework-agnostic `TouchRecorder` captures the same observed edges from any loop (a raw OpenAI or Anthropic agent, your own scheduler) by declaring each step's `reads()` and `writes()`. See [`examples/example_touch_capture.py`](examples/example_touch_capture.py). **Roadmap:** LangChain, CrewAI, MCP, and OpenTelemetry.
## Examples and Integrations
Point `auditable` at a scenario and it builds the same typed graph. Each row links a runnable example; browse them all in [`examples/README.md`](examples/README.md).
| Scenario | Pillar | Builds the graph from | Run |
|---|---|---|---|
| Capture a real LangGraph agent | LIVE → POST | a live `StateGraph` run (`instrument`) | [`example_langgraph_capture.py`](examples/example_langgraph_capture.py) |
| Capture a real LLM agent (live model) | LIVE → POST | a model-driven `StateGraph` run (`instrument` + OpenAI-compatible) | [`example_langgraph_llm_agent.py`](examples/example_langgraph_llm_agent.py) |
| Capture any tool loop by hand | LIVE → POST | declared resource touches (`TouchRecorder`) | [`example_touch_capture.py`](examples/example_touch_capture.py) |
| Capture a LangGraph agent, then replay and reverse it | LIVE → POST | a live `StateGraph` run lowered to replayable records (`instrument` + `to_records`) | [`example_langgraph_live_replay.py`](examples/example_langgraph_live_replay.py) |
| Lint a declared plan before deploy | PRE | a framework-agnostic plan dict (`declared_plan_v1`) | [`example_pre_lint_plan.py`](examples/example_pre_lint_plan.py) |
| Recover a payment as the budget drifts | LIVE | a decision captured live (`audit` + `replay`) | [`example_live_replay.py`](examples/example_live_replay.py) |
| Monitor a run as it streams, name the keystone live | LIVE | a run scored prefix by prefix (`LiveSession`) | [`example_live_monitor.py`](examples/example_live_monitor.py) |
| Rank a tau-bench run, name the keystone | POST | a tau-bench trajectory (`tau_bench_prior_db_reads_v1`) | [`example_post_rank_run.py`](examples/example_post_rank_run.py) |
| Walk one payment through every pillar | PRE, LIVE, POST | the full lifecycle (`own_record_v1` for POST) | [`example_end_to_end.py`](examples/example_end_to_end.py) |
The first four rows capture a real run; see [Plug In Your Agent](#plug-in-your-agent) for the two-line setup. **Roadmap:** LangChain, CrewAI, MCP, and OpenTelemetry.
## The Lifecycle
`auditable` runs the same detection-and-report pass over one typed decision graph at three points in an agent's life. The graph kernel stays constant; only the pillar changes (when it fires, what it scores).
| Pillar | When It Fires | Public Entry | Focus |
|---|---|---|---|
| **PRE** | Before deploy | `analyze_plan` | Read-only structural lints on a declared plan. Names the control-flow chokepoint. Dependency-state risk withheld. |
| **LIVE** | While running | `audit` + `replay` + `ActionGate` | Capture a decision, re-decide under live state, route a fix (allow, block, review, rollback) through a rail. The sharpest pillar. |
| **POST** | After a run | `analyze_run` | Rank a finished run by structural blast share. Name the keystone the run rests on, so you review that step first. |
## Install
```bash
pip install auditable
```
The core is dependency-free and torch-free. Structural-graph analysis (`analyze_plan` for PRE, `analyze_run` for POST) needs the optional graph extra (NetworkX):
```bash
pip install "auditable[graph]"
```
The LIVE example runs on the core install alone.
Docs: https://auditable-ai.readthedocs.io/
## The Three Pillars, in Detail
PRE: lint the plan before deploy (four read-only lints, the chokepoint, a coverage report)
Point `analyze_plan` at a declared plan (a plain dict, the neutral target a LangGraph, CrewAI, or AutoGen front-end would lower into) and it runs read-only structural lints over the plan graph. Every check is a pure NetworkX query: no value is executed, and every finding is a structural design warning, not a validated failure prediction.
```python
from auditable.graph.pre import analyze_plan
from auditable.graph.adapters import declared_plan_v1
plan = {
"nodes": [
# 0: read a volatile price, but grant scope far beyond what it read.
{"idx": 0, "agent": "planner", "kind": "tool_call",
"reads": [{"id": "price", "volatile": True}],
"scope": ["price", "ledger", "vendor_db"]},
# 1: a decision that rests on the unpinned, un-revalidated price.
{"idx": 1, "agent": "planner", "kind": "decision",
"reads": [{"id": "price", "producer": 0, "volatile": True}],
"control_preds": [0]},
# 2: a consequential write of 'order', with no prior read of 'order'
# and no re-read of 'price' between the volatile read and the action.
{"idx": 2, "agent": "executor", "kind": "tool_call",
"reads": [{"id": "price", "producer": 0, "volatile": True}],
"writes": ["order"], "control_preds": [1]},
]
}
report = analyze_plan(plan, adapter=declared_plan_v1)
print(report)
```
The four shipping lints, all read-only queries at `severity='warning'`:
| Lint | Fires When |
|---|---|
| `write_with_no_prior_read` | A node writes a resource that nothing in its backward slice ever read. |
| `flippable_dependency_annotation` | An unpinned, non-revalidated volatile dependency feeds a decision. This is an annotation; the would-it-flip question needs runtime values and is out of scope at PRE. |
| `scope_vs_snapshot` | Granted tool scope strictly exceeds the snapshot the node read. |
| `missing_revalidation_barrier` | A volatile read reaches a consequential action with no intervening re-read. Drift confirmation needs runtime values and is out of scope at PRE. |
The report also names the **execution-topology keystone**: the structural chokepoint of the declared plan, the node that the most other nodes transitively follow in control flow (the argmax of `execution_reach` over the `handoff_to` projection). This is a structural design lint, a separate concept from the POST blast-radius keystone, and it does not predict failure.
Alongside the lints, the **Preflight Coverage Report** is a descriptive coverage-readiness view, explicitly not a risk score. It reports the dependency-edge grade mix, the observed fraction, the saturation ratio, the exact no-score reason the runtime scorer would apply, which declared reads, writes, and edges still lack a resource identity, and the declared revalidation barriers per resource.
Two boundaries, stated plainly. Dependency-state blast-share risk is **withheld** at PRE: the declared dependency layer is declared-only (observed fraction zero), so `analyze_plan` returns `state_b_risk=None` with `state_b_withheld=True` and a reason string, and it raises rather than emit a number if a scored verdict ever came back. A table-stakes OWASP-Agentic and CWE rule floor is **planned**, not shipping.
See [`examples/example_pre_lint_plan.py`](examples/example_pre_lint_plan.py) and the [PRE rules reference](docs/pre-rules.md).
POST: rank a finished run, find the keystone (`analyze_run` over a recorded trajectory)
`analyze_run` reads a recorded agent run, builds one decision graph, and ranks every step by how much of the run transitively rests on it, so you review the keystone first. On a tau-bench airline trajectory, the one reservation read that both later writes depend on is the keystone.
```python
from auditable import analyze_run
from auditable.graph.adapters import tau_bench_prior_db_reads_v1
report = analyze_run(run, adapter=tau_bench_prior_db_reads_v1)
k = report.keystone
print(k.idx, k.node_attrs["tool"]) # 2 get_reservation_details
```
The score is an uncalibrated triage ranking, not a calibrated probability. In a no-score state (`no_score:single_decision`, `no_score:low_coverage`) the scores are `None`, so a withheld score never reads as zero risk. The corpus write-to-read edges are modeled (a conservative prior-read upper bound, not a causal label); the report carries these caveats in `report.notes`. The trajectory is modeled on [tau-bench](https://github.com/sierra-research/tau-bench) (Sierra Research, MIT). See [`examples/example_post_rank_run.py`](examples/example_post_rank_run.py) and the [POST analysis reference](docs/post-analysis.md).
How It Works: the two-layer graph, the signed record, the rail (kernel internals)
*auditable links a run into one graph with two edge layers: execution (control flow, observed from the trace) over dependency (what each step relied on). When a step rested on a value that has since gone stale, like `price`, `replay` catches it. The record itself binds three spans per decision (data, model, harness).*
The graph kernel has two edge layers, and the distinction is load-bearing. Execution edges (`emits`, `handoff_to`) are observed from the trace. Dependency edges (`depends_on`) are declared, inferred, or observed, and every edge records how it is known: the corpus and plan adapters declare or infer them, while the LangGraph capture path and the `TouchRecorder` read them off a real run as observed channel touches. PRE and POST both run over this same typed graph; `audit()` is the ergonomic capture entry, so you are never asked to build the graph by hand.
One agent decision crosses three spans, and `auditable` binds all three in a single signed, hash-chained record:
| Span | What the Record Binds | Signal in v0.1 |
|---|---|---|
| **Data** | What the agent read and the dependency snapshot it relied on | Snapshot freshness |
| **Model** | Which model produced the output, and its stated basis | Decision-basis trust flag |
| **Harness** | The action executed and its cost | A static cost-cap rule, plus the replay verdict |
`replay()` re-derives whether the action still holds under the live dependency state versus the snapshot the agent used, and returns one of four routed verdicts: `ALLOW`, `ROLLBACK` (justified on the snapshot but not on live state, the stale-state case), `BLOCK` (justified on neither), or `HUMAN_REVIEW` (the policy raised `ReplayUndecidable`). A `Policy` is any callable `(state, action) -> (justified, reason)`. The `ActionGate` then executes that verdict through a `Rail`: a post-commit `ROLLBACK` or `BLOCK` calls `rail.compensate(receipt)` to reverse a committed action, rather than printing a recommendation. The shipped `ReferenceLedger` is an in-process reference rail for demos and tests; it is not a production payment rail.
Records are signed and hash-chained: each record carries a `prev_digest` and a content-addressed `record_id`. Two sinks ship today, `MemorySink` (in-process) and `FileSink` (append-only JSONL, durable across process exit, fails closed on a corrupt tail).
Ingestion is source-agnostic through the public `Adapter` protocol. The corpus and plan adapters ship `tau_bench_prior_db_reads_v1` (public-corpus trajectory, POST), `own_record_v1` (auditable's own signed records, POST), and `declared_plan_v1` (a framework-agnostic declared plan dict, PRE). For a real run, the LangGraph capture path (`auditable.integrations.langgraph.instrument`) and the generic `TouchRecorder` produce observed dependency edges with a structured `ResourceRef`, matched superstep-aware and reducer-aware. The declared-plan adapter stays the neutral PRE seam a LangGraph or CrewAI front-end would lower a plan into; it is not a parser for any framework.
See the [architecture reference](docs/architecture.md) for the full kernel, adapters, and sinks.
Using a single layer (standalone auditors as inputs to the record)
Each span's check is a standalone `Auditor` that runs on its own, with no agent and no chain, and returns a signed `Report`. `DataAuditor` scores snapshot freshness, `ModelAuditor` produces a decision-basis trust flag, and `HarnessAuditor` applies one static cost-cap rule.
```python
import time
from auditable import DataAuditor, DependencySnapshot
snapshot = DependencySnapshot(state={"budget_remaining": 1000}, captured_at=time.time() - 7 * 86400)
report = DataAuditor(max_age_seconds=86400).assess(snapshot)
print(report.flag, report.score) # stale 1.0
```
These standalone auditors are inputs to the record, not the headline. The composition (capture, replay, recovery, and the lifecycle pillars) is the main line; the modules feed it. See [`examples/example_standalone_report.py`](examples/example_standalone_report.py).
Scope, stated honestly (what ships today vs. what is planned)
**What ships today.** The full signed chain, replay under live state, executed recovery through a rail-neutral gate, two sinks (in-memory and append-only JSONL), the POST `analyze_run` ranking, the PRE `analyze_plan` lints plus preflight coverage report, and real-run capture into observed dependency edges (LangGraph via `instrument`, any loop via `TouchRecorder`).
The release does **not** yet claim a learned data-anomaly method, a calibrated model-trust score, calibrated cross-layer risk, or live incremental scoring. The POST structural score is an uncalibrated ranking; the compound report is a transparent, explicitly uncalibrated debug bundle. Everything beyond this list is on the [roadmap](#roadmap).
## Roadmap
What ships today, and what is next
Shipping today: the full capture, replay, and recover chain; PRE plan lints; POST run ranking; signed, hash-chained records; two sinks. Next:
- **Smarter drift detection.** Learn what healthy dependency state looks like and flag anomalies, not just staleness (PyOD backend, with freshness as the fallback).
- **One calibrated risk score.** Combine the data, model, and action signals into a single calibrated score, with the model treated as a first-class node in the graph.
- **Live scoring.** Score decisions on the prefix graph as the agent runs, not only after the run completes. (Capturing what each step read and wrote already ships for LangGraph and the `TouchRecorder`.)
- **Automatic fixes.** Refresh stale data, quarantine bad inputs, fall back to a safer model, or require human sign-off.
- **Built-in CI checks.** A table-stakes OWASP-Agentic and CWE rule floor for continuous integration.
- **Integrations (1.0).** LangGraph capture ships now (`instrument`); next are one-line hooks for LangChain and CrewAI, an MCP server, export to OpenTelemetry and LangSmith, downloadable evidence bundles, and a stable public API.
## Related Projects
`auditable` is the SDK in a small family of open agent-reliability projects:
- **[awesome-auditable-ai](https://github.com/yzhao062/awesome-auditable-ai)** — a curated reading list on AI agent reliability and auditing.
- **[GRADE](https://arxiv.org/abs/2606.22741)** — the research behind the typed two-layer graph this library is built on (also in [References](#references)).
## References
Compared tools, benchmarks, and method
The capability marks in the table above are per each project's public documentation as of 2026.
**Compared tools**
`[1]` [LangSmith](https://docs.langchain.com/langsmith)
`[2]` [Langfuse](https://github.com/langfuse/langfuse)
`[3]` [Arize Phoenix](https://github.com/Arize-ai/phoenix)
`[4]` [Weights & Biases Weave](https://github.com/wandb/weave)
`[5]` [DeepEval](https://github.com/confident-ai/deepeval)
`[6]` [Ragas](https://github.com/explodinggradients/ragas)
`[7]` [promptfoo](https://github.com/promptfoo/promptfoo)
`[8]` [OpenAI Evals](https://github.com/openai/evals)
`[9]` [NeMo Guardrails](https://github.com/NVIDIA-NeMo/Guardrails)
`[10]` [Guardrails AI](https://github.com/guardrails-ai/guardrails)
`[11]` [Lakera Guard](https://www.lakera.ai/lakera-guard)
**Benchmarks and datasets**
`[12]` [tau-bench](https://github.com/sierra-research/tau-bench) (arXiv:2406.12045)
`[13]` [Who&When: Which Agent Causes Task Failures and When?](https://github.com/ag2ai/Agents_Failure_Attribution) (arXiv:2505.00212)
**Method**
`[14]` [GRADE: Graph Representation of LLM Agent Dependency and Execution](https://arxiv.org/abs/2606.22741)
## Citation
If you use `auditable` in your work, please cite the GRADE paper, the typed two-layer graph model the library is built on:
```bibtex
@article{zhao2026grade,
title = {GRADE: Graph Representation of LLM Agent Dependency and Execution},
author = {Zhao, Yue},
journal = {arXiv preprint arXiv:2606.22741},
year = {2026}
}
```
## License
[Apache-2.0](LICENSE).