{"id":47587701,"url":"https://github.com/heymumford/cognilateral-trust","last_synced_at":"2026-04-01T16:40:36.947Z","repository":{"id":345194987,"uuid":"1184881876","full_name":"heymumford/cognilateral-trust","owner":"heymumford","description":"AI that tells you when it's guessing. Confidence tiers + accountability for AI agents. pip install cognilateral-trust","archived":false,"fork":false,"pushed_at":"2026-03-26T01:46:25.000Z","size":222,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-03-26T17:15:39.385Z","etag":null,"topics":["accountability","agents","ai","anthropic","confidence","crewai","dspy","epistemic","guardrails","langgraph","llm","openai","python","safety","trust"],"latest_commit_sha":null,"homepage":"https://pypi.org/project/cognilateral-trust/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/heymumford.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-18T02:48:20.000Z","updated_at":"2026-03-26T01:46:28.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/heymumford/cognilateral-trust","commit_stats":null,"previous_names":["heymumford/cognilateral-trust"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/heymumford/cognilateral-trust","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heymumford%2Fcognilateral-trust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heymumford%2Fcognilateral-trust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heymumford%2Fcognilateral-trust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heymumford%2Fcognilateral-trust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/heymumford","download_url":"https://codeload.github.com/heymumford/cognilateral-trust/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/heymumford%2Fcognilateral-trust/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31290538,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T13:12:26.723Z","status":"ssl_error","status_checked_at":"2026-04-01T13:12:25.102Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["accountability","agents","ai","anthropic","confidence","crewai","dspy","epistemic","guardrails","langgraph","llm","openai","python","safety","trust"],"created_at":"2026-04-01T16:40:33.822Z","updated_at":"2026-04-01T16:40:36.942Z","avatar_url":"https://github.com/heymumford.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cognilateral-trust\n\n**Your AI agent is about to act. Should it?**\n\n`cognilateral-trust` answers that question. You give it a confidence score. It tells you whether to proceed, verify, or escalate — and produces an immutable audit record of every decision.\n\n[![PyPI](https://img.shields.io/pypi/v/cognilateral-trust)](https://pypi.org/project/cognilateral-trust/)\n[![Python](https://img.shields.io/pypi/pyversions/cognilateral-trust)](https://pypi.org/project/cognilateral-trust/)\n[![License](https://img.shields.io/github/license/heymumford/cognilateral-trust)](LICENSE)\n\n```bash\npip install cognilateral-trust\n```\n\nZero dependencies. Python 3.11+.\n\n### Calibration (TrustBench v1.0.0)\n\n| Domain | Calibration Error | Scenarios |\n|--------|------------------:|----------:|\n| Factual | 0.575 | 40 |\n| Reasoning | 0.550 | 40 |\n| Ambiguous | 0.250 | 40 |\n| Out-of-distribution | 0.250 | 40 |\n| Adversarial | 0.250 | 40 |\n| **Overall** | **0.375** | **200** |\n\nLower is better. Run `trust-bench run --model your-model` to reproduce. [Full methodology](docs/CALIBRATION.md).\n\n---\n\n## What It Is\n\nAn epistemic trust layer for AI agents. One function call between your agent's decision and its action.\n\n**The problem:** AI agents act on confidence they haven't earned. A model says \"I'm 90% sure\" — but is that calibrated? Is this action reversible? Does it affect a person? Most frameworks skip these questions.\n\n**The solution:** `evaluate_trust()` takes a confidence score and returns a routing decision — ACT, verify evidence, or escalate to a human. Every call produces an accountability record. No external services, no API keys, no dependencies.\n\n## How It Works\n\n```\nYour agent's confidence (0.0 → 1.0)\n        │\n        ▼\n┌─────────────────────────────────────┐\n│         evaluate_trust()            │\n│                                     │\n│  1. Map confidence → epistemic tier │\n│     (C0-C9, ten levels)            │\n│                                     │\n│  2. Route by tier:                  │\n│     C0-C3 → basic (act freely)     │\n│     C4-C6 → warrant_check (verify) │\n│     C7-C9 → sovereignty_gate       │\n│                                     │\n│  3. Check context:                  │\n│     - Is this reversible?           │\n│     - Does it touch external systems│\n│     - Welfare constraint (D-05)     │\n│                                     │\n│  4. Return: ACT or ESCALATE        │\n│     + accountability record         │\n└─────────────────────────────────────┘\n        │\n        ▼\n  ACT (safe to proceed)\n  — or —\n  ESCALATE (needs human review, with reasons)\n```\n\n## What You Put In\n\n| Input | Type | Required | What It Means |\n|-------|------|----------|---------------|\n| `confidence` | `float` (0.0-1.0) | Yes | How confident is your agent in this action? |\n| `is_reversible` | `bool` | No (default: `True`) | Can this action be undone? |\n| `touches_external` | `bool` | No (default: `False`) | Does this affect systems outside your control? |\n| `context` | `dict` | No | Free-form metadata for the audit trail |\n\n**That's it.** One float is the minimum. The other parameters sharpen the routing.\n\n## What You Get Out\n\n| Output | Type | What It Tells You |\n|--------|------|-------------------|\n| `should_proceed` | `bool` | **The answer.** Act or escalate. |\n| `tier` | `ConfidenceTier` | Which of 10 epistemic tiers (C0-C9) this confidence maps to |\n| `route` | `str` | `\"basic\"`, `\"warrant_check\"`, or `\"sovereignty_gate\"` |\n| `accountability_record` | `AccountabilityRecord` | Immutable record: who decided, why, when, at what confidence |\n\nThe accountability record is the durable value. It answers \"why did the agent do that?\" after the fact.\n\n## Quick Start\n\n```python\nfrom cognilateral_trust import evaluate_trust\n\n# Your agent has 70% confidence in its next action\nresult = evaluate_trust(0.7)\n\nif result.should_proceed:\n    perform_action()\nelse:\n    escalate_to_human(result.accountability_record.reasons)\n```\n\n### With context\n\n```python\nresult = evaluate_trust(\n    0.85,\n    is_reversible=False,     # can't undo this\n    touches_external=True,   # affects a live system\n)\n# verdict: ESCALATE — irreversible + external at sovereignty-gate tier\n```\n\n### Extract confidence from LLM output\n\nDon't have a confidence score? Extract one from the LLM's response:\n\n```python\nfrom cognilateral_trust import extract_confidence\n\n# Works with OpenAI response dicts, Anthropic response dicts, or plain text\nconfidence = extract_confidence(llm_response)\nresult = evaluate_trust(confidence)\n```\n\n```python\nfrom cognilateral_trust import extract_confidence_from_text\n\nconfidence = extract_confidence_from_text(\"I'm about 73% confident in this answer\")\n# =\u003e 0.73\n```\n\nParses percentages, decimals, verbal qualifiers (\"highly confident\", \"somewhat uncertain\"), and logprobs.\n\n### Nutrition Label — Tell the End User\n\nAttach a standard disclosure to any AI-generated output:\n\n```python\nfrom cognilateral_trust import nutrition_label\n\nlabel = nutrition_label(0.7, calibration_accuracy=0.625)\nprint(label)\n# Trust evaluated. Confidence: 0.70 (C7). Calibration: 62.5%. Verdict: ACT.\n```\n\nFor responses that weren't evaluated:\n\n```python\nfrom cognilateral_trust import not_evaluated_label\n\nlabel = not_evaluated_label()\n# Not trust-evaluated. No confidence assessment was performed on this response.\n```\n\nThe person downstream deserves to know.\n\n---\n\n## Deeper Capabilities\n\n### Calibration — Learn From Outcomes\n\nTrack whether your agent's confidence was actually justified:\n\n```python\nfrom cognilateral_trust import CalibratedTrustEngine\n\nengine = CalibratedTrustEngine()\n\n# Evaluate\neval_id = engine.evaluate(0.8)\n\n# Later, record what actually happened\nengine.record_outcome(eval_id, correct=True)\n\n# Get calibration stats\nstats = engine.calibration_report()\n# Brier score, ECE, per-tier accuracy\n```\n\nOver time, this tells you: \"When your agent says 80%, is it right 80% of the time?\"\n\n### Warrants — Evidence-Backed Confidence That Decays\n\nConfidence should erode when evidence gets stale:\n\n```python\nfrom cognilateral_trust import Warrant, evaluate_trust_with_warrant\n\nwarrant = Warrant(\n    confidence=0.9,\n    evidence_source=\"unit tests pass\",\n    ttl_seconds=3600,  # valid for 1 hour\n)\n\n# Effective confidence decays linearly toward 0 as the warrant ages\nresult = evaluate_trust_with_warrant(warrant)\n```\n\n### Claims Extraction + Fidelity Verification\n\nExtract claims from text and verify them against source material:\n\n```python\nfrom cognilateral_trust import extract_claims, verify_fidelity\n\n# Extract structured claims\nclaims = extract_claims(\"The model achieves 95% accuracy, outperforming GPT-4.\")\n# =\u003e [factual: \"achieves 95% accuracy\", comparative: \"outperforming GPT-4\"]\n\n# Verify against source\nresult = verify_fidelity(\n    claim=\"The system handles 10,000 requests per second\",\n    source=\"Our benchmarks show 10,000 req/s under load\",\n)\n# result.supported = True, result.score = 0.78\n```\n\n### Epistemic Firewall — Prevent Acting on Hunches\n\nBlock actions that demand tested evidence when only hunches exist:\n\n```python\nfrom cognilateral_trust import check_epistemic_mismatch, EpistemicLevel\n\nresult = check_epistemic_mismatch(\n    demanded=EpistemicLevel.VALIDATED,  # deploy requires validated evidence\n    supplied=EpistemicLevel.OBSERVED,   # we only have observations\n)\n# result.is_mismatch = True, result.gap = 3\n```\n\nSeven levels: `RAW` \u003c `OBSERVED` \u003c `MEASURED` \u003c `TESTED` \u003c `VALIDATED` \u003c `FALSIFIABLE` \u003c `GOVERNANCE`.\n\n### Sovereignty Gate — The Welfare Constraint\n\nThe D-05 hard constraint: welfare-critical actions **always** escalate, regardless of confidence:\n\n```python\nfrom cognilateral_trust import evaluate_sovereignty\n\ndecision = evaluate_sovereignty(\n    confidence=0.99,\n    is_reversible=True,\n    tests_pass=True,\n    welfare_affected=True,  # someone's wellbeing is at stake\n)\n# verdict: ESCALATE — welfare gate overrides confidence\n```\n\n### Middleware + Decorators\n\n```python\nfrom cognilateral_trust import trust_gate\n\n@trust_gate(min_confidence=0.6)\ndef deploy(confidence: float, **kwargs):\n    \"\"\"Only runs if confidence \u003e= 0.6\"\"\"\n    ...\n```\n\n### Persistence — Survive Restarts\n\n```python\nfrom cognilateral_trust import JSONLPredictionStore, JSONLAccountabilityStore\n\npredictions = JSONLPredictionStore(\"./data/predictions.jsonl\")\naccountability = JSONLAccountabilityStore(\"./data/accountability.jsonl\")\n```\n\n### Agent Lifecycle — Trust-Gated Spawning\n\n```python\nfrom cognilateral_trust import spawn_gate\n\n@spawn_gate(min_confidence=0.7)\ndef create_worker(confidence: float):\n    \"\"\"Only spawns if parent agent has sufficient trust\"\"\"\n    ...\n```\n\n---\n\n## Framework Integrations\n\n### LangGraph\n\n```python\nfrom cognilateral_trust import evaluate_trust\n\ndef trust_node(state):\n    result = evaluate_trust(state[\"confidence\"])\n    return {**state, \"proceed\": result.should_proceed}\n```\n\nFull example: [`examples/langgraph_trust_node.py`](examples/langgraph_trust_node.py)\n\n### CrewAI\n\n```python\nfrom cognilateral_trust import evaluate_trust\n\ndef trust_check(confidence, **kwargs):\n    result = evaluate_trust(confidence, **kwargs)\n    return \"PROCEED\" if result.should_proceed else f\"ESCALATE: {result.accountability_record.reasons}\"\n```\n\nFull example: [`examples/crewai_trust_tool.py`](examples/crewai_trust_tool.py)\n\n### Any Framework\n\n`evaluate_trust()` takes a float, returns a decision. Wrap it however your framework expects.\n\n---\n\n## CLI\n\n```bash\n$ trust-check 0.7\nACT — C7 (sovereignty_gate)\n\n$ trust-check 0.92 --irreversible\nESCALATE — C9 (sovereignty_gate): irreversible action at sovereignty-grade tier\n\n$ trust-check 0.3 --json\n{\"confidence\": 0.3, \"tier\": \"C3\", \"route\": \"basic\", \"should_proceed\": true, ...}\n```\n\n## Hosted API\n\nDon't want to install anything? Use the hosted API:\n\n```bash\ncurl -X POST https://cognilateral.com/api/v1/evaluate \\\n  -H \"Authorization: Bearer YOUR_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"confidence\": 0.7, \"is_reversible\": true, \"touches_external\": false}'\n```\n\nSign up at [cognilateral.com](https://cognilateral.com) — free tier includes 100 evaluations/day.\n\n## Examples\n\n| Example | What It Shows |\n|---------|---------------|\n| [`demo_trust_agent.py`](examples/demo_trust_agent.py) | Full agent with extraction, warrants, and sovereignty |\n| [`demo_calibration_loop.py`](examples/demo_calibration_loop.py) | Track and improve agent calibration over time |\n| [`langgraph_trust_node.py`](examples/langgraph_trust_node.py) | Trust gate for LangGraph pipelines |\n| [`crewai_trust_tool.py`](examples/crewai_trust_tool.py) | CrewAI tool wrapping trust evaluation |\n| [`openai_trust_wrapper.py`](examples/openai_trust_wrapper.py) | Confidence extraction from OpenAI responses |\n| [`anthropic_trust_wrapper.py`](examples/anthropic_trust_wrapper.py) | Confidence extraction from Anthropic responses |\n| [`dspy_trust_module.py`](examples/dspy_trust_module.py) | DSPy module with trust-gated assertions |\n| [`github_actions_trust_gate.yml`](examples/github_actions_trust_gate.yml) | CI gate: block auto-merge when confidence is low |\n\n## Why This Exists\n\nGuardrails protect systems. Trust protects people.\n\nWhen an AI agent acts on misplaced confidence, the person at the other end pays — the patient who gets a wrong diagnosis, the student who gets a fabricated citation, the operator who ships a broken deploy at 3am because the model said it was fine.\n\nThis library exists so AI systems can say \"I'm not sure enough to act on this\" **before** the damage happens.\n\n## License\n\nApache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheymumford%2Fcognilateral-trust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fheymumford%2Fcognilateral-trust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fheymumford%2Fcognilateral-trust/lists"}