{"id":51836705,"url":"https://github.com/leocelis/cairn","last_synced_at":"2026-07-22T23:02:45.528Z","repository":{"id":371962342,"uuid":"1304470303","full_name":"leocelis/cairn","owner":"leocelis","description":"Deterministic entity/world-model engine for AI agents — resolve, traverse, and assemble trustworthy context with zero LLM calls on the hot path.","archived":false,"fork":false,"pushed_at":"2026-07-18T00:04:12.000Z","size":581,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-18T02:06:56.276Z","etag":null,"topics":["ai-agents","context-engineering","deterministic","entity-resolution","knowledge-graph","python","rag","retrieval","zero-dependencies"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leocelis.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":"ROADMAP.md","authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-17T23:50:07.000Z","updated_at":"2026-07-18T00:04:15.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/leocelis/cairn","commit_stats":null,"previous_names":["leocelis/cairn"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/leocelis/cairn","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leocelis%2Fcairn","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leocelis%2Fcairn/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leocelis%2Fcairn/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leocelis%2Fcairn/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leocelis","download_url":"https://codeload.github.com/leocelis/cairn/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leocelis%2Fcairn/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35780803,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-22T02:00:06.236Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-agents","context-engineering","deterministic","entity-resolution","knowledge-graph","python","rag","retrieval","zero-dependencies"],"created_at":"2026-07-22T23:02:44.763Z","updated_at":"2026-07-22T23:02:45.519Z","avatar_url":"https://github.com/leocelis.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cstrong\u003eCairn\u003c/strong\u003e\u003cbr\u003e\n  \u003cem\u003eThe deterministic entity / world-model engine for AI agents — resolve entities, decide whether to retrieve, route signals, assemble minimal context.\u003c/em\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://pypi.org/project/cairn-engine/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/cairn-engine?style=flat-square\u0026label=cairn-engine\u0026color=blue\" alt=\"PyPI — cairn-engine\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/cairn-retrieval/\"\u003e\u003cimg src=\"https://img.shields.io/pypi/v/cairn-retrieval?style=flat-square\u0026label=cairn-retrieval\u0026color=blue\" alt=\"PyPI — cairn-retrieval\"\u003e\u003c/a\u003e\n  \u003ca href=\"LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-green?style=flat-square\" alt=\"License\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/leocelis/cairn/actions/workflows/ci.yml\"\u003e\u003cimg src=\"https://img.shields.io/github/actions/workflow/status/leocelis/cairn/ci.yml?branch=main\u0026style=flat-square\u0026label=tests\" alt=\"Tests\"\u003e\u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/badge/python-3.11%20%7C%203.12%20%7C%203.13-blue?style=flat-square\u0026logo=python\u0026logoColor=white\" alt=\"Python 3.11–3.13\"\u003e\n  \u003cimg src=\"https://img.shields.io/badge/dependencies-zero-brightgreen?style=flat-square\" alt=\"Zero dependencies\"\u003e\n  \u003ca href=\"https://ivdframework.dev\"\u003e\u003cimg src=\"https://img.shields.io/badge/IVD-intent--verified-purple?style=flat-square\" alt=\"IVD\"\u003e\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## What is Cairn?\n\nA **cairn** is a stack of stones left by travelers, marking the way for those who follow. Cairn is the same idea for AI agents: a **navigation layer** of canonical entities and relations that marks the route to context — built offline, human-approved, consulted before every retrieval.\n\nCairn is a **storage-agnostic routing and composition layer**, not a database. It orchestrates retrieval over stores you already have (files, SQL, vector DB, graph DB). It does **not** own storage, run LLM ingestion on the hot path, or dump an index into the prompt.\n\n**Status:** `cairn-engine` and `cairn-retrieval` are **released on PyPI** (0.1.0) — feature-complete, 120 tests, ruff + mypy strict, zero runtime dependencies, byte-stable output. Next: the benchmark phase (eval vs. agentic-grep / vector-RAG / Graphiti) — quality claims stay honest until it lands. See [`ROADMAP.md`](ROADMAP.md).\n\n## Why\n\nAlways-on retrieval measurably hurts (SRACG, AAAI 2026: −2.6 to −3.6pp vs. no retrieval; selective: +2.4 to +7.1pp), and context length alone degrades accuracy even with perfect retrieval ([Du et al. 2025, arXiv:2510.05381](https://arxiv.org/abs/2510.05381)). LLM-built knowledge graphs drift — non-deterministic extraction, 3–6 LLM calls per episode. Every design rule Cairn implements is traced to a primary source in [`docs/patterns/`](docs/patterns/). Cairn inverts both problems:\n\n- **The frozen alias table is the determinism boundary.** Entity resolution is a lookup cascade over a human-approved, frozen ontology — same input, byte-identical output, zero LLM.\n- **Skipping retrieval is a first-class answer.** A two-stage LLM-free gate decides `none | grep | semantic | graph | composite` *before* any signal runs.\n- **Storage stays yours.** Adapters (in-memory, SQLite, openCypher graph DBs) execute Cairn's bounded traversal intent — the agent never writes a graph query, and Cairn never imports a DB driver.\n\n## Packages (monorepo)\n\n| Package | What it is | State |\n|---------|-----------|-------|\n| **`cairn-engine`** | The deterministic **entity / world-model engine** — resolution cascade (exact → normalized → fuzzy → opt-in semantic → opt-in constrained LLM arbiter), bounded bi-temporal traversal, ontology authoring with a mandatory human gate, cross-system entity index, storage adapters. Pure stdlib, zero dependencies. | [released on PyPI](https://pypi.org/project/cairn-engine/) |\n| **`cairn-retrieval`** | The **agent-retrieval layer** on top — adaptive gate (OP-33), lexical BM25 + scan, opt-in semantic signal, RRF fusion, budgeted context assembler with trace manifest, and `retrieve()` end-to-end. Depends only on `cairn-engine`. | [released on PyPI](https://pypi.org/project/cairn-retrieval/), 0.x until benchmarked |\n\n## What's proven — and what's not\n\nCairn separates what is **test-enforced today** from what still needs the benchmark phase. The determinism and purity claims are structural and verified on every CI run; the *quality* claim is deliberately not asserted until it is measured against baselines.\n\n| Claim | Status | Basis |\n|---|---|---|\n| Deterministic, byte-stable output — same input → identical bytes across runs | ✅ test-enforced | byte-stability + sockets-disabled tests |\n| Zero generative-LLM and zero network on the default path | ✅ test-enforced | hot-path purity test (a counting embedder receives 0 calls) + static import scan of every core file |\n| Zero runtime dependencies in `cairn-engine` core | ✅ test-enforced | `dependencies = []`; CI on Python 3.11 / 3.12 / 3.13 |\n| Storage-agnostic — identical behavior across ≥ 2 backends | ✅ test-enforced | in-memory ↔ SQLite differential test |\n| Closed world — a miss is `unresolved`, never a synthesized ID | ✅ test-enforced | resolution + cross-system-index constraint tests |\n| Grounding a consumer with Cairn raises correct-canonical-id accuracy | ✅ measured | **59% → 100%** (errors eliminated) on a public benchmark — [`benchmarks/entity_resolution/`](benchmarks/entity_resolution/) |\n| Retrieval **quality** beats agentic-grep / vector-RAG / Graphiti at lower cost | ❌ **not yet measured** | Phase 4 (OP-31 eval) — the honest gap; \"best\" stays a thesis until then |\n| Scales to 100k+ entities | ⚠️ same API proven across backends; large-corpus performance unexercised | adapters exist; no scale benchmark yet |\n\nFull test suite: **120 tests, ruff + mypy strict, byte-stable**, green on 3.11–3.13.\n\n## Quickstart\n\n```bash\npip install cairn-engine cairn-retrieval\n```\n\nOr from source, for development:\n\n```bash\ngit clone https://github.com/leocelis/cairn \u0026\u0026 cd cairn\nmake install   # python3.11 venv + editable installs + dev deps\nmake check     # ruff + mypy strict + full test suite\n```\n\nThirty seconds of the entity engine:\n\n```python\nfrom cairn_engine import Entity, InMemoryAliasTable, InMemoryGraph, Ref, Relation, resolve, traverse\n\nentities = [\n    Entity(\"org::acme\", \"Acme Inc\", \"org\", aliases=(\"acme\", \"ACME Corporation\"),\n           refs=(Ref(doc_id=\"doc::acme_overview\"),),\n           relations=(Relation(\"supplier_of\", \"org::beta\"),)),\n    Entity(\"org::beta\", \"Beta LLC\", \"org\", aliases=(\"beta\",),\n           refs=(Ref(doc_id=\"doc::beta_contract\"),)),\n]\ntable = InMemoryAliasTable.from_entities(entities)   # build -\u003e freeze -\u003e read-only\ngraph = InMemoryGraph.from_entities(entities)\n\nresolved, unresolved = resolve(\"what does ACME Corporation supply?\", table=table)\n# resolved[0].canonical_id == \"org::acme\"  (tier: normalized — deterministic, no LLM)\n\nresult = traverse(\"org::acme\", graph=graph, depth=2)\n# hop-ordered DocumentRefs: doc::acme_overview (hop 0), doc::beta_contract (hop 1)\n```\n\nAnd the retrieval layer in one call:\n\n```python\nfrom cairn_retrieval import DocMeta, LexicalIndex, RetrievalEngine\n\nengine = RetrievalEngine(\n    table=table,\n    corpus={\"doc::acme_overview\": DocMeta(\"Acme overview…\", tokens=18),\n            \"doc::beta_contract\": DocMeta(\"Beta contract…\", tokens=22)},\n    graph=graph,\n    lexical=LexicalIndex.from_documents({\"doc::acme_overview\": \"Acme overview…\",\n                                         \"doc::beta_contract\": \"Beta contract…\"}),\n)\nresult = engine.retrieve(\"how do acme and beta relate\", budget=200)\n# gate -\u003e strategy; signals -\u003e RRF fusion -\u003e budgeted, provenance-tagged context.\n# A query the gate routes to \"none\" returns NO context — the skip is the point.\n```\n\nRunnable demos for every subsystem live in [`examples/`](examples/) — all self-contained except the WordPress ones, which take your own corpus (see `examples/wordpress_corpus_prep.py`).\n\n## Design invariants (each test-enforced)\n\nNon-negotiable constraints from the [charter](cairn_system_intent.yaml); each maps to an executing test:\n\n- **Zero generative-LLM and zero network on the default hot path** — the gate, resolver, signals, fusion, and assembler are deterministic; LLM/embedder/DB-driver enter only as caller-supplied callables\n- **Zero runtime dependencies** in the `cairn-engine` core — pure stdlib\n- **Storage-agnostic core** — adapters for in-memory, SQLite, openCypher graph DBs; Cairn never owns the bytes and never exposes a query language\n- **Explicit canonical ontology** — stable IDs you supply and approve; a miss is an explicit `unresolved`, never a synthesized ID (closed world)\n- **Local-first** — the full default path runs offline with no API key\n- **One API, progressive scale** — same call sites for an in-memory corpus and a backend-backed one\n\nPublic API: `resolve()` · `traverse()` · `gate()` · `retrieve()` — plus authoring (`author_from_text`, `dedup_candidates`), bi-temporal state (`as_at`, `supersede`), and the cross-system entity index.\n\n## Repository layout\n\n```\ncairn/\n├── LICENSE · README.md · ROADMAP.md · CHANGELOG.md · CONTRIBUTING.md\n├── pyproject.toml                # uv workspace root\n├── cairn_system_intent.yaml      # system charter (IVD intent format)\n├── packages/\n│   ├── cairn-engine/             # entity / world-model engine  (import: cairn_engine)\n│   │   ├── src/cairn_engine/{entity,adapters}/\n│   │   ├── intents/              # module intents — constraints map 1:1 to tests\n│   │   └── tests/{unit,integration}/\n│   └── cairn-retrieval/          # gate · signals · fusion · assembler · retrieve()\n│       ├── src/cairn_retrieval/\n│       ├── intents/\n│       └── tests/unit/\n├── examples/                     # runnable demos for every subsystem\n├── benchmarks/                   # reproducible, honestly-scored evidence\n└── docs/\n    ├── PRD.md                    # use cases + functional requirements\n    ├── architecture/cairn_architecture.svg\n    ├── patterns/                 # the distilled design rules (TH-1..5, OP-1..36)\n    └── research/                 # foundations/ · context/ · databases/ · tools/\n```\n\n**Start here:** [`cairn_system_intent.yaml`](cairn_system_intent.yaml) (thesis + invariants) → [`docs/patterns/`](docs/patterns/) (the design rules the code implements) → [`examples/full_engine_e2e.py`](examples/full_engine_e2e.py) (the whole engine, narrated).\n\n## How it's built\n\nCairn is developed **intent-first** ([IVD](https://ivdframework.dev)): every module has an intent file declaring its constraints *before* the code, each constraint maps 1:1 to a named test with hand-computed golden fixtures, and joint-satisfaction tests assert all of a module's constraints on the same output. The pattern YAMLs in `docs/patterns/` cite the primary sources (Cormack \u0026 Clarke 2009, Snodgrass 1985, Reiter 1978, Mallen 2023, Du 2025, …) each design rule rests on.\n\nRelated projects: [IVD](https://github.com/leocelis/ivd) · [Horizon](https://github.com/leocelis/horizon)\n\n## Roadmap\n\n1. Charter + research + patterns + PRD ✅\n2. `cairn-engine` 0.1 — entity engine, validated on a real corpus ✅\n3. `cairn-engine` 0.2 — bi-temporal state, SQLite + openCypher adapters, cross-system index ✅\n4. `cairn-retrieval` 0.1 — gate, signals, fusion, assembler, `retrieve()` ✅\n5. **Eval (next)** — benchmark harness (OP-31) vs. agentic-grep / vector-RAG / Graphiti / Mem0 ⏳\n6. Hardening \u0026 ecosystem — more adapters, reranker tier, docs site 💡\n\nFull detail in [`ROADMAP.md`](ROADMAP.md).\n\n## When Cairn is *not* the right choice\n\n- You want the richest possible graph regardless of drift → an LLM-in-the-loop builder (Graphiti/Zep) extracts more, denser relations.\n- Pure document semantic search with no entity/relational structure → a vector DB alone is simpler and sufficient.\n\n## Contributing\n\nSee [`CONTRIBUTING.md`](CONTRIBUTING.md). The short version: intent before code, constraint tests with hand-computed fixtures, `make check` must be green, and PRs that add storage engines or dependencies to core will be declined — adapters only.\n\n## Community\n\n- **Ask a question / propose a design:** [GitHub Discussions](https://github.com/leocelis/cairn/discussions)\n- **Bug reports:** [GitHub Issues](https://github.com/leocelis/cairn/issues)\n- **Security:** please follow the [security policy](SECURITY.md) — do not open a public issue for vulnerabilities\n\n## Author\n\n**Leo Celis** — [leocelis.com](https://leocelis.com)\n\n## Legal\n\nSee [`LEGAL.md`](LEGAL.md) for what Cairn is and is not, IP / trademark / patent terms, acceptable use, warranty disclaimer, limitation of liability, indemnification, and governing law, and [`TERMS_OF_SERVICE.md`](TERMS_OF_SERVICE.md) for the conditions of use you accept by using the software. Security policy: [`SECURITY.md`](SECURITY.md).\n\n## License\n\n[MIT](LICENSE) — Copyright (c) 2026 Leo Celis\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleocelis%2Fcairn","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleocelis%2Fcairn","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleocelis%2Fcairn/lists"}