{"id":51523369,"url":"https://github.com/hoomanesteki/agentic-rag-knowledge-ai-platform","last_synced_at":"2026-07-08T18:01:31.689Z","repository":{"id":368737169,"uuid":"1286575486","full_name":"hoomanesteki/agentic-rag-knowledge-ai-platform","owner":"hoomanesteki","description":"Skein is an agentic AI platform that answers questions by weaving together structured and unstructured data, grounding every response in a knowledge graph, hybrid vector search, and a governed semantic layer. It escalates to humans when unsure, and runs free locally or scales to the cloud.","archived":false,"fork":false,"pushed_at":"2026-07-02T02:09:16.000Z","size":52,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-02T02:20:44.469Z","etag":null,"topics":["agentic-ai","dbt","drift-detection","fastapi","graphrag","human-in-the-loop","knowledge-graph","lakehouse","langchain","langgraph","llm","mlflow","mlops","neo4j","pyspark","qdrant","rag","reranker","semantic-layer","vector-search"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/hoomanesteki.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-01T23:19:58.000Z","updated_at":"2026-07-02T02:09:20.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/hoomanesteki/agentic-rag-knowledge-ai-platform","commit_stats":null,"previous_names":["hoomanesteki/agentic-rag-knowledge-ai-platform"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/hoomanesteki/agentic-rag-knowledge-ai-platform","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoomanesteki%2Fagentic-rag-knowledge-ai-platform","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoomanesteki%2Fagentic-rag-knowledge-ai-platform/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoomanesteki%2Fagentic-rag-knowledge-ai-platform/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoomanesteki%2Fagentic-rag-knowledge-ai-platform/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/hoomanesteki","download_url":"https://codeload.github.com/hoomanesteki/agentic-rag-knowledge-ai-platform/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/hoomanesteki%2Fagentic-rag-knowledge-ai-platform/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35273524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-08T02:00:06.796Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic-ai","dbt","drift-detection","fastapi","graphrag","human-in-the-loop","knowledge-graph","lakehouse","langchain","langgraph","llm","mlflow","mlops","neo4j","pyspark","qdrant","rag","reranker","semantic-layer","vector-search"],"created_at":"2026-07-08T18:01:30.338Z","updated_at":"2026-07-08T18:01:31.678Z","avatar_url":"https://github.com/hoomanesteki.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Skein Lite\n\n[![CI](https://github.com/hoomanesteki/agentic-rag-knowledge-ai-platform/actions/workflows/ci.yml/badge.svg)](https://github.com/hoomanesteki/agentic-rag-knowledge-ai-platform/actions/workflows/ci.yml)\n\n**[Read the showcase →](https://hoomanesteki.github.io/agentic-rag-knowledge-ai-platform/)**:\narchitecture, evaluation, the decisions behind it, and an honest look at the failure modes. The\nsite's source is in [`showcase/`](showcase/) (Quarto, rendered to GitHub Pages by CI).\n\nA local-first, **domain-swappable agentic RAG platform**. It answers over a mix of structured and\nunstructured data, cites every answer (or honestly says it does not know), and routes each turn\nthrough a **master orchestrator** to the right specialist, handing off to a human when unsure.\n\n**One engine, any topic:** a new domain is a config folder, not code. Tests and CI run fully\noffline on fakes; the models (Groq, Cohere) and stores (Qdrant, DuckDB, Neo4j) are config swaps.\n\n## What it does\n\n- **Grounded or honest.** Hybrid retrieval (dense + sparse, RRF) with a reranker, sentence-level\n  citation checks, and an abstain gate. Retrieved text is sanitized against prompt injection.\n- **A master orchestrator, not a prompt.** The default brain (`CHAT_BRAIN=omni`) routes each turn to\n  a specialist lane, shopping (Sara), care, complaint, answers, or escalation (Tiffany), all sharing\n  **one gated pipeline**, so specialization sharpens tone, never safety. Free deterministic layers\n  handle most turns (81.6%); a cheap 8B tie-break lifts the rest to 85.9% with 100% escalation\n  recall. (A `linear` and a LangGraph `agent` brain are also selectable.) Diagram below.\n- **Guards that do not trust the model.** Order PII, prompt injection, customer enumeration, and\n  gender-correct recommendations are enforced deterministically in code, before the prompt. See\n  [the guardrails](#guardrails-enforced-in-code-not-in-the-prompt).\n- **A real data stack.** The medallion (bronze, silver, gold) is modeled in **dbt** with schema,\n  relationship, and PII-masking tests, generated from each domain's manifest. A **semantic layer**\n  (`metrics.yaml`) is the single source of truth the agent, the eval, and the dashboards all read.\n- **Governed numbers.** A read-only DuckDB metric layer answers \"what is the return rate for size\n  M\" from a validated single-SELECT query, never free-form SQL, and the number is cited as its own\n  evidence.\n- **A knowledge graph.** Neo4j nodes and typed edges built from gold; relational questions (\"which\n  supplier makes X\") answer from the graph via allowlisted traversals, not free Cypher.\n- **Human-in-the-loop flywheel.** Escalations land in a review queue; an operator answers; the\n  answer becomes a retrievable verified chunk and grows the eval set.\n- **Observability and MLOps.** Langfuse traces every turn. Four monitoring pillars read the one\n  trace store, data drift, model quality, system health, and business KPIs. Weekly **continuous\n  training** retrains, gates, and registers a candidate to a versioned **model registry**; a human\n  promotes it, so nothing self-ships.\n- **Guided and voiced.** Per-domain starter prompts, spoken input (Groq Whisper), spoken replies\n  (browser voice by default, ElevenLabs when keyed), a storefront-style demo UI with the chat\n  widget, and a backoffice dashboard at `/admin`.\n- **One engine, any domain.** A domain is a config folder (`domains/\u003cname\u003e/`: seed data plus a\n  manifest), never engine code. The demo ships `apparel_ecommerce`; a leak linter fails CI if a\n  single brand, product, metric, or persona name reaches an engine folder, which is what keeps the\n  swap honest.\n- **Fails soft, never dark.** Every provider degrades in layers instead of dying: embeddings roll\n  trial key to paid key to local sparse, the LLM large to small to a safe reply, voice premium to\n  browser to text, and the graph and metric layers are additive. Full chain in\n  [docs/fallbacks.md](docs/fallbacks.md).\n\n## The stack\n\n| Layer | Choice | Where |\n| --- | --- | --- |\n| API | FastAPI: SSE chat, JWT auth, rate limiting, Turnstile, degraded mode | `api/` |\n| Brain | Master orchestrator (routes each turn to a lane) over one gated pipeline; `CHAT_BRAIN` also selects a linear or a LangGraph-supervisor brain | `rag/`, `pipeline/` |\n| Retrieval | Qdrant hybrid (dense + sparse, server-side RRF), Cohere `embed-v4.0` + `rerank-v3.5` | `adapters/`, `retrieval/` |\n| Generation | Groq Llama 3.3 70B (large) and Llama 3.1 8B (small) | `adapters/groq.py` |\n| Voice | Groq Whisper in; browser voice or ElevenLabs Flash v2.5 out (key stays server-side) | `adapters/groq_whisper.py`, `adapters/elevenlabs.py` |\n| Analytics | DuckDB + dbt medallion, `metrics.yaml` semantic layer | `dbt/`, `data/` |\n| Graph | Neo4j, loaded from gold | `knowledge/`, `adapters/neo4j_store.py` |\n| MLOps | Langfuse tracing, MLflow (Postgres-backed compose server or `./mlruns`), RAGAS eval, four drift monitors, CI eval gate, weekly continuous training + a versioned model registry | `mlops/`, `evaluation/` |\n| Web | Next.js 14 storefront demo with the chat widget and `/admin` | `web/` |\n\nEvery provider sits behind an adapter interface, and the defaults are offline fakes\n(`adapters/fakes.py`), so a fresh clone verifies end to end with no keys.\n\n## The system at a glance\n\nA turn flows top to bottom: the API hands it to the brain, the brain answers through one gated path\nover three retrieval backends.\n\n```text\n   Browser (chat + /admin)   ·   Voice\n              |\n              v\n   FastAPI          SSE chat, auth, rate limit, degraded mode\n              |\n              v   CHAT_BRAIN=omni\n   The brain        route the turn, then answer through ONE gated pipeline\n              |\n              v\n   Retrieval        Qdrant vectors  +  DuckDB metrics  +  Neo4j graph\n              |\n              v\n   A grounded, cited answer   ·   or an honest abstain / handoff to a human\n```\n\n## The brain: a manager and five specialists\n\nThe default brain is the master orchestrator. It routes each turn to one specialist, and they all\nanswer through the same gated pipeline, so no lane gets a weaker safety surface.\n\n```text\n                        a shopper turn\n                             |\n                             v\n              +------------------------------+\n              |     MASTER ORCHESTRATOR      |   three cheap-first layers:\n              |          rag/omni.py         |     0  reach a person?   (regex, $0)\n              +--------------+---------------+     1  intent guards      ($0, most turns)\n                             |                     2  cheap 8B tie-break (only if unclear)\n       +--------+--------+---+-----+--------+-----------+\n       v        v        v         v        v           v\n    stylist   care   complaint  answers  escalation   (unclear?\n    (Sara)  (order) (make right)(facts)  (Tiffany)     ask one\n                                                        question)\n       |        |        |         |        |\n       +--------+----+---+---------+--------+\n                     v\n      ONE gated pipeline   ·   pipeline/answer.py\n      understand -\u003e retrieve -\u003e ground -\u003e answer / abstain\n      the SAME PII gate, gender filter, and abstain gate on every lane\n```\n\nThe lanes are **data rows, not code**, a short focus added to one shared prompt, so a new specialist\nis a new row. Routing lives in `rag/router.py`; `CHAT_BRAIN` also selects `linear` (no routing) or\n`agent` (a LangGraph supervisor over three specialists in `rag/supervisor.py`).\n\n## The data architecture\n\nThe engine reads only a domain's manifest, so the same code builds any domain. On top of that, the\nanalytics and semantic layer is real dbt: tested, documented, and lineage-traced.\n\n```text\n   domains/\u003cname\u003e/          the pack: data + a manifest, no engine code\n   +----------------------------------------------------------------+\n   |  seed/structured/*.csv        seed/unstructured/*.jsonl        |\n   |  domain.yaml  (schema, PII, graph edges, metrics, suggestions) |\n   +---------------------------+------------------------------------+\n                               |  the manifest drives everything\n         structured           |            unstructured\n              v                |                  v\n   +--------------------------+|      +-------------------------+\n   |  dbt medallion (DuckDB)  ||      |  chunk + context prefix |\n   |                          ||      |  Cohere embeddings      |\n   |   bronze  raw text       ||      |          v              |\n   |     v     (lineage)      ||      |  Qdrant hybrid index    |\n   |   silver  typed + PII    ||      +-------------------------+\n   |     v     masked         ||\n   |   gold    curated        ||   dbt tests on every build:\n   +-----------+--------------+|     not_null, unique, relationships,\n               |               |     is_masked (any declared PII column)\n        +------+------+        |\n        v             v        |   semantic layer: metrics.yaml is the single\n   +---------+  +-----------+  |   source of truth, read by the agent, the\n   | metric  |  | knowledge |  |   eval, and the dashboards. dbt exposures\n   | layer   |  |  graph    |  |   name those consumers, so lineage answers\n   | (read-  |  | (gold -\u003e  |  |   \"what does this table feed\".\n   | only)   |  |  Neo4j)   |  |\n   +---------+  +-----------+  |\n                               v\n     the same transform runs two ways (an in-app Python builder and dbt);\n     a parity test proves the gold is byte-identical.\n```\n\nA leak linter in `make check` greps every engine folder for each pack's brand, product, metric,\nand glossary vocabulary and fails the build on a hit, which is what keeps the engine reusable\nacross domains. The apparel pack's content is written for realism: 158 products, 810 reviews in\ntotal (including 304 human-voice reviews, a positive and an honest-critical one for most products),\n140 product descriptions grounded in fabric, fit, and care detail, and synthetic orders whose fake\nPII exercises the order gate below. See [docs/semantic-layer.md](docs/semantic-layer.md).\n\n## How one turn works\n\n```text\n   question\n      |\n      v\n   understand ------ expand a short follow-up with the prior turns, repair catalog\n      |              typos, pick a route (factual / relational / qualitative / metric)\n      v\n   retrieve -------- dense + sparse, fused with RRF, then reranked\n      |\n      v\n   ground ---------- sentence-level citation check; retrieved text is sanitized\n      |              and framed as data, never as instructions\n      +-- confident? -- yes --\u003e answer with [1][2] citations\n      |                  no\n      +-- in scope? ---- no --\u003e \"I do not have enough information\" (abstain)\n                         yes\n                          +--\u003e the agent loop retries the hard tail, then escalates\n```\n\n## Guardrails enforced in code, not in the prompt\n\nThe riskiest behaviours are deterministic, applied before the model sees anything, so they hold no\nmatter what the model would say (all in `pipeline/answer.py`, exercised by tests):\n\n- **Order PII needs name + email.** Order documents only enter retrieval for a first-person\n  account question, and each one must then pass an identity check against the shopper's own words:\n  both the account email and the holder's name, where a name token derivable from the email does\n  not count as a second factor. An unverified order document is dropped before it reaches the\n  prompt, so an email-only turn cannot leak a name, an order number, or a tracking link.\n  Third-party lookups (\"orders placed by x@y\") never qualify.\n- **Prompt injection is refused, not resisted.** Requests to reveal or override the system prompt\n  get a deterministic refusal before retrieval; retrieved text is sanitized and every prompt frames\n  context as untrusted data.\n- **Customer enumeration is refused.** \"Who bought X\" and \"list your customers\" are declined\n  before retrieval, so no reviewer or account-holder name can surface.\n- **A stated gender is a hard constraint.** Opposite-gender SKUs are filtered out of the retrieval\n  hits, and opposite-gender picks are redacted clause by clause from guides and reviews before the\n  model sees them. Product gender is read from the domain manifest, never hardcoded in the engine.\n- **Harmful requests are declined**, with the pattern scoped so ordinary shopping phrasing (an\n  \"explosive sprint\") never trips it.\n\n## Quick start\n\nNeeds [uv](https://docs.astral.sh/uv/) (it manages Python 3.12) and Docker.\n\n```bash\nmake setup                 # venv + locked dependencies\ncp .env.example .env       # fill in GROQ_API_KEY and COHERE_API_KEY for real runs\nmake check                 # lint, tests, domain validation, leak check, eval gate (fully offline)\nmake doctor                # if a step hangs or fails, this says why (Docker, .env, keys)\nmake up                    # Qdrant, Postgres, Neo4j, MLflow in Docker (preflighted)\nmake dbt-build             # build + test the semantic layer (medallion + governance tests)\nmake ingest \u0026\u0026 make graph-load                   # build the vector index and the graph\nmake serve                 # API on :8000     (omni master orchestrator by default; CHAT_BRAIN=linear|agent to switch)\ncd web \u0026\u0026 npm install \u0026\u0026 npm run dev             # web chat on :3000\n```\n\nDemo login: `demo` / `Canada54321`. Admin console at `/admin` (`admin` / `skein-admin-2026`).\nA new topic is a new folder under `domains/`: scaffold and validate one with the `domain-pack`\nskill, then point `DOMAIN` at it (the starter prompts and everything else follow the manifest).\nVoice input needs `TRANSCRIBE_PROVIDER=groq`; spoken replies use the browser voice by default, or\nElevenLabs with\n`TTS_PROVIDER=elevenlabs` and a key. Tracing needs the `LANGFUSE_*` keys. `make reproduce` runs\nthe whole offline verification in one command.\n\n## Evaluate\n\n```bash\nmake eval        # hit@k, MRR, entity recall, abstain recall, false-abstain rate on the golden set\nmake ablation    # dense vs hybrid vs +rerank, per language -\u003e docs/eval-report.md\nmake ragas       # faithfulness, answer relevance, context precision/recall (LLM judge)\nmake gate        # the offline CI eval gate (also runs in CI)\nmake drift       # drift across the four monitors from recent traffic, per language\nmake ct          # one Continuous Training cycle: retrain, gate, propose a promotion (human-gated)\nmake registry    # the model registry: versions, stages, the current champion\nmake promote     # gate the config through MLflow stages (dev -\u003e staging -\u003e prod) by eval score\nmake dbt-docs    # the dbt lineage graph and column docs\n```\n\nOnly real runs (keys + `make up` + `make ingest`) produce real numbers; offline they are zero by\ndesign. The ablation lands in [docs/eval-report.md](docs/eval-report.md) (currently the offline\nplaceholder until a keyed run fills it).\n\nTwo of these run fully offline on recorded fixtures and produce real signal: the CI gate blocks a\nregression, and the drift monitor flags a distribution shift. The walkthrough is in\n[notebooks/02-evaluation.ipynb](notebooks/02-evaluation.ipynb):\n\n| The gate blocks a regression | Drift catches a shift |\n| --- | --- |\n| ![eval gate blocks a regression](docs/img/eval-gate.png) | ![drift PSI](docs/img/drift-psi.png) |\n\n## Observability, monitoring, and the three loops\n\nEvery turn is traced once; four monitoring pillars read the same trace store, so a bad answer is\nexplained, not guessed at.\n\n```text\n   every turn --\u003e Langfuse span + request trace: model, tokens, latency, cost, grounding\n                    |\n     FOUR MONITORING PILLARS, all off the one trace store:\n       data      --\u003e drift monitors: embedding distance, retrieval PSI, confidence PSI, feedback\n       model     --\u003e RAGAS + routing accuracy + the offline eval gate\n       system    --\u003e /admin/health:   p95 latency, error rate, cost/turn, throughput\n       business  --\u003e /admin/business: containment vs escalation, answer rate, $/turn, $/session\n                    |\n                    \\--\u003e MLflow: eval runs, drift, CT cycles, promotions (nothing is a screenshot)\n\n   CI  every push     --\u003e make check: lint, tests, domain + leak check, eval gate\n                          + dbt governance + gold parity + web build + dependency audits\n   CD  when CI green  --\u003e ship the serving config\n   CT  weekly / drift --\u003e retrain (OPRO) -\u003e gate -\u003e REGISTER a proposed model version;\n                          a human promotes it (make registry-promote). Nothing self-ships.\n```\n\nCI asks *is the code correct?*, CD ships it, and CT asks *as new data and drift arrive, is a\nretrained candidate better and safe?* Retraining and evaluation are automated; deployment is\nhuman-gated (`mlops/ct.py`, `mlops/model_registry.py`, `.github/workflows/ct.yml`).\n\n## The thinking\n\n- **The plan, theme by theme:** [docs/plan/](docs/plan/) (the big picture, split into stages, each\n  with a short result note).\n- **Architecture end to end:** [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) (data flow, the agentic\n  loop, and the fallback chain).\n- **Reliability and fallbacks:** [docs/fallbacks.md](docs/fallbacks.md), every provider's layered\n  backup plan, one table.\n- **MLOps evidence:** [docs/mlops/](docs/mlops/), a real drift run over the demo traffic and the\n  MLflow promotion runs, with the command to view them.\n- **Decisions and tradeoffs:** [docs/BUILD-PLAN.md](docs/BUILD-PLAN.md) Part A, the\n  [semantic layer](docs/semantic-layer.md), and [model selection](docs/model-selection.md) (why\n  Groq + Cohere, evidenced against the live Health view).\n- **Showcase roadmap:** [docs/SHOWCASE-ROADMAP.md](docs/SHOWCASE-ROADMAP.md) (the staged plan and\n  progress log).\n- **Build log and deliberate deferrals:** [docs/DEV-NOTES.md](docs/DEV-NOTES.md).\n- **Deploy:** [docs/DEPLOY.md](docs/DEPLOY.md) (Vercel, Cloud Run at min-instances 0, the hosted\n  stores, the cost cap, and the keepalive job).\n- **Notebooks:** [notebooks/](notebooks/) walk the data architecture and the eval step by step.\n\n## Development\n\nShort-lived `build/\u003cstep\u003e` branches, one stage each, merged to `main` only when `make check` is\ngreen and an independent review has passed. CI (`.github/workflows/ci.yml`) runs the same checks\nplus the eval gate, the dbt build and governance tests, the web build, and dependency audits on\nevery change.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoomanesteki%2Fagentic-rag-knowledge-ai-platform","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhoomanesteki%2Fagentic-rag-knowledge-ai-platform","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhoomanesteki%2Fagentic-rag-knowledge-ai-platform/lists"}