{"id":50312442,"url":"https://github.com/sathviknayak123/lexpilot","last_synced_at":"2026-05-28T22:01:34.651Z","repository":{"id":355061456,"uuid":"1191770690","full_name":"SathvikNayak123/LexPilot","owner":"SathvikNayak123","description":"Multi-agent RAG system for Indian Supreme Court judgments with citation verification, knowledge graph traversal, and adversarial guardrails. Built on OpenAI Agents SDK with native agent handoffs.","archived":false,"fork":false,"pushed_at":"2026-05-01T16:15:58.000Z","size":329,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-01T18:19:51.249Z","etag":null,"topics":["agents-sdk","docker","fastapi","knowledge-graph","neo4j","openai","postgres","rag","ragas-evaluation"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/SathvikNayak123.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":"citation/__init__.py","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-03-25T15:13:13.000Z","updated_at":"2026-05-01T16:16:02.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/SathvikNayak123/LexPilot","commit_stats":null,"previous_names":["sathviknayak123/lexpilot"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/SathvikNayak123/LexPilot","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SathvikNayak123%2FLexPilot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SathvikNayak123%2FLexPilot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SathvikNayak123%2FLexPilot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SathvikNayak123%2FLexPilot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/SathvikNayak123","download_url":"https://codeload.github.com/SathvikNayak123/LexPilot/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/SathvikNayak123%2FLexPilot/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33627941,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-28T02:00:06.440Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agents-sdk","docker","fastapi","knowledge-graph","neo4j","openai","postgres","rag","ragas-evaluation"],"created_at":"2026-05-28T22:01:33.720Z","updated_at":"2026-05-28T22:01:34.640Z","avatar_url":"https://github.com/SathvikNayak123.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LexPilot — AI Legal Intelligence for Indian Law\n\nMulti-agent RAG system for Indian Supreme Court judgments with citation verification, knowledge graph traversal, and adversarial guardrails. Built on OpenAI Agents SDK with native agent handoffs.\n\n---\n\n## Architecture\n\n```\n  User Query\n      |\n      v\n  [Input Guardrail]  injection / off-topic / advice classifier\n      |\n      v\n  LexPilot Orchestrator\n      |---\u003e Contract Analyst\n      |---\u003e Precedent Researcher\n      |---\u003e Compliance Auditor\n      |---\u003e Risk Scorer\n      |\n      v\n  GraphRAG Retrieval\n      Dense (SBERT) + Sparse (BM25) -\u003e RRF -\u003e Cross-Encoder rerank\n      -\u003e Neo4j enrichment (authority / citation / overruled)\n      |\n      v\n  [Output Guardrail]  3-Tier Citation Verifier\n      Tier 1: EXISTS   Tier 2: NOT OVERRULED   Tier 3: NLI\n```\n\n---\n\n## Key Features\n\n### Hybrid Retrieval\n- **Dense search** via `all-mpnet-base-v2` sentence embeddings stored in Qdrant\n- **Sparse search** via BM25 (rank-bm25, corpus-wide IDF, pickle-cached)\n- **RRF fusion** (k=60) combining both ranked lists by position\n- **Cross-encoder reranking** (`ms-marco-MiniLM-L-6-v2`, top-20 candidates)\n- **Parent-child chunking** — child chunks indexed for precision, parent chunks fetched for LLM context\n- Achieves **93% Recall@5** and **85% MMR** on 100-query benchmark\n\n### Legal Knowledge Graph (Neo4j)\n- Judgment nodes with typed citation edges: `CITES`, `OVERRULES`, `DISTINGUISHED_FROM`, `APPLIED`\n- Edge types classified via zero-shot **InLegalBERT**, falls back to regex if model unavailable\n- Graph re-ranker applies multiplicative boosts: court authority (SC \u003e HC \u003e District) × citation importance × overruled penalty (0.1×)\n- Surfaces overruled-judgment warnings and citation-chain context at query time\n\n### 3-Tier Citation Verification\nEliminates hallucinated citations before the response leaves the system:\n\n| Tier | Check | Mechanism |\n|------|-------|-----------|\n| 1 | Citation exists | Lookup in `citation_index` with format normalization (11+ SCC/AIR/SCR variants) |\n| 2 | Not overruled | `is_overruled` flag populated from Neo4j OVERRULES edges |\n| 3 | Accurately characterized | NLI (entailment ≥ contradiction) + cosine similarity ≥ 0.50 |\n\n- **79% citation accuracy** on adversarial benchmark (fabricated citations, overruled precedents, mischaracterization)\n- Fabricated citations replaced inline; overruled citations flagged `[OVERRULED]`; mischaracterizations appended with correction\n\n### DPDP Act 2023 Compliance Auditor\n- Exhaustive map-reduce clause scanning via Instructor-validated structured outputs\n- Rejects hallucinated section numbers via schema validation\n- **74% compliance recall** on 12 annotated privacy policies\n\n### Guardrails\n- **Input**: Classifier agent (4-way: `legal_query` | `legal_advice_request` | `injection` | `off_topic`)\n  - Injection and off-topic queries are hard-blocked\n  - Legal advice requests are reframed as informational rather than blocked\n- **Output**: Citation verifier tripwire — fabricated citations abort the response and trigger regeneration\n\n---\n\n## Project Structure\n\n```\nlexpilot_agents/          # Agent definitions + guardrails\n  orchestrator.py         # Master agent with 4 specialist handoffs\n  contract_agent.py       # Clause extraction \u0026 contract analysis\n  precedent_agent.py      # Case law research \u0026 citation chain tracing\n  compliance_agent.py     # DPDP Act 2023 auditing\n  risk_agent.py           # Legal risk scoring\n  guardrails/\n    input_guardrails.py   # Injection / off-topic filter\n    output_guardrails.py  # Citation verification tripwire\n\nretrieval/\n  hybrid_search.py        # Dense + Sparse → RRF → Rerank → Parent hydration\n  qdrant_store.py         # Dual-vector Qdrant client (dense + sparse)\n  bm25.py                 # BM25 sparse encoder (rank-bm25 wrapper)\n  indexer.py              # Full indexing pipeline\n\nknowledge_graph/\n  graph_builder.py        # Builds Neo4j judgment graph from parsed docs\n  graph_retriever.py      # GraphRAG: vector results + batched Neo4j enrichment\n  neo4j_client.py         # Async Neo4j operations\n\ncitation/\n  extractor.py            # Regex citation extractor (11 Indian reporter formats)\n  verifier.py             # 3-tier verification with batched DB lookup\n\ningestion/\n  pdf_parser.py           # PyMuPDF (text) + pdfplumber (tables) dual-engine parser\n  document_processor.py   # Async orchestration: parse → LLM citation extraction\n  models.py               # ParsedBlock, ParsedDocument pydantic models\n\nevaluation/\n  benchmark.py            # 50+ landmark SC cases: retrieval, QA, precision tests\n  ragas_eval.py           # RAGAS suite: faithfulness, answer_relevancy, context_precision\n  adversarial.py          # 25-case adversarial suite (injection, fabrication, overruled)\n\nscripts/\n  build_bm25_index.py     # Corpus-wide BM25 index builder (2-pass, O(corpus))\n  populate_citation_metadata.py  # Sync Neo4j overruled status → Postgres citation_index\n  diagnose_graphrag.py    # Cross-validate Neo4j ↔ Qdrant node counts and edge stats\n\napi/routes/chat.py        # FastAPI SSE streaming endpoint\nfrontend/                 # Next.js UI with streaming chat + confidence banners\n```\n\n---\n\n## Quick Start\n\n### Prerequisites\n- Python 3.11+\n- Node.js 18+\n- Running instances of: **Qdrant**, **Neo4j**, **PostgreSQL**, **Redis**\n\n### 1. Install dependencies\n```bash\npip install -e \".[dev]\"\ncd frontend \u0026\u0026 npm install\n```\n\n### 2. Configure environment\n```bash\ncp .env.example .env\n# Set: OPENROUTER_API_KEY, QDRANT_URL, NEO4J_URI, DATABASE_URL, REDIS_URL\n```\n\n### 3. Ingest documents\n```bash\n# Parse PDFs, chunk, embed, and index into Qdrant + Postgres\npython -m ingestion.document_processor --dir data/judgments/\n\n# Build BM25 sparse index from the indexed corpus\npython scripts/build_bm25_index.py\n\n# Build Neo4j knowledge graph\npython scripts/build_semantic_graph.py\n\n# Populate citation metadata (overruled status, holding summaries)\npython scripts/populate_citation_metadata.py\n```\n\n### 4. Start the API\n```bash\nuvicorn api.main:app --reload --port 8000\n```\n\n### 5. Start the frontend\n```bash\ncd frontend \u0026\u0026 npm run dev\n```\n\n---\n\n## Evaluation\n\n### Retrieval \u0026 QA benchmark\n```bash\npython -m evaluation.benchmark\n# Measures Recall@5, MMR across 50+ landmark SC judgments\n```\n\n### RAGAS generation quality\n```bash\npython -m evaluation.ragas_eval\n# Faithfulness, answer_relevancy, context_precision, context_recall\n```\n\n### Adversarial citation test suite\n```bash\npython -m evaluation.adversarial\n# 25 test cases: prompt injection, fabricated citations, overruled precedents,\n# legal advice seeking, off-topic queries\n```\n\n### Diagnose graph coverage\n```bash\npython scripts/diagnose_graphrag.py\n# Cross-validates Neo4j node counts vs Qdrant, reports edge counts per type\n```\n\n---\n\n## Benchmark Results\n\n| Metric | Score | Notes |\n|--------|-------|-------|\n| Recall@5 | 93% | 100-query retrieval benchmark |\n| MMR | 85% | Diversity-adjusted relevance |\n| Citation accuracy | 79% | Adversarial fabrication benchmark |\n| DPDP compliance recall | 74% | 12 annotated privacy policies |\n\n---\n\n## Tech Stack\n\n| Layer | Technology |\n|-------|------------|\n| Agent framework | OpenAI Agents SDK (native handoffs) |\n| LLM routing | OpenRouter (Gemini 3 Flash) |\n| Vector store | Qdrant (dual dense+sparse vectors) |\n| Sparse retrieval | BM25 via rank-bm25 |\n| Reranking | cross-encoder/ms-marco-MiniLM-L-6-v2 |\n| Knowledge graph | Neo4j 5.x |\n| NLI (citation check) | cross-encoder/nli-deberta-v3-xsmall |\n| Legal NER | InLegalBERT (typed relation extraction) |\n| Database | PostgreSQL + SQLAlchemy async |\n| Structured outputs | Instructor |\n| API | FastAPI + SSE streaming |\n| Frontend | Next.js 15 + Tailwind 4 |\n| Observability | structlog + Langfuse |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsathviknayak123%2Flexpilot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsathviknayak123%2Flexpilot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsathviknayak123%2Flexpilot/lists"}