{"id":51759111,"url":"https://github.com/mussolene/contextir","last_synced_at":"2026-07-19T11:00:46.372Z","repository":{"id":372162439,"uuid":"1304715435","full_name":"mussolene/contextir","owner":"mussolene","description":"Local-first adaptive context compiler and privacy gateway for LLMs and agents","archived":false,"fork":false,"pushed_at":"2026-07-19T10:39:21.000Z","size":31733,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-19T11:00:19.278Z","etag":null,"topics":["agents","context-compression","llm","privacy","python","semantic-ir"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mussolene.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":"NOTICE","maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-07-18T07:56:52.000Z","updated_at":"2026-07-19T10:39:25.000Z","dependencies_parsed_at":null,"dependency_job_id":"f46163dd-e56f-475f-a8aa-65b98003f630","html_url":"https://github.com/mussolene/contextir","commit_stats":null,"previous_names":["mussolene/contextir"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/mussolene/contextir","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mussolene%2Fcontextir","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mussolene%2Fcontextir/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mussolene%2Fcontextir/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mussolene%2Fcontextir/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mussolene","download_url":"https://codeload.github.com/mussolene/contextir/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mussolene%2Fcontextir/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35650226,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-19T02:00:06.923Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agents","context-compression","llm","privacy","python","semantic-ir"],"created_at":"2026-07-19T11:00:18.898Z","updated_at":"2026-07-19T11:00:46.350Z","avatar_url":"https://github.com/mussolene.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ContextIR\n\nContextIR is a local-first adaptive context compiler and privacy gateway for\nlanguage models. It turns text into a compact, inspectable intermediate\nrepresentation while retaining critical source fragments when a lossy summary\nwould be unsafe.\n\nStatus: **stable 1.0 API**. ContextIR follows semantic versioning for the public\nPython surface and the `contextir.v2` contract. Semantic compression and PII\ndetection still require evaluation on each deployment's data.\n\n## Quick Start\n\n```bash\npython3 -m pip install contextir\nollama pull qwen3:0.6b\ncontextir run --model qwen3:0.6b --text \"Reply with only READY.\"\n```\n\n```python\nfrom contextir import ContextPipeline, OllamaClient\n\npipeline = ContextPipeline(invoke=OllamaClient(\"qwen3:0.6b\"))\nresult = pipeline.run(\n    \"If payment 42 is complete, do not send it again.\",\n    source_lang=\"en\",\n    target_lang=\"en\",\n)\nprint(result.answer)\n```\n\nThe release wheel avoids a source checkout and Git dependency. A pinned Git\ninstall remains available for environments that prefer source builds.\n\nFor a two-minute integration and safe restoration example, see\n[Quick start](docs/QUICKSTART.md).\n\n## Why\n\nLong agent histories and tool outputs consume model context with repeated facts,\nformatting, and sensitive values. ContextIR provides three explicit modes:\n\n- `raw`: masked source text, without protocol overhead;\n- `hybrid`: semantic events plus source fragments containing negation,\n  conditions, numbers, quotes, or protected placeholders;\n- `semantic`: compact events only, for callers that accept lossy compression.\n\n`auto` first distinguishes exhaustive tasks, document retrieval, and\noperational history. Exhaustive tasks stay raw. Document QA uses local lexical\nretrieval to keep the query, formatting instructions, and evidence spans\nverbatim. Operational history continues to use inspectable events and critical\nsource fragments.\n\n`ContextPipeline` is the product entry point. It measures candidate prompts\nwith a configurable target-model tokenizer, rejects compression without useful\nsavings, verifies output safety, and performs bounded fallback from semantic to\nhybrid to raw context. The bundled model clients also reserve output tokens\nand configurable chat-template overhead from their context window; ContextIR\npacks ranked retrieval evidence to the remaining budget or refuses an unsafe\nprompt before contacting the model.\n\nFor a query whose highest-ranked evidence segment is itself larger than the\nwindow, enable bounded chunked retrieval explicitly:\n\n```bash\ncontextir run \\\n  --model qwen3:0.6b \\\n  --context-length 4096 \\\n  --chunked-retrieval \\\n  --text \"Read the document... Question: What is the release code? Answer:\"\n```\n\nThis path chunks only retrieved evidence for reasoning tasks, selects relevant\nchunks locally, validates grounded map outputs, and performs at most one reduce\ncall. It does not approximate exhaustive counting or transform tasks.\n\n## Install\n\nFrom a checkout:\n\n```bash\npython3 -m pip install -e .\n```\n\nOptional Presidio integration:\n\n```bash\npython3 -m pip install -e '.[privacy]'\n```\n\n## Advanced Compiler API\n\n```python\nfrom contextir import ContextIR\n\ngateway = ContextIR()\nbundle = gateway.compile_private(\n    \"If payment 42 is complete, do not send it again. Email person@example.test.\",\n    source_lang=\"en\",\n    target_lang=\"en\",\n    mode=\"hybrid\",\n)\n\ncontract = bundle.contract\nmodel_prompt = gateway.render_prompt(contract)\n\n# Restore only placeholders explicitly allowed by the application.\nanswer = gateway.restore(\"Contact PII_EMAIL_1\", bundle, allowed={\"PII_EMAIL_1\"})\n```\n\n`compile()` returns only the public contract. `compile_private()` additionally\nreturns the local PII vault and source-reference map. Neither is inserted into\nthe model prompt.\n\nFor model-facing applications, prefer `ContextPipeline` over calling the\ncompiler directly. Use `task=\"transform\"` for translation, rewriting, and\nsummarization where numbers, negation, constraints, and placeholders must\nsurvive. Normal reasoning answers are not incorrectly required to echo the\nrequest.\n\n## CLI\n\n```bash\ncontextir run \\\n  --backend ollama \\\n  --model qwen3:0.6b \\\n  --context-length 8192 \\\n  --max-output-tokens 256 \\\n  --prompt-overhead-tokens 32 \\\n  --text \"Summarize the current agent state.\"\n\ncontextir compile \\\n  --text \"Если платеж 42 выполнен, не отправляй его повторно.\" \\\n  --source-lang ru \\\n  --target-lang en \\\n  --mode hybrid \\\n  --out /tmp/contextir.json\n\ncontextir render --contract /tmp/contextir.json\n```\n\nThe source checkout also exposes `python3 scripts/contextir.py`.\n\n## Contract\n\nContextIR v2 records:\n\n- source and target language;\n- intent and confidence;\n- typed entities and protected placeholders;\n- events with predicate, arguments, polarity, modality, condition, and source\n  reference;\n- execution and privacy constraints;\n- optional lexical concepts;\n- included source fragments and unresolved source references;\n- prompt-size and latency statistics.\n\nThe packaged JSON Schema is available through:\n\n```python\nfrom contextir.schemas import load_contract_schema\n```\n\n## Privacy\n\nThe default detector is deliberately small and recognizes common emails,\nphones, payment-card-like values, and API-key-like tokens. Presidio can be used\nas an optional detector and extended with project-specific recognizers.\n\nAutomated PII detection is not a security boundary by itself. Keep the vault\nlocal, avoid logging prompt bodies, use explicit restoration allowlists, and\nevaluate recognizers on data representative of the deployment.\n\n## Current Evidence\n\nThe checked-in compiler smoke benchmark reports:\n\n- 9 compiler and 10 product-pipeline cases;\n- 0 expectation failures;\n- 0 pipeline failures;\n- 0 PII leaks into public contracts or rendered prompts;\n- `0.3627` prompt/source ratio on the one compression-eligible repeated-context\n  case.\n\nThe first model-level A/B exposed a large quality loss in v0.3.0. Query-aware\nrouting corrected that failure. On the v0.5.0 nine-case Qwen3 8B run, raw and\nauto both scored `0.7272`, while auto reduced model input by 70.0% and mean\nlatency from `53.8 s` to `2.51 s`. An external 1,500-row Presidio Research\nbenchmark measured `0.8471` precision and `1.0000` recall for the dependency-free\nemail/phone/card profile. These are bounded results, not universal quality or\nprivacy claims; broader official tasks and deployment-specific PII evaluation\nremain required.\n\nA constrained-window retrieval smoke run packs the same long input to fit a\n256-token Ollama context. Qwen3 0.6B and 8B both recovered `cobalt-seven`; the\nbackend reported 229 prompt tokens with 16 tokens reserved for output.\n\nAn oversized single-segment follow-up uses bounded chunked retrieval with a\n256-token context. Qwen3 0.6B and 8B both recovered `cobalt-seven` from the\nsame selected chunk using one map call at 75% of the available prompt budget.\n\nA constrained 2K-context A/B over ten official LongBench QA/retrieval examples\nreduced measured backend input to 39.0% of raw for both tested models. Qwen3 8B\nquality increased from `0.1668` to `0.6983` with a paired 95% bootstrap delta\nof `[0.2086, 0.8225]`. Qwen3 0.6B increased from `0.1889` to `0.3121`, but its\ndelta interval `[-0.0154, 0.3392]` does not establish a reliable quality gain.\nAll ten official cases used one packed direct call; a separate synthetic\noversized-segment diagnostic activated the map path and recovered the answer\nwith both models.\n\nSee the [local model A/B card](docs/benchmarks/LOCAL_MODEL_AB.md) and\n[benchmark roadmap](docs/BENCHMARKS.md).\n\n## Research Layer\n\nThe repository retains the earlier RU/EN lexical graph, graph embeddings,\nsynthetic translator, checkpoints, and roundtrip experiments. They are not\nloaded by the default package path. Use the `research` extra and source checkout\nfor those experiments.\n\nThe old WordNet keyword roundtrip is preserved as a baseline, not presented as\nthe production compiler. See [RESEARCH_HISTORY.md](docs/RESEARCH_HISTORY.md).\n\n## Development\n\n```bash\npython3 -m pip install -e '.[dev,research]'\npython3 -m unittest discover -s tests -v\npython3 scripts/evaluate_contextir.py\npython3 -m build\npython3 -m twine check dist/*\n```\n\nArchitecture, scope, contribution, and security guidance:\n\n- [Architecture](docs/ARCHITECTURE.md)\n- [Quick start](docs/QUICKSTART.md)\n- [Product scope](docs/PRODUCT_SCOPE.md)\n- [Benchmarks](docs/BENCHMARKS.md)\n- [Contributing](CONTRIBUTING.md)\n- [Security](SECURITY.md)\n\n## License\n\nContextIR code is licensed under Apache-2.0. Research datasets and derived\nlexical artifacts retain their upstream terms. See\n[THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md) before redistributing them.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmussolene%2Fcontextir","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmussolene%2Fcontextir","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmussolene%2Fcontextir/lists"}