{"id":51847105,"url":"https://github.com/inth3shadows/terse","last_synced_at":"2026-07-23T14:01:58.771Z","repository":{"id":367669090,"uuid":"1281836596","full_name":"inth3shadows/terse","owner":"inth3shadows","description":"Lossless-first compression layer for AI-agent tool outputs — selective per-tool, byte-faithful, no decode step.","archived":false,"fork":false,"pushed_at":"2026-07-18T22:54:04.000Z","size":802,"stargazers_count":0,"open_issues_count":2,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-19T00:25:05.363Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/inth3shadows.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-27T01:38:14.000Z","updated_at":"2026-07-18T22:54:08.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/inth3shadows/terse","commit_stats":null,"previous_names":["inth3shadows/terse"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/inth3shadows/terse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inth3shadows%2Fterse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inth3shadows%2Fterse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inth3shadows%2Fterse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inth3shadows%2Fterse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/inth3shadows","download_url":"https://codeload.github.com/inth3shadows/terse/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/inth3shadows%2Fterse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35804545,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-23T02:00:06.683Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-07-23T14:01:58.068Z","updated_at":"2026-07-23T14:01:58.757Z","avatar_url":"https://github.com/inth3shadows.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# terse\n\nThe **lossless-first** MCP compression proxy: it makes tool output smaller without\never changing what your agent reads — byte-faithful by default, lossy only where you\nexplicitly opt in.\n\nterse reduces tokens two ways: one that carries the day-to-day value, and one that is\nharder for a competitor to copy. Keeping those straight is the whole positioning.\n\n**1. The lossless codec — the value.** This is the lever that does the work. terse\nremoves only *structural* overhead: pretty-print whitespace, keys repeated once per\nrecord, repeated values, repeated nested schema. The transformed bytes **are** the\nmodel's input — a denser but still legible representation, not an offload. There is no\ndecode step, no ML model in the loop, and every transform has an exact inverse (a\nround-trip gate asserts `decompress(compress(x)) == x` over the whole corpus). This is\nthe guarantee most tools in this space decline to make: headroom's JSON path is lossless\non uniform arrays but **falls back to dropping rows** on larger/irregular record sets,\nrecoverable only via a `retrieve` round-trip against a cache that expires (verified,\nv0.32.0; default 30-min TTL); Anthropic/OpenAI context-editing **drops** old tool results\nserver-side. terse never silently mutates what the model sees — and \"lossless\" is the\ncategory, not the token count. In terse's own production ledger this codec is where\nessentially all the savings come from (see Status). Its one honest caveat: the\ntabularization primitive is public (formats like [TOON](https://toonformat.dev/) publish\nit standalone, MIT-licensed, ~40% on flat arrays), so a motivated competitor could clone\nthe codec in a weekend.\n\n**2. The stateful cross-call diff — the defensible axis.** When the same tool is called\nagain — poll a list, re-read a file — terse emits a lossless *delta* against the prior\nresult instead of the whole payload (**~73% smaller on the repeated call** in the model\nbelow). This is the one axis a stateless encoder **architecturally cannot reach**: TOON,\nheadroom's stateless per-call compressor, and server-side history-pruning all pay the\nfull column every call because none of them remember the last result — terse can only do\nit because it lives in the session as a transparent proxy. That makes it the harder half\nto copy. But it is a **bonus tier, not the headline**: it only pays off when a workload\nactually repeats a call with a similar-enough payload. That measured ~0.4% of results in\nterse's own 7-day traffic — but most of that was **structural, not workload**: results\narriving as N content blocks were excluded from diffing outright, which was 71% of tokens.\nThe cross-block join (below) removed that exclusion, and across every third-party server\nbenchmarked in BENCHMARKS §6 a repeated call now produces a delta. How often *your* loop\nrepeats a call is still yours to measure (`terse stats`).\nWhen your loop *does* re-fetch mostly-unchanged results it compounds hard; when it\ndoesn't, it costs nothing (lossless, and emitted only when smaller). Default-on since its\nvalidation program completed (see Status).\n\nAround those two sits the **bundle** that turns a byte filter into a control plane you\ndon't want to rip out: MCP-native proxy packaging (transparent to any downstream\nserver, no client-side reformatting), a **live savings ledger** (`terse stats`), a\nfluency-gated lossy escape hatch, and self-installing ops tooling (`install-mcp`,\n`mcp-status`) — each diff/lossy tier validated by a behavioral eval before it was ever\nturned on by default.\n\nIt is **selective by design**. Measurement on real tool output showed the win is\nstrongly per-tool (0–30%): large on record/symbol-shaped verbose output, near-zero\non already-minified or already-projected tools. So terse applies per-tool policy\nrather than compressing everything blindly.\n\n## How It Works\n\nterse transforms a tool's JSON output through a tiered, fully-lossless pipeline,\nthen (optionally) serves it through a per-tool policy that decides which tiers run.\n\n- **Tier 0 — minify**: strip insignificant whitespace.\n- **Tier 0 — tabularize**: a list of uniform records becomes one header + value\n  rows (keys written once, not once per record), recursively hoisting nested\n  uniform-dict columns into a shared header.\n- **Tier 0.5 — dictionary code**: repeated string values *and repeated whole subtrees*\n  are folded into an inline legend (`~0`, `~1`, …) proven disjoint from every literal in\n  the payload. Committed only when it actually saves tokens, so it never regresses.\n- **Tier 0.7 — cross-call diff (stateful, ON by default)**: when the same tool is called\n  repeatedly, the proxy emits a lossless delta against the prior result instead of\n  the full payload (the 91%-overlap headroom). Self-describing, verified to reconstruct\n  exactly, and emitted only when smaller — falls back to the full form otherwise.\n  Default-on since its validation program completed (fluency, nested-record coverage,\n  and the drift soak — see Status); opt out with `proxy --no-diff` / `install-mcp\n  --no-diff` or a policy-file `\"diff\": false`.\n  Record-shaped JSON gets a row/key diff; non-JSON results (file reads, source excerpts,\n  log tails) get a separate content-defined-chunking (CDC) diff — a rolling hash cuts\n  chunk boundaries by content, not position, so an edit anywhere only perturbs the\n  chunk(s) it overlaps and the rest is sent as references to the prior result. Each\n  shape keeps its own diff base per tool.\n- **Cross-block join (ON by default)**: some MCP servers return one record per content\n  block, so each block is a lone object the codec above can barely fold and the diff tier\n  skips entirely (it reasons about one logical payload). When every text block of a result\n  is a JSON object, the proxy joins them into one record array before compressing — so\n  `tabularize`/`dictionary` fold across records *and* the whole result becomes\n  diff-eligible. This changes the number of content blocks the client sees (N → 1), which\n  the MCP spec permits (block count carries no meaning). Opt out with\n  `proxy --no-join-blocks` / `install-mcp --no-join-blocks` or a policy-file\n  `\"join_blocks\": false`; lossy field rules still resolve per block, before the join.\n- **Tier 1 — lossy (opt-in, per field)**: `truncate` caps and annotates a field marked\n  `{\"lossy\":\"truncate\",\"max\":N}`, gated by an acceptable-loss check (only marked,\n  non-`critical` fields may differ, each only as a valid truncation). `drop-to-retrieve`\n  replaces a marked field with a handle, stores the original per session, and serves it\n  back via a synthetic `terse.retrieve` tool the proxy injects — gated so a drop is\n  accepted only if the handle resolves to the exact original. `summarize` (needs a model)\n  is still parsed but deferred — warned and left lossless. Off everywhere by default.\n\nEvery transform has an exact inverse, and a round-trip gate asserts\n`decompress(compress(x)) == x` over the whole corpus. The transformed bytes *are*\nthe model's input — a denser but still-readable representation, not an offload.\n\nThe proxy also keeps a **live savings ledger** (on by default; `--no-stats` to opt\nout): one payload-free JSONL record per result — sizes, tokens, and the decision\ntaken, never content — so `terse stats` can answer \"how much did terse actually save\nme this week?\" from real sessions, not just the synthetic corpus.\n\n## Install\n\nNeeds Python 3.11+.\n\n```bash\nuv tool install terse-mcp   # global `terse` CLI  (or: pipx install terse-mcp)\n```\n\nOr `pip install terse-mcp` into a virtualenv for library/embedded use.\n\n## Quick Start (under a minute)\n\nterse sits between your MCP client and a server and shrinks the server's tool results in\nflight. **No config needed** — the proxy is lossless-everywhere by default:\n\n```bash\n# 1. Wrap ANY stdio MCP server. Your agent talks to it exactly as before;\n#    terse compresses the results it returns, losslessly.\nterse proxy -- uvx some-mcp-server --flags\n\n# 2. See what it saved (the payload-free ledger is on by default):\nterse stats\n```\n\nWant to eyeball the codec first, no server involved?\n\n```bash\necho '[{\"id\":1,\"state\":\"open\",\"repo\":\"acme/widgets\"},{\"id\":2,\"state\":\"open\",\"repo\":\"acme/widgets\"},{\"id\":3,\"state\":\"open\",\"repo\":\"acme/widgets\"},{\"id\":4,\"state\":\"open\",\"repo\":\"acme/widgets\"},{\"id\":5,\"state\":\"open\",\"repo\":\"acme/widgets\"},{\"id\":6,\"state\":\"open\",\"repo\":\"acme/widgets\"}]' | terse gate -\n# → round-trip lossless: PASS ; ~36% fewer cl100k tokens\n```\n\n(Savings grow with record count and repetition; on a single tiny object terse correctly\ndeclines and passes it through unchanged — it never inflates what it can't shrink.)\n\n## Does terse help my server?\n\nThe win is per-tool and terse only keeps what pays, so it never hurts — but it helps a lot\nmore on some shapes than others. Point it at a server and run `terse stats` to see for real;\nas a rule of thumb:\n\n| terse helps most | terse barely moves |\n|---|---|\n| record/array JSON (lists of objects) | already-minified or already-projected output |\n| repeated values or nested repeated subtrees | free-text-dominated results (logs, prose, diffs) |\n| verbose REST-ish payloads (GitHub, Jira, DB rows) | tiny single objects |\n| tools you call repeatedly (cross-call diff) | binary / non-JSON blobs (passed through untouched) |\n\n## Wire it into your MCP client (permanent)\n\n`install-mcp` rewrites your MCP config to launch a server *through* terse — reversible, and\ntransparent to the client. It needs a per-tool policy; the smallest useful one is:\n\n```bash\necho '{\"version\":1,\"defaults\":{\"tiers\":[\"minify\",\"tabularize\",\"dictionary\"]}}' \u003e terse-policy.json\n```\n\n```bash\n# Claude Code, user scope (~/.claude.json) — wrap a server you've already registered by name:\nterse install-mcp --policy terse-policy.json \u003cserver-name\u003e\n\n# Project scope (a committed .mcp.json instead):\nterse install-mcp --policy terse-policy.json --scope project --file .mcp.json \u003cserver-name\u003e\n\nterse mcp-status                       # confirm what's wrapped\nterse uninstall-mcp \u003cserver-name\u003e      # cleanly restore the original entry\n```\n\nOther MCP clients (Cursor, etc.) read the same config shape — wherever a server is launched\nas `cmd --flags`, launch it as `terse proxy -- cmd --flags` to get the same effect.\nSee [USAGE.md](USAGE.md) for tuning a policy (`terse tune`) and reading `terse stats`.\n\n**From source** (contributors): `uv sync` then `uv run terse ...`; `uv run pytest` is the\nlossless gate.\n\n## Project Structure\n\n```\nsrc/terse/\n  transforms.py  lossless tiers (minify, tabularize, dict coding) + round-trip gate\n  policy.py      selective per-tool policy: load, match, apply\n  proxy.py       MCP stdio middleware: compress a downstream server's tool results\n  stats.py       live savings ledger (payload-free) + the `terse stats` aggregation\n  capture.py     corpus capture (shape-tagged envelopes) + shape classifier\n  measure.py     per-payload + cross-tokenizer token measurement\n  probes.py      value-redundancy + cross-call-overlap ceiling probes\n  fluency/       does a model read the compressed form as accurately as raw JSON?\n                 (questions / scoring / answerers / harnesses / pack behind one facade)\n  tokenize.py    cl100k / o200k token counting\n  report.py      markdown reports (savings, per-tool, probes, tokenizer, fluency)\n  html_report.py charted HTML companion (inline SVG, no JS/CDN) for measure/verify\n  cli.py         entrypoint: gate / capture / measure / probe / validate / compress / proxy / stats / fluency\nscripts/\n  gen_stress_corpus.py  synthetic stress corpus for the fluency eval\n  bench/                terse-vs-TOON token benchmark on a real GitHub-API corpus\n                        (fetch_corpus.sh, benchmark.py, diff_demo.py, toon_encode.mjs)\n  bench/mcp_servers/    what terse does, zero-config, to popular third-party MCP servers\n                        (mcp_probe.py harness + pinned repo/web fixtures; BENCHMARKS §6)\ntests/           round-trip, measurement, probe, policy, and fluency tests\npolicy.example.json   selective policy encoding the measured per-tool insight\ncorpus/          captured tool outputs (gitignored; may contain real data)\n```\n\n## Verify it yourself\n\nterse sits in your agent's critical path, so it earns trust by inspection. See\n[VERIFY.md](VERIFY.md) for the full walkthrough — or generate a self-contained\nreport (lossless gate + per-tool token savings) in one command:\n\n```bash\nterse verify --out reports/verify-report.md          # bundled sample, zero setup\nterse verify --corpus corpus --out report.md         # your own captured traffic\nterse verify --html --out reports/verify-report.md   # + a charted HTML report alongside it\nterse verify --corpus corpus --json                  # machine-readable gate + savings (CI-checkable)\n```\n\n## Benchmarks: terse vs alternatives\n\nA head-to-head token-reduction benchmark on a corpus of **real, public GitHub API\npayloads** (`scripts/bench/`) — the nested, record-shaped output that dominates real MCP\ntool traffic. Everything below is **lossless and verified per payload** (a row is dropped\nfrom the total if either encoder fails its round-trip), counted in `cl100k_base` (the same\ntokenizer terse uses). Reproduce end-to-end:\n\n```bash\ncd scripts/bench \u0026\u0026 npm install          # pins the official @toon-format/toon encoder\n./fetch_corpus.sh                        # OR use the committed corpus snapshot as-is\nuv run scripts/bench/benchmark.py        # terse vs TOON vs baselines\nuv run scripts/bench/diff_demo.py        # terse's cross-call diff (its own axis)\n```\n\nThe only directly-comparable public tool is **[TOON](https://toonformat.dev/)** — a\nlossless encoding that shares terse's tabularization primitive. The corpus is compact JSON\n(no pretty-print whitespace), so `minify` saves ~0% and every number below is *pure\nstructural* gain, the hardest honest case:\n\n| payload (real GitHub API) | records | raw tok | terse | TOON |\n|---|--:|--:|--:|--:|\n| gh_pulls | 30 | 151,165 | **76.1%** | −8.4% |\n| gh_workflow_runs | 20 | 76,032 | **80.3%** | −7.5% |\n| gh_issues | 30 | 48,032 | **32.7%** | −8.0% |\n| gh_commits | 30 | 69,652 | **26.5%** | −4.5% |\n| gh_dir_listing | 24 | 6,736 | **31.4%** | −7.7% |\n| gh_rate_limit | 1 obj | 357 | **13.4%** | −36.7% |\n| gh_repo_single | 1 obj | 1,652 | 0.0% | −4.4% |\n| gh_commits_flat | 30 | 10,886 | **2.4%** | 1.7% |\n| gh_labels | 9 | 632 | 15.2% | **19.0%** |\n| **weighted total** | | 365,144 | **58.3%** | **−7.1%** |\n\n*(% = fewer cl100k tokens than raw; higher is better; **bold** = winner.)*\n\n**Honest reading of this:**\n\n- **On real nested API records, terse wins decisively and TOON regresses** (−7.1% overall\n  — *worse* than raw). TOON is built for **flat, uniform arrays**; GitHub records are\n  deeply nested and non-uniform (a PR embeds repeated `user`/`head`/`base`/`repo`\n  subtrees), which terse's dictionary tier folds and TOON's tabular layout cannot — it\n  adds key-path overhead instead. terse's headline 76% on `gh_pulls` is exactly this:\n  60 repeated copies of the same repo object collapsed to one legend entry.\n- **TOON is not beaten everywhere — and the boundary is value repetition, not column\n  width.** On `gh_labels` (9 records × 7 columns — TOON's designed sweet spot) TOON leads,\n  **+19.0% vs terse's +15.2%**. terse's decisive corpus win comes from **nested repeated\n  subtrees and long repeated string values** — its dictionary and subtree-aliasing tiers fold\n  them, TOON's flat tabular layout cannot — which is exactly what real GitHub records carry.\n  On *stripped-flat synthetic tables* with none of that redundancy, the two converge: a seeded\n  column-width sweep (`uv run scripts/bench/width_sweep.py`) shows them within a few points at\n  every width, trading the lead by parity, with **no clean column-count crossover** (an earlier\n  claim of a ≤3/≥4 boundary did not reproduce — see BENCHMARKS.md). So the honest frame is:\n  terse wins where records repeat or nest (real tool output); TOON stays competitive on flat,\n  low-redundancy uniform tables — \"different niche,\" not \"terse strictly dominant.\"\n- **Neither tool helps much when free text dominates** (`gh_commits_flat`: long commit\n  messages, ~2% either way) or on tiny single objects — matching terse's own \"selective,\n  0–30%, per-tool\" claim rather than contradicting it.\n**Cross-call diff — the axis no stateless encoding has.** When the same tool is called\nagain (poll a list, re-read a file), terse emits a lossless *delta* against the prior\nresult instead of the whole payload. TOON, minify, and terse's own single-shot codec all\npay the full column every call. Modeling one repeated call per payload (two records\nchanged, one appended — the poll-again pattern), the **second** call costs\n(`uv run scripts/bench/diff_demo.py`):\n\n| repeated call | records | full re-send (terse) | diff | diff smaller |\n|---|--:|--:|--:|--:|\n| gh_commits_flat | 30 | 10,681 | 812 | **92.4%** |\n| gh_commits | 30 | 51,623 | 6,273 | **87.8%** |\n| gh_issues | 30 | 32,608 | 4,448 | **86.4%** |\n| gh_dir_listing | 24 | 4,779 | 977 | **79.6%** |\n| gh_pulls | 30 | 37,776 | 15,292 | **59.5%** |\n| gh_workflow_runs | 20 | 15,370 | 12,336 | 19.7% |\n| **weighted total** | | 152,837 | 40,138 | **73.7%** |\n\nThe diff cost scales with *what changed*, not with payload size — so its win compounds\nexactly where token cost otherwise does: a long agent loop re-fetching mostly-unchanged\nresults. (`gh_workflow_runs` is lower here only because its records are large and this\nmodel changed a big nested field; a status/timestamp churn would diff far smaller.) This\nis on top of the single-shot reduction above, and stacks with it. These figures are the\ndiff's win *when it fires* on a repeated call; how often that actually happens is a\nseparate question, and workload-dependent — measure yours with `terse stats`. Treat this\nas a defensible bonus tier, not the headline lever (that's the codec — see the\npositioning note at the top and Status).\n\n**Tools not benchmarked head-to-head, and why (no invented numbers):**\n\n| Tool | Why not a like-for-like row |\n|---|---|\n| **headroom** (`headroom-ai`, headroomlabs-ai) | The closest *product* competitor and far more adopted (star figures cited vary, ~29–49k — unverified). But its JSON compressor is a **deterministic Rust transform, not an ML model** (verified, v0.32.0): lossless on uniform arrays, yet **falling back to dropping rows** on larger/irregular record sets, recoverable only via a `retrieve` round-trip against a **time-boxed, backend-dependent cache** (default 30-min TTL; SQLite in the proxy path, in-memory if constructed directly — gone after expiry, eviction, or process exit). A separate, optional text/log compressor *is* ML (a keyless model download). Not comparable on a lossless token axis: terse's guarantee is unconditional — no cache, no TTL, no ML, no egress. (The `headroom` package on PyPI is a different, unrelated CLI.) |\n| **LLMLingua-2** (Microsoft, 6.4k★) | Lossy prompt compression via a trained token-classifier; operates on **input prompts**, not structured tool output. Verified on a JSON payload it strips the syntax (`{`, `}`, `:`, `\"`) as low-information and emits **invalid, unparseable JSON** (and silently truncates past its 512-token window). Different axis entirely. |\n| **Anthropic context editing** / OpenAI equivalents | **Native, server-side, lossy** history-pruning (drop oldest tool results past a threshold), no local artifact to run keylessly. This — not any third-party tool — is the real strategic overlap with terse for first-party API users. |\n| **Atlassian mcp-compressor** (97★) | Primarily compresses tool **schemas/descriptions** at connect time (lossless deferred-disclosure) — complementary and stackable with terse (`terse proxy -- mcp-compressor -- \u003cserver\u003e`). Caveat: an opt-in `--toonify` flag *does* reformat call **results** into TOON, so \"schemas only\" isn't strictly true — but that path is off by default and is a single static pass with no diffing, per-tool policy, or cross-call state, out-competed by terse's codec on that axis. |\n\nAdoption honesty: terse is new (just published to PyPI, few/no stars); TOON (24.9k★) and headroom\n(widely adopted, star figures cited vary ~29–49k) are far more established. terse's\ndefensible wedge is narrow and specific — *unconditionally lossless (no expiring\nretrieve-cache), no ML dependency, MCP-transparent, plus cross-call diffing* — not breadth\nof adoption.\n\n## Related Documentation\n\n- [Benchmarks](BENCHMARKS.md) — dated, reproducible numbers: terse-vs-TOON (§1–2), the\n  cross-call diff axis (§3), competitors (§4), the live production ledger (§5), and\n  popular third-party MCP servers + a repo-size sweep (§6)\n- [Verify it yourself](VERIFY.md) — prove losslessness, savings, and no-egress locally\n- [Technical Reference](TECHNICAL.md) — architecture, pipeline, policy schema, limitations\n- [Usage Guide](USAGE.md) — running the CLI day-to-day and reading its output\n- [Changelog](CHANGELOG.md) — notable changes per release\n\n## Status\n\nA working, measured, selective **lossless** library, CLI, and MCP\nstdio proxy. The proxy's open question — *does a model read the compressed form as\nwell as raw JSON?* — now has a measured answer: on a stress corpus, Claude Haiku 4.5\nand Gemini 2.5 Flash match raw-JSON accuracy on the compressed form (100% paired) at a\n37% token saving (`terse fluency`; see TECHNICAL.md). Whole-subtree aliasing (folding\nrepeated objects, not just strings) is built. Cross-call diffing is a lossless tier\nthat is now **on by default** — its full validation program passed: pair fluency\n(`fluency --diff`, 4-model panel 100%), the nested-record surface (`structure`: diff\n100% vs full-terse 94%), and long-chain drift soaked from both sides — mechanically\n(`tests/test_diff_soak.py` — exact reconstruction hundreds of chained hops deep) and\nbehaviorally (`fluency --diff-soak` — no depth-correlated accuracy loss up to the\nkeyframe bound). Opt out per proxy (`--no-diff`) or per policy (`\"diff\": false`).\nCross-block joining (N content blocks\nfolded into one record array before compressing) is built and on by default — it removed\nthe structural exclusion that kept 71% of real traffic out of the diff tier entirely.\nThe Tier 1 lossy modes `truncate` and\n`drop-to-retrieve` are built (opt-in, off by default); `summarize` remains designed but\nnot yet built — see TECHNICAL.md \"Known Limitations\".\n\nEvidence now spans three kinds: a fixed public corpus (BENCHMARKS §1–4), terse's own\nlive production ledger (§5), and **popular third-party MCP servers** measured zero-config\nwith pinned fixtures (§6) — filesystem, git, memory, fetch, plus serena and\nplaywright-mcp. The §6 headline: the codec pays on JSON output (18–58%, depending on\nwhether the server pretty-prints) while *every* text-shaped tool is 0% one-shot yet still\nwins on a repeat — so the codec is the JSON-specific lever and the diff is the broad,\nshape-independent one.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finth3shadows%2Fterse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Finth3shadows%2Fterse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Finth3shadows%2Fterse/lists"}