{"id":50734016,"url":"https://github.com/tailscale/tsheadroom","last_synced_at":"2026-06-10T12:00:40.519Z","repository":{"id":363736392,"uuid":"1263078057","full_name":"tailscale/tsheadroom","owner":"tailscale","description":"\"Headroom\" context compression layer as wrapped in an Aperture pre-request hook","archived":false,"fork":false,"pushed_at":"2026-06-10T05:27:57.000Z","size":169,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-10T07:20:50.519Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tailscale.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-08T15:46:17.000Z","updated_at":"2026-06-09T22:30:47.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tailscale/tsheadroom","commit_stats":null,"previous_names":["tailscale/tsheadroom"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/tailscale/tsheadroom","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Ftsheadroom","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Ftsheadroom/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Ftsheadroom/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Ftsheadroom/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tailscale","download_url":"https://codeload.github.com/tailscale/tsheadroom/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tailscale%2Ftsheadroom/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34151276,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-10T02:00:07.152Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-06-10T12:00:23.227Z","updated_at":"2026-06-10T12:00:40.509Z","avatar_url":"https://github.com/tailscale.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tsheadroom\n\nTransparent, org-wide LLM context compression for [Aperture](https://github.com/tailscale/aperture).\n\nCoding agents and RAG pipelines send the model the same bulky, low-value payloads over and over: multi-thousand-line tool outputs, search dumps, build logs, file listings. You pay for every one of those tokens on every turn, and they crowd out the context window. tsheadroom strips that waste out *before* the request reaches the provider, for every team in your org at once, with no client, SDK, or app changes. It plugs into Aperture as a `pre_request` hook.\n\nIt runs [Headroom](https://github.com/chopratejas/headroom)'s `compress()` function, which crushes bulky content while leaving prompts, instructions, and recent turns intact. If there's nothing worth compressing (or anything goes wrong), the request passes through unchanged. tsheadroom can never block or break a request.\n\nIf you run Aperture, you already run a [Tailscale](https://tailscale.com/) tailnet. tsheadroom joins that tailnet as a device (using [`tsnet`](https://tailscale.com/kb/1244/tsnet)) and Aperture calls out to it. No public endpoint, no API keys: access is gated by your tailnet's [grants](https://tailscale.com/docs/features/access-control/grants).\n\n## What you get\n\nOn a representative tool-heavy request (a 400-entry file listing returned as a tool result):\n\n| | Before | After |\n|---|---|---|\n| The bulky tool result | ~41 KB | ~25 KB (~40% smaller) |\n| Full request body | ~47 KB | ~25 KB |\n| The user's prompt, system prompt, recent turns | n/a | untouched |\n\n- **Token + cost savings** scale with how much tool/log/search output your traffic carries. Chat-only and short requests are left alone (see [What gets compressed](#what-gets-compressed)).\n- **Transparent.** Enforced at the Aperture grant level. Users and apps don't change anything and don't even know it's there.\n- **Fail-safe by design.** If it's unreachable, slow, or crashed, tsheadroom always answers `allow` and the original request proceeds. It is structurally incapable of blocking a request.\n\n## What it costs you\n\nDecide whether this fits before you wire it in:\n\n- **Memory**: text compression uses an ML model held resident in each worker (~600 MB/worker; ~4.8 GB at the default pool size of 8). Tool-output-only compression (the base install) needs far less. Size `-pool-size` deliberately.\n- **Latency**: warm requests add single-digit milliseconds. The *first* request after startup can block while the ML model loads (and, on a fresh host, downloads). That one-time load is ~60s; see [ML model loading](#ml-model-loading-and-timeouts). You set the ceiling through Aperture's hook `timeout`.\n- **Ops**: one long-lived process running as a tailnet device, ideally under `systemd`. Targets Linux (see [Requirements](#requirements)).\n- **Prompt cache**: compressing *historical* context changes the cached prefix, which can cause a prompt-cache miss on the next turn. See the [note below](#hook-response-format).\n\n## How it works\n\n```\nAperture --(pre_request hook: POST /)--\u003e tsheadroom (tsnet :80)\n                                              │\n                                              ├─ pool of persistent Python workers\n                                              │     each runs: headroom.compress(messages, model, **config)\n                                              │\n                                              └─ reply: {\"action\":\"modify\",\"request_body\":{…}}\n                                                     or  {\"action\":\"allow\"}\n```\n\nThe tsheadroom Go binary owns a small pool of long-lived Python worker processes (so the Headroom pipeline is built once, not per request) and supervises them (auto-restart on crash, fail-open on timeout). Compression runs out-of-process, so a slow or faulty `compress()` call can't take down the listener.\n\nFor each request Aperture forwards, tsheadroom hands the `messages` array to `compress()` and returns a `modify` action with the rewritten body, or `allow` if there's nothing to gain or anything goes wrong.\n\n## Requirements\n\n**On the Aperture side:**\n\n- A running Aperture instance with at least one LLM provider configured, and access to edit its configuration (`config.hujson`) and/or your tailnet policy file.\n- A Tailscale tailnet (you have one if you run Aperture).\n\n**On the host that runs tsheadroom:**\n\n- **Linux**, for the cloud hosts this is meant to run on. The binary also builds and runs on macOS, which is convenient for local development and testing.\n- **Go 1.26.4+** to build the binary (required by the pinned Tailscale dependency).\n- **Python with `headroom-ai` installed.** Use a version `headroom-ai` ships a prebuilt wheel for — Python 3.10–3.13 as of this writing. The newest Python release often has no wheel yet (3.14 at the time of writing), and pip then falls back to a Rust source build that fails. If unsure which versions are covered, check `Requires: Python` and the wheel filenames on [headroom-ai's PyPI page](https://pypi.org/project/headroom-ai/).\n- **A Tailscale [auth key](https://tailscale.com/docs/features/access-control/auth-keys)** (`TS_AUTHKEY`) so the device can join your tailnet unattended. Generate one in the Tailscale admin console under **Settings** \u003e **Keys**.\n\n### Choose tool-output or text compression\n\n`headroom-ai` has two compression paths, and which extras you install decides what actually compresses:\n\n| You install | What compresses | Memory | Rust toolchain |\n|---|---|---|---|\n| `headroom-ai` (base) | Tool outputs (SmartCrusher, pure Python) | low | not needed |\n| `headroom-ai[ml]` | Tool outputs + text/prose (Kompress, ML) | ~600 MB/worker | not needed |\n\nInstall `[ml]` if you want text/prose compression, or if you want the default configuration to work as advertised: `compress_system_messages` is on by default and Headroom also uses the ML model as its fallback for mixed content, so under defaults the model is exercised by ordinary traffic. With only the base wheel, the text-targeting knobs (`compress_user_messages`, `target_ratio`) change routing but produce no savings: only tool-output compression is active. See [Tune compression](#tune-compression-runtime-config).\n\n## Install\n\n```shell\n# HTTPS (no SSH key needed):\ngit clone https://github.com/tailscale/tsheadroom.git\ncd tsheadroom\nmake                      # produces ./build/tsheadroom (Linux)\n```\n\nCross-compiling from a Mac for a Linux host:\n\n```shell\nGOOS=linux GOARCH=amd64 go build -o build/tsheadroom .   # or GOARCH=arm64\n```\n\nSet up the Python interpreter (a virtualenv is recommended). `headroom-ai` ships prebuilt wheels on supported Python versions, so no Rust toolchain is needed:\n\n```shell\npython3.13 -m venv /opt/tsheadroom/venv                   # a supported version, called explicitly (3.13 shown)\n/opt/tsheadroom/venv/bin/python --version                 # confirm it matches a version with a wheel\n/opt/tsheadroom/venv/bin/pip install 'headroom-ai[ml]'    # or just 'headroom-ai' for tool-output only\n```\n\nName the interpreter explicitly (`python3.13`, or whichever supported version you have) rather than using bare `python3`, which may resolve to a release too new to have a wheel — common on an up-to-date Homebrew or a pyenv default. A venv on an unsupported version makes the `pip install` fail trying to build `headroom-ai` from Rust source (see [Troubleshooting](#install-fails-building-headroom-ai)). If you do not have a supported interpreter, install one with `brew install python@3.13`, `pyenv install 3.13`, or your platform's package manager — substituting the version number for whatever `headroom-ai` currently ships wheels for.\n\nCopy the worker script next to wherever you'll run the service:\n\n```shell\ncp worker.py /opt/tsheadroom/worker.py\n```\n\nYou now have the three things tsheadroom needs at runtime: the binary, a Python interpreter with `headroom-ai`, and `worker.py`.\n\n### Install fails building headroom-ai\n\nIf `pip install 'headroom-ai[ml]'` fails with `maturin failed` and an error like:\n\n```\nerror: the configured Python interpreter version (3.14) is newer than PyO3's maximum supported version (3.13)\n```\n\nyour virtualenv is on a Python release newer than `headroom-ai`'s latest published wheel (the exact versions in the message will change over time). With no matching wheel, pip tries to compile `headroom-ai` from Rust source and PyO3 rejects the build. Confirm the version with `/opt/tsheadroom/venv/bin/python --version`. To fix it, delete the venv and recreate it with a supported interpreter (3.13 shown here — use whatever [headroom-ai](https://pypi.org/project/headroom-ai/) currently ships a wheel for):\n\n```shell\nrm -rf /opt/tsheadroom/venv\npython3.13 -m venv /opt/tsheadroom/venv\n/opt/tsheadroom/venv/bin/python --version                 # confirm a supported version\n/opt/tsheadroom/venv/bin/pip install 'headroom-ai[ml]'\n```\n\nA correct install downloads prebuilt `.whl` files (their names contain the Python tag, e.g. `cp313`); if you instead see `Building wheel for headroom-ai` or `maturin`, you are still on an unsupported Python version.\n\n## Run\n\n```shell\nTS_AUTHKEY=tskey-auth-xxxx ./build/tsheadroom \\\n  -hostname tsheadroom \\\n  -python /opt/tsheadroom/venv/bin/python \\\n  -worker /opt/tsheadroom/worker.py \\\n  -state-dir /var/lib/tsheadroom \\\n  -config /var/lib/tsheadroom/tsheadroom.config.json\n```\n\nThis joins the tailnet as `tsheadroom` and listens for hook calls on `:80` at path `/`. Other tailnet devices (including Aperture) reach it at `http://tsheadroom.\u003cyour-tailnet\u003e.ts.net/` (find `\u003cyour-tailnet\u003e`, your [tailnet name](https://tailscale.com/kb/1217/tailnet-name), in the Tailscale admin console).\n\n\u003e **First run on a fresh host**: with `[ml]` installed, the workers load (and on a brand-new host, download ~600 MB of model from HuggingFace) at startup. The very first compression request can block for up to ~60s while that happens; after that, requests are milliseconds. To avoid a slow cold start in production, warm the cache once before serving traffic, or set `HF_TOKEN` (see [ML model loading](#ml-model-loading-and-timeouts)).\n\n`TS_AUTHKEY` is only needed on first start. Once `-state-dir` is populated you can omit it on restarts.\n\n### Run it as a service\n\nFor durability, and for the cloud/Linux hosts this is meant for, wrap it in a service manager. A minimal `systemd` unit:\n\n```\n[Unit]\nDescription=tsheadroom (Aperture context-compression hook)\nAfter=network-online.target\nWants=network-online.target\n\n[Service]\nExecStart=/opt/tsheadroom/tsheadroom \\\n  -python /opt/tsheadroom/venv/bin/python \\\n  -worker /opt/tsheadroom/worker.py \\\n  -state-dir /var/lib/tsheadroom \\\n  -config /var/lib/tsheadroom/tsheadroom.config.json\nEnvironment=TS_AUTHKEY=tskey-auth-xxxx\nStateDirectory=tsheadroom\nRestart=always\n\n[Install]\nWantedBy=multi-user.target\n```\n\n`StateDirectory=tsheadroom` provisions `/var/lib/tsheadroom` with the right ownership. After the device has authenticated once, you can drop the `TS_AUTHKEY` line.\n\n### Flags\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `-python` | `python3` | Interpreter with `headroom-ai` installed (used to launch workers). |\n| `-worker` | `worker.py` | Path to the worker script. |\n| `-hostname` | `tsheadroom` | Device name on the tailnet. |\n| `-pool-size` | `8` | Number of persistent Python workers. Each holds a resident copy of the ML model (~600 MB) when text compression is active (~4.8 GB at 8), so raise it deliberately. |\n| `-max-compress` | `60s` | Hard cap on a single worker call before the worker is recycled; the sole worker-side timeout. Covers one-time ML model loads. |\n| `-addr` | `:80` | Listen address on the tsnet device. |\n| `-state-dir` | tsnet default | tsnet state directory. Persist this: it holds the device identity (`tailscaled.state`). See note below. |\n| `-config` | `tsheadroom.config.json` | Path to the tunable compression configuration file. Loaded at startup, rewritten on `PUT /config`. |\n| `-local-addr` | off | Serve plain HTTP here instead of tsnet, for local testing only. |\n| `-v` | off | Log a one-line per-request summary to stdout. See [Verify](#verify-its-working). |\n\n\u003e **State (`-state-dir`)**: must be a stable, writable, persistent path. It contains `tailscaled.state`, the device's private key. Treat it as a secret and keep it on durable storage, or the device re-authenticates as a *new* device on every restart.\n\u003e\n\u003e **Config path (`-config`)**: the default is resolved against the current working directory. In a service, always set an absolute path (for example, under your state dir) so a different launch CWD doesn't silently read/write a different file.\n\n## Wire it into Aperture\n\ntsheadroom is configured as a `pre_request` hook in two places in your Aperture configuration (`config.hujson`).\n\n### 1. Define the hook\n\nAdd it to the top-level `hooks` map, pointing at your tsheadroom device:\n\n```json\n\"hooks\": {\n  \"headroom\": {\n    \"url\": \"http://tsheadroom.\u003cyour-tailnet\u003e.ts.net/\",\n    \"fail_policy\": \"fail_open\", // if tsheadroom is unreachable, send uncompressed\n    \"timeout\": \"30s\",           // tsheadroom's only client-facing latency ceiling\n  },\n},\n```\n\n| Field | Required | Notes |\n|---|---|---|\n| `url` | yes | Your tsheadroom device's tailnet URL, path `/`. |\n| `fail_policy` | no (default `fail_open`) | Keep `fail_open`. See below. |\n| `timeout` | no (default `5s`) | Raise to `30s`. This is the entire latency budget; the `5s` default will cut off cold/large compressions. |\n| `preference` | no (default `0`) | Set only if you stack tsheadroom with other hooks. Higher runs first; ties break alphabetically. A `modify` is visible to later hooks. |\n\ntsheadroom needs no `apikey` or `authorization`. It's reached only over your tailnet and gated by your tailnet's [grants](https://tailscale.com/docs/features/access-control/grants).\n\n### 2. Attach it to a grant\n\nA hook does nothing until a grant references it through `send_hooks`:\n\n```json\n{\n  \"src\": [\"*\"],\n  \"app\": {\n    \"tailscale.com/cap/aperture\": [\n      {\n        \"models\": \"**\",            // compress all models, or scope, for example \"anthropic/**\"\n        \"send_hooks\": [\n          {\n            \"name\": \"headroom\",\n            \"events\": [\"pre_request\"],\n            \"send\": [\"request_body\"],\n          },\n        ],\n      },\n    ],\n  },\n},\n```\n\n\u003e [!NOTE]\n\u003e Where you put this grant changes the `dst` rule. The example above is the Aperture configuration file (`config.hujson`) form, where `dst` is not used, so omit it. If you instead place the grant in your [tailnet policy file](https://tailscale.com/docs/reference/syntax/policy-file), you must add an explicit `dst` (for example, `\"dst\": [\"tag:aperture\"]`, where `tag:aperture` is a [tag](https://tailscale.com/docs/features/tags) you've applied to your Aperture device). A tag gives a non-user device a stable identity at authentication time and doubles as a selector you can target in grants, so it's the durable way to name your Aperture device here. Omitting `dst` makes the grant silently apply to nothing and your hook never fires.\n\nNotes:\n\n- `send: [\"request_body\"]` is required and is the only input tsheadroom uses. A `modify` hook replaces the *entire* request body, so Aperture must send it. (The model name is read from inside the body; tsheadroom doesn't need `user_message`; it operates on the whole `messages` array.)\n- Keep `fail_open`. tsheadroom always answers `200` with `allow`/`modify` and never blocks; `fail_open` ensures that even if the device is *unreachable* (or a compression runs past the `timeout`), requests still proceed (uncompressed). `fail_closed` would instead block with HTTP 503, not what you want here.\n- The hook `timeout` is the only client-facing latency ceiling. tsheadroom has no soft timeout of its own: it waits for the compression and returns, bounded only by this `timeout` (after which Aperture forwards the original) and by its internal `-max-compress` worker cap. Set it to how long you're willing to make a request wait. `30s` matches Headroom's own compression budget and comfortably fits large, late-session requests. There's no separate tsheadroom knob to keep in sync.\n- Scope `models` (FQN glob, for example `\"anthropic/**\"`, or `\"**\"` for everything) if you don't want to compress all providers.\n\nFull hook/grant field reference: [Aperture configuration documentation](https://tailscale.com/docs/aperture/configuration).\n\n### Hook response format\n\ntsheadroom returns one of two [`GuardrailResponse`](https://tailscale.com/docs/aperture/configuration) actions, always with HTTP `200`:\n\n```json\n{ \"action\": \"allow\" }\n```\n\nNothing changed. The original request proceeds. Returned for short/chat-only requests, bodies with no `messages` array, errors, or timeouts.\n\n```json\n{ \"action\": \"modify\", \"request_body\": { \"model\": \"…\", \"messages\": [ … ] } }\n```\n\nThe `messages` array was compressed; every other field of the body is preserved unchanged. The modified body replaces the original wholesale.\n\ntsheadroom never returns `\"block\"`.\n\n\u003e [!NOTE]\n\u003e Prompt-cache interaction. tsheadroom compresses *older* context (it protects the most recent turns), so a `modify` changes the cached prefix and can trigger a prompt-cache miss on the next turn. For tool/log-heavy traffic the token savings typically dominate; for cache-heavy chat workloads, weigh this and scope `models`/knobs accordingly.\n\n## Verify it's working\n\n### 1. Locally, before touching the tailnet\n\nRun with `-local-addr` to serve plain HTTP (no tailnet) and `-v` for per-request logging:\n\n```shell\n./build/tsheadroom -local-addr 127.0.0.1:8080 -v \\\n  -python /opt/tsheadroom/venv/bin/python -worker ./worker.py\n```\n\nSend a request with a large tool result, exactly the shape Headroom is built to compress:\n\n```shell\npython3 - \u003c\u003c'PY' | curl -s -X POST http://127.0.0.1:8080/ -H 'Content-Type: application/json' -d @-\nimport json\nbig = json.dumps([{\"id\": i, \"path\": f\"/src/m{i}.py\", \"status\": \"unchanged\",\n                   \"hash\": \"deadbeef\"*4} for i in range(400)])\nprint(json.dumps({\"request_body\": {\"model\": \"claude-sonnet-4-5-20250929\", \"messages\": [\n    {\"role\": \"user\", \"content\": \"list files\"},\n    {\"role\": \"assistant\", \"content\": [{\"type\": \"tool_use\", \"id\": \"t1\", \"name\": \"ls\", \"input\": {}}]},\n    {\"role\": \"user\", \"content\": [{\"type\": \"tool_result\", \"tool_use_id\": \"t1\", \"content\": big}]},\n    {\"role\": \"user\", \"content\": \"summarize\"}]}}))\nPY\n# -\u003e {\"action\":\"modify\",\"request_body\":{…}}\n```\n\n\u003e The first request may take ~60s (the `[ml]` model is loading/downloading). It isn't hung. Subsequent requests return in milliseconds.\n\nThe proof is in the `-v` log line. Cold first request, then warm:\n\n```\nrequest in_msgs=4 in_bytes=47227 out_bytes=24885 dur_ms=56956 worker_ms=42 cold=true  model_limit=200000 -\u003e modify\nrequest in_msgs=4 in_bytes=47227 out_bytes=24885 dur_ms=12    worker_ms=11 cold=false model_limit=200000 -\u003e modify\n```\n\n| Field | Meaning |\n|---|---|\n| `in_bytes` / `out_bytes` | Request body size before/after. `out_bytes \u003c in_bytes` on a `modify` is compression happening. |\n| `dur_ms` / `worker_ms` | Total handler time / time inside the Python worker. A large gap with `cold=true` is the one-time model load. |\n| `cold=true` | This request paid the model load. Expect it once per worker. |\n| `-\u003e modify` / `-\u003e allow(...)` | The action returned, with a reason (`modify`, `allow(noop)`, `allow(error)`, `allow(passthrough)`, `allow(read-error)`). |\n\nConfirm the inverse too. A short chat returns `allow`:\n\n```shell\ncurl -s -X POST http://127.0.0.1:8080/ -H 'Content-Type: application/json' \\\n  -d '{\"request_body\":{\"model\":\"claude-sonnet-4-5-20250929\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}}'\n# -\u003e {\"action\":\"allow\"}\n```\n\n### 2. End-to-end through Aperture\n\nOnce the hook + grant are live:\n\n1. Make a real LLM call through your Aperture endpoint that includes a substantial tool result (or run a coding-agent session that does).\n1. On the tsheadroom device, watch the `-v` output (or `journalctl -u tsheadroom -f` under systemd). A request that reaches it and compresses logs `-\u003e modify` with `out_bytes \u003c in_bytes`.\n\nIf you see the request arrive and log `modify`, the hook is firing and compressing in production.\n\n### Nothing compressing? Checklist\n\nAll `allow`, no `modify`? Work down this list:\n\n| Symptom | Check |\n|---|---|\n| No requests reach the device at all | The hook is firing? In a tailnet policy-file grant, is `dst` set (see the note above)? Does the grant `src` match your users and `models` match your traffic? |\n| Requests arrive but always `allow(passthrough)` | The body has no `messages` array (for example, Gemini `contents` or embeddings), so those pass through by design. |\n| Requests arrive but always `allow(noop)` | Nothing worth compressing: short chat, prose-only, or no substantial tool result. This is correct behavior. See [What gets compressed](#what-gets-compressed). |\n| Text/prose isn't shrinking | The `[ml]` extra isn't installed, or `compress_user_messages` is off. Install `headroom-ai[ml]` and check `GET /config`. |\n| Occasional `allow(error)` under load or on first request | A compression exceeded the Aperture hook `timeout` and failed open. Raise `timeout`, or pre-warm the model (cold start). |\n\n## Tune compression (runtime config)\n\ntsheadroom exposes Headroom's compression knobs as live, persisted configuration, with no restart needed. Settings load from the `-config` file at startup (defaults if missing), and a small HTTP API on the same device reads and changes them on the fly; every change is written back to the file, so the service restarts with your last state.\n\n```shell\n# Read the current settings\ncurl -s http://tsheadroom.\u003cyour-tailnet\u003e.ts.net/config\n\n# Change one or more (partial updates merge onto current values)\ncurl -s -X PUT http://tsheadroom.\u003cyour-tailnet\u003e.ts.net/config \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"compress_user_messages\": true, \"target_ratio\": 0.3}'\n```\n\nThe default configuration (what `GET /config` returns out of the box):\n\n```json\n{\n  \"compress_user_messages\": false,\n  \"compress_system_messages\": true,\n  \"protect_recent\": 4,\n  \"protect_analysis_context\": true,\n  \"target_ratio\": null,\n  \"min_tokens_to_compress\": 250,\n  \"kompress_model\": null\n}\n```\n\nChanges take effect on the next request. Invalid values are rejected with `400` and leave the current configuration untouched (for example, `target_ratio: 5` → `target_ratio must be in (0, 1] or null`). You can also edit the `-config` JSON file directly and restart.\n\nTunable parameters (these mirror Headroom's `CompressConfig`):\n\n| field | type | default | effect |\n|---|---|---|---|\n| `compress_user_messages` | bool | `false` | Also compress user-message content (off = protect it). Turn on for prose/RAG-heavy inputs. *(Needs `[ml]`.)* |\n| `compress_system_messages` | bool | `true` | Compress system messages. |\n| `protect_recent` | int | `4` | Leave the last N messages untouched. `0` = compress everything. |\n| `protect_analysis_context` | bool | `true` | Detect \"analyze/review\" intent and protect code. |\n| `target_ratio` | float \\| null | `null` | Keep-ratio for text compression: `null` = model decides (~aggressive), `0.5` = keep 50%. Must be in `(0, 1]`. *(Needs `[ml]`.)* |\n| `min_tokens_to_compress` | int | `250` | Skip messages shorter than this. |\n| `kompress_model` | string \\| null | `null` | Override the Kompress model id; `null` = default. |\n\n\u003e **Access**: the `/config` endpoint is gated only by your [tailnet policy file](https://tailscale.com/docs/reference/syntax/policy-file): anyone who can reach the device can read and change its configuration. Lock the device down accordingly (see [Security](#security-and-data-handling)).\n\n### ML model loading and timeouts\n\nThe ML text compressor loads a ~600 MB model on first use (one-time, then cached on disk and resident in each worker). This load takes several seconds; a large, late-session conversation can itself take a few seconds to compress. There is one worker timeout, and the client-facing wait is owned by Aperture:\n\n- **`-max-compress` (60s)** bounds the *worker*: a single call may run this long before the worker is treated as wedged and recycled. tsheadroom imposes no shorter deadline: a slow call runs to completion (the model finishes loading, the worker stays warm) and the result is used if Aperture is still waiting.\n- **Aperture's hook `timeout`** bounds the *client*: when it fires, Aperture (with `fail_open`) forwards the original request uncompressed. This is the only client-facing latency ceiling, and it lives in the Aperture configuration. There is no tsheadroom-side deadline to keep in sync.\n\nThe result: a cold worker pays at most one slow (or, behind Aperture's timeout, uncompressed `allow`) request while the model loads, then it works, with no restart needed. To avoid even that, workers preload the model at startup whenever the persisted configuration could route content through the ML compressor. With `headroom-ai[ml]` installed this is the default: `compress_system_messages` is on and Headroom uses the ML model as its fallback for tool/mixed content, so it's needed under ordinary traffic, and workers come up warm. (Each preloaded worker holds its own resident copy of the model, which is why `-pool-size` defaults low.)\n\nThe model downloads from HuggingFace on first ever use (cached under `~/.cache/huggingface` thereafter). transformers revalidates that cache against the Hub on each cold load; tsheadroom skips that network round-trip (and its anonymous rate-limiting) by running workers offline once the model is cached (`HF_HUB_OFFLINE`/`TRANSFORMERS_OFFLINE`, set automatically; your own values are respected). To stay online with higher rate limits (for example, to let a fresh host download the model), set `HF_TOKEN` in the environment; it takes precedence over the automatic offline mode. A worker has ~60s to report ready, so a slow *cold* download on a fresh host can exceed that and make workers retry: warm the cache once (start a worker with the cache empty, or run a single `compress` under the worker's interpreter) before serving production traffic.\n\n## What gets compressed\n\nBy default Headroom uses a conservative coding-agent profile: it compresses tool outputs (`tool_result` blocks / `role:\"tool\"` messages) and other bulky content, but protects user messages, the system prompt, and the most recent turns. So short chats (even long *prose* in a user message) return `allow`. You see `modify` when a request carries a substantial tool result.\n\ntsheadroom only inspects the `messages` array (Anthropic/OpenAI shape). A request body without one (for example, Gemini's `contents`, or an embeddings call) passes through unchanged (`allow`); only `messages` is ever rewritten.\n\n## Security and data handling\n\n- **What tsheadroom sees**: the full plaintext request body of every matched LLM call, including prompts, tool results, and everything in `messages`. It processes them in memory and does not persist request content to disk.\n- **Reachability**: tsheadroom has no auth of its own. Anyone who can reach the device over your tailnet can call the hook and read/modify its `/config`. Restrict access with [grants](https://tailscale.com/docs/features/access-control/grants), and give the device a [tag](https://tailscale.com/docs/features/tags) so you can target it: a tag is the device's identity at authentication time and serves as the selector grants match on (for example, `tag:tsheadroom` in a grant's `src`/`dst`).\n- **Egress**: the only outbound network call is the one-time HuggingFace model download, which you can eliminate by pre-seeding the cache and running offline (see above). Compression itself makes no network calls.\n- **Secrets**: `-state-dir` holds the device's private key, so keep it on durable, protected storage.\n\n## Development\n\n```shell\nmake            # build ./build/tsheadroom (Linux)\nmake test-go    # Go tests only: fake worker, no Python needed\nmake test       # Go tests + Python tests (Python tests run without headroom-ai;\n                # the real-headroom integration test self-skips unless installed)\nmake test PYTHON=/opt/tsheadroom/venv/bin/python  # also runs the real-headroom integration test\n```\n\nGo tests use a fake worker (no Python needed). Python tests use a fake `headroom` by default and run the real-headroom integration test only when `$PYTHON` has `headroom-ai` installed.\n\n## Bugs\n\ntsheadroom is not affiliated with [Headroom](https://github.com/chopratejas/headroom). File bugs about the `compress()` function and its compression behavior there. For bugs in tsheadroom (Aperture integration, request/response parsing, or program behavior), use the [tsheadroom issue tracker](https://github.com/tailscale/tsheadroom/issues).\n\n## Contributing\n\nContributions are welcome. See [CONTRIBUTING.md](CONTRIBUTING.md). We require a DCO `Signed-off-by` line on commits (`git commit -s`), and every source file carries the Tailscale copyright/SPDX header (enforced by `TestLicenseHeaders`). Follow the [Code of Conduct](CODE_OF_CONDUCT.md). Report security issues privately per [SECURITY.md](SECURITY.md).\n\n## License\n\nBSD 3-Clause. See [LICENSE](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftailscale%2Ftsheadroom","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftailscale%2Ftsheadroom","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftailscale%2Ftsheadroom/lists"}