https://github.com/hinanohart/cachelens
Offline prompt-cache precondition checker: first-divergence diff, lint, CI gate for Anthropic/OpenAI/DeepSeek/Gemini
https://github.com/hinanohart/cachelens
anthropic cli llm openai prompt-cache python
Last synced: about 1 month ago
JSON representation
Offline prompt-cache precondition checker: first-divergence diff, lint, CI gate for Anthropic/OpenAI/DeepSeek/Gemini
- Host: GitHub
- URL: https://github.com/hinanohart/cachelens
- Owner: hinanohart
- License: mit
- Created: 2026-05-29T03:26:23.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2026-06-10T13:32:57.000Z (about 1 month ago)
- Last Synced: 2026-06-10T14:08:35.914Z (about 1 month ago)
- Topics: anthropic, cli, llm, openai, prompt-cache, python
- Language: Python
- Homepage: https://github.com/hinanohart/cachelens
- Size: 159 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# cachelens
> **Pre-Alpha (v0.1.0a4)** — offline prompt-cache precondition checker.
cachelens is a **fully offline, CPU-only** Python tool that checks whether your LLM prompt prefixes satisfy provider cache requirements — *before* you send requests. It finds the **first point where cache breaks** and tells you **why**, so you can fix it in CI rather than discover it on your invoice.
Supports Anthropic, OpenAI, DeepSeek, and Gemini. Zero runtime dependencies.
---
## What it does
- **`diff`**: Compare two requests and find the first divergence that would break cache continuity — returns `STABLE → BROKEN` state transition + byte offset + cause classification.
- **`lint`**: Check a single request for cache-breaking anti-patterns (timestamps, UUIDs, non-deterministic tool order, missing thresholds, dynamic system prompts).
- **`check`**: CI wrapper — exits non-zero when cacheability drops below threshold.
All output carries a `determinism` label (`LINT` for v0.1.0) so over-claim is type-impossible.
## What it does NOT do
- **Does not** predict actual cache hit/miss rates (depends on provider TTL, load, eviction — internal state).
- **Does not** produce Anthropic token counts offline (no public local tokenizer; token counts require the `count_tokens` API — deferred to v0.1.1).
- **Does not** produce dollar cost estimates (deferred to v0.1.1 with `--with-cost` + real-token opt-in).
- `Determinism.SIM` type value exists but SIM path is not implemented in v0.1.0.
---
## Install
```bash
pip install cachelens # zero runtime deps
pip install "cachelens[tiktoken]" # exact token counts (OpenAI/DeepSeek)
pip install "cachelens[hf]" # HuggingFace tokenizers
pip install "cachelens[all]" # both
```
---
## Quick start
```bash
# diff two request files (synthetic fixture shown)
cachelens diff tests/fixtures/stable.json tests/fixtures/broken.json
# lint a single request
cachelens lint tests/fixtures/stable.json --provider anthropic --explain
# CI check against a trace
cachelens check tests/fixtures/trace_stable.jsonl --threshold 0.8
echo "exit: $?" # 0=stable, 1=broken, 2=input-error (3=abstain reserved for v0.1.1)
```
### Exit codes (contract)
| Code | Meaning |
|---|---|
| 0 | Cache prefix is stable |
| 1 | Cache broken (threshold breach) — CI should fail |
| 2 | Input error (bad file, unknown provider) |
| 3 | Abstain — reserved for v0.1.1 (not emitted in v0.1.0) |
---
## How it works
The core pipeline: requests are **normalized** (JCS-style key sorting strips spurious byte differences), split into **segments** (system → tools → messages in cache-cascade order), then either **diffed** against a second request for first-divergence detection or **linted** for known anti-patterns. All operations are deterministic and require no network calls.
---
## Python API
```python
from cachelens import diff, lint, Request, Trace, Divergence
req_a = Request.from_file("a.json")
req_b = Request.from_file("b.json")
result = diff(req_a, req_b, provider="anthropic")
# result.state: "STABLE" | "BROKEN"
# result.first_divergence_offset: int (byte offset)
# result.cause: Cause enum
# result.determinism: Determinism.LINT
```
### Building a Request from SDK kwargs
```python
from cachelens import lint
from cachelens.model import Request
# Convert Anthropic SDK kwargs directly
req = Request.from_anthropic_kwargs(
model="claude-3-5-sonnet-20241022",
system=[{"type": "text", "text": "You are helpful.", "cache_control": {"type": "ephemeral"}}],
messages=[{"role": "user", "content": "Hello"}],
)
result = lint(req, provider="anthropic")
print(result.passed) # True / False
```
---
## GitHub Action
```yaml
- uses: hinanohart/cachelens@v0.1.0a4
with:
trace: "traces/latest.jsonl"
threshold: "0.8"
provider: "anthropic"
```
---
## Provider support (v0.1.0)
| Provider | Cache type | Determinism tier | Offline guarantee |
|---|---|---|---|
| Anthropic | Explicit `cache_control`, max 4 breakpoints | `LINT` | Prefix stability, cascade invalidation, threshold bool |
| OpenAI | Automatic ≥1024 tok, 128-tok increment | `LINT` | Prefix stability |
| DeepSeek | Automatic, 64-tok unit, 0th-token origin | `LINT` | Prefix stability |
| Gemini | Implicit (2.5+ default) + explicit CachedContent | `LINT` | Prefix stability, best-practice lint |
All providers: `Determinism.LINT` (byte-prefix stability). Token simulation (`SIM`) is v0.1.1.
---
## Cause classifications
When `diff` or `lint` reports a problem, it returns one of these `Cause` values:
| Cause | Description |
|---|---|
| `timestamp` | A timestamp or date-time value changed between requests |
| `uuid` | A UUID or random ID changed between requests |
| `tool_order` | Tool definitions appear in different order (non-deterministic) |
| `dynamic` | A dynamic/non-deterministic value changed |
| `benign_suffix` | Trailing addition that still breaks exact prefix match |
| `normalized_only` | Divergence only in raw bytes; logical content matches |
| `unknown` | Cause could not be classified |
---
## Comparison
| Feature | cachelens | prompt-cache-optimizer (v0.5) |
|---|---|---|
| Offline static gate / CI | Yes (non-zero exit) | No (runtime SDK wrapper) |
| Cause classification | Yes (enum + `--explain` prescription) | No (char diff only) |
| Python / pytest / pre-commit | Yes | No (TypeScript only) |
| Deterministic JSON canonicalization (JCS-style key sort) | Yes (strict + logical modes) | No |
| Token/cost estimates | v0.1.1 (opt-in, labeled) | Yes (online) |
---
## Synthetic fixtures disclaimer
All fixtures in `tests/fixtures/` are **synthetic, hand-crafted examples** for unit testing. They do not represent real provider responses or measured performance data. Numbers in test output are derived from the fixture inputs — not from provider APIs.
---
## License
MIT. See [LICENSE](LICENSE).