https://github.com/rustkit-ai/mcpkill
Semantic cache proxy for MCP servers — cut Claude's token usage by up to 79%
https://github.com/rustkit-ai/mcpkill
ai caching claude cli developer-tools llm mcp proxy rust semantic-cache token-optimization
Last synced: 15 days ago
JSON representation
Semantic cache proxy for MCP servers — cut Claude's token usage by up to 79%
- Host: GitHub
- URL: https://github.com/rustkit-ai/mcpkill
- Owner: rustkit-ai
- License: mit
- Created: 2026-03-18T22:31:17.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2026-03-19T08:41:44.000Z (4 months ago)
- Last Synced: 2026-06-26T06:15:17.193Z (about 1 month ago)
- Topics: ai, caching, claude, cli, developer-tools, llm, mcp, proxy, rust, semantic-cache, token-optimization
- Language: Rust
- Size: 57.6 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
mcpkill
Semantic cache proxy for MCP servers — cut Claude's token usage by up to 79%
Install ·
Quick start ·
How it works ·
Benchmarks ·
Options
---
Every time Claude calls an MCP tool — `read_file`, `search_docs`, `list_directory` — the server dumps its **entire** response into the context window. A 500-line file? 500 lines every single call, even if you already asked about it ten minutes ago.
**mcpkill wraps any MCP server and intercepts every response.** It chunks the content, embeds it locally, caches it in SQLite, and sends Claude only the chunks that are actually relevant to the current question. Completely transparent — Claude sees a normal MCP server.
```
Without mcpkill With mcpkill
───────────────────────────────── ─────────────────────────────────────────
Claude asks about auth middleware Claude asks about auth middleware
↓ ↓
MCP returns entire 600-line file mcpkill checks semantic cache (~5 ms)
↓ ↓
2 400 tokens consumed Cache hit → re-ranks stored chunks
↓
380 tokens consumed ← same answer
```
---
## Benchmarks
Measured on a real session — 5 questions about a Rust codebase using `server-filesystem`:
```
Raw Filtered Saved
──────────────────────────────────────────────────────
read_file proxy.rs 1 637 t 390 t − 76%
read_file filter.rs 1 504 t 374 t − 75%
read_file cache.rs 2 763 t 348 t − 87%
read_file proxy.rs ★ same 390 t − 76% ← cache hit, 0 ms
read_file filter.rs ★ same 374 t − 75% ← cache hit, 0 ms
──────────────────────────────────────────────────────
Total 9 045 t 1 876 t − 79%
★ similar follow-up question, served entirely from cache
```
**80% cache hit rate from the very first session.** The more you work on a codebase, the higher it gets.
> Savings depend on the MCP server. Servers that dump raw content (filesystem, GitHub, search)
> see 60–90%. Servers that already do smart retrieval (e.g. context7) see 20–35%
> from chunking alone.
---
## Install
**Script (recommended — auto-detects OS and architecture):**
```bash
curl -fsSL https://raw.githubusercontent.com/rustkit-ai/mcpkill/main/install.sh | sh
```
**Cargo:**
```bash
cargo install mcpkill
```
**Pre-built binaries** for macOS (Intel + Apple Silicon), Linux (x86_64 + aarch64), and Windows
are available on the [Releases page](https://github.com/rustkit-ai/mcpkill/releases).
> On first run, the embedding model (~23 MB) downloads automatically to `~/.fastembed_cache`
> and is reused on every subsequent run. No GPU required.
---
## Quick start
### One command per server
```bash
mcpkill install filesystem -- npx -y @modelcontextprotocol/server-filesystem ~/projects
mcpkill install context7 -- npx -y @upstash/context7-mcp
mcpkill install github -- npx -y @modelcontextprotocol/server-github
```
This pre-warms the embedding model, then registers the server in Claude Code under your user scope.
Restart Claude — done. Verify with `claude mcp list`.
### Manual config
Add `mcpkill --` in front of any MCP server command in `.claude/settings.json`:
```json
{
"mcpServers": {
"filesystem": {
"command": "mcpkill",
"args": ["--", "npx", "-y", "@modelcontextprotocol/server-filesystem", "~/projects"]
},
"context7": {
"command": "mcpkill",
"args": ["--", "npx", "-y", "@upstash/context7-mcp"]
},
"github": {
"command": "mcpkill",
"args": ["--", "npx", "-y", "@modelcontextprotocol/server-github"]
}
}
}
```
---
## How it works
mcpkill intercepts every `tools/call` response over stdio — Claude never knows it's there.
**First call (cache miss)**
1. MCP server returns a large response
2. mcpkill splits it into semantic chunks:
- Markdown → split by `#` / `##` / `###` headers
- JSON array → one chunk per element
- JSON object → one chunk per top-level key
- Plain text → split by blank lines
3. Each chunk is embedded locally with `all-MiniLM-L6-v2` (~5 ms total, no network)
4. The 4 most relevant chunks are returned to Claude
5. All chunks are stored in `~/.mcpkill.db` (SQLite, no external services)
**Subsequent calls (cache hit)**
1. Query is embedded (~5 ms)
2. Cosine similarity search across stored query embeddings
3. If similarity ≥ 0.85 → cache hit
4. Stored chunks are re-ranked against the new query, top-4 returned instantly
5. The MCP server's response is discarded before it touches Claude's context
**See your savings at any time:**
```bash
mcpkill stats
```
```
┌─────────────────────────────────────────┐
│ mcpkill cache stats │
├─────────────────────────────────────────┤
│ Cache entries 47 │
│ Stored chunks 312 │
│ Cache hits 132 (80%) │
├─────────────────────────────────────────┤
│ Tokens (original) 68 200 │
│ Tokens (returned) 4 100 │
│ Tokens saved ~64 100 (94%) │
├─────────────────────────────────────────┤
│ DB size 1.80 MB │
└─────────────────────────────────────────┘
```
---
## Options
```
mcpkill [OPTIONS] -- [ARGS...]
mcpkill install [--scope user|local|project] -- [ARGS...]
mcpkill stats [--json]
mcpkill clear [--all | --expired | --older-than ]
mcpkill print-config
```
| Flag | Default | Description |
|------|---------|-------------|
| `--max-chunks` | `4` | Number of chunks returned per response |
| `--threshold` | `0.85` | Cosine similarity threshold for cache hits |
| `--ttl-days` | `7` | Evict entries not accessed in N days |
| `--max-db-mb` | `100` | Max cache size in MB before LRU eviction |
| `--cache-db` | `~/.mcpkill.db` | Path to the SQLite cache |
| `--dry-run` | off | Log decisions without modifying responses |
| `--stats-interval` | `0` | Print stats every N seconds (0 = off) |
| `-v / --verbose` | off | Show cache hit/miss on stderr |
**Persist settings in `~/.mcpkill.toml`** — CLI flags always win:
```toml
threshold = 0.80 # lower = more cache hits, less precise
max_chunks = 6 # more chunks = more context, more tokens
ttl_days = 14 # increase for stable libraries
max_db_mb = 200 # increase for large codebases
```
**Tuning guide:**
| Situation | Adjustment |
|-----------|------------|
| Hitting cache on unrelated queries | Raise `--threshold` to `0.90` |
| Missing hits on clearly similar queries | Lower `--threshold` to `0.80` |
| Losing important detail in responses | Raise `--max-chunks` to `6` |
| Library changes frequently (e.g. canary docs) | Lower `--ttl-days` to `1` |
| Debugging unexpected behavior | `--dry-run --verbose` |
---
## Cache management
```bash
mcpkill clear --expired # remove entries older than TTL
mcpkill clear --older-than 14 # remove entries unused for 14+ days
mcpkill clear --all # wipe the entire cache
```
---
## Privacy
Everything runs locally. mcpkill never sends your code, queries, or embeddings anywhere.
The only network calls are the normal ones to your MCP servers.
---
MIT · built by [rustkit-ai](https://github.com/rustkit-ai)