{"id":50832800,"url":"https://github.com/tathagat22/reel","last_synced_at":"2026-06-14T01:08:15.822Z","repository":{"id":357921673,"uuid":"1239177099","full_name":"tathagat22/reel","owner":"tathagat22","description":"VCR for LLM APIs — record and replay OpenAI/Anthropic/Gemini calls including streaming.","archived":false,"fork":false,"pushed_at":"2026-05-14T22:03:02.000Z","size":259,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-05-14T22:08:55.564Z","etag":null,"topics":["ai","anthropic","gemini","llm","mocking","openai","proxy","pytest","python","replay","testing","vcr"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tathagat22.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-05-14T20:52:52.000Z","updated_at":"2026-05-14T22:03:05.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/tathagat22/reel","commit_stats":null,"previous_names":["tathagat22/reel"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/tathagat22/reel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tathagat22%2Freel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tathagat22%2Freel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tathagat22%2Freel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tathagat22%2Freel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tathagat22","download_url":"https://codeload.github.com/tathagat22/reel/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tathagat22%2Freel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34305805,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-13T02:00:06.617Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","anthropic","gemini","llm","mocking","openai","proxy","pytest","python","replay","testing","vcr"],"created_at":"2026-06-14T01:08:10.628Z","updated_at":"2026-06-14T01:08:15.815Z","avatar_url":"https://github.com/tathagat22.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Reel — VCR for LLM APIs\n\n[![CI](https://github.com/tathagat22/reel/actions/workflows/ci.yml/badge.svg)](https://github.com/tathagat22/reel/actions/workflows/ci.yml)\n[![docs](https://github.com/tathagat22/reel/actions/workflows/docs.yml/badge.svg)](https://tathagat22.github.io/reel/)\n[![PyPI](https://img.shields.io/pypi/v/reel-vcr.svg)](https://pypi.org/project/reel-vcr/)\n[![License](https://img.shields.io/badge/license-Apache_2.0-blue.svg)](LICENSE)\n[![Python](https://img.shields.io/badge/python-3.11_|_3.12_|_3.13_|_3.14-blue.svg)](pyproject.toml)\n\nRecord OpenAI, Anthropic, and Gemini API calls once, then replay them in your tests, your CI, or your local dev loop — for free, forever. **No mocks. No SDK changes. No real network in CI. No surprise bills.**\n\n![Reel — record once, replay forever](docs/demos/reel-demo.gif)\n\n```bash\npip install reel-vcr\nreel auto -c tape.jsonl \u0026\nexport OPENAI_BASE_URL=http://127.0.0.1:7878/v1\npython your_app.py\n# First call: forwarded to OpenAI, captured to tape.jsonl.\n# Every call after: replayed from disk in ~3 ms. Zero spend.\n```\n\nCassettes are plain JSONL — you `grep` them, `jq` them, `git diff` them in PRs. Secrets and PII are scrubbed at capture time, so they're safe to commit.\n\n\u003e **Docs**: \u003chttps://tathagat22.github.io/reel/\u003e · **Issues**: \u003chttps://github.com/tathagat22/reel/issues\u003e\n\n---\n\n## See it in action — Claude Opus demo\n\nThe same `claude -p` job run three times against three real markdown docs. **First run** records to disk and pays Anthropic for Opus tokens. **Runs 2 and 3** serve every call from the cassette in 2-3 ms and pay nothing.\n\n[![Opus demo — record once, replay free](docs/demos/opus-demo.gif)](docs/demos/opus-demo.mp4)\n\nPer-call latency in this run, straight from the proxy log:\n\n| Run | Call 1 | Call 2 | Call 3 |\n|---|---|---|---|\n| 1 (record) | 1865 ms | 1708 ms | 2183 ms |\n| 2 (replay) | **2 ms** | **2 ms** | **3 ms** |\n| 3 (replay) | **2 ms** | **2 ms** | **3 ms** |\n\nOutput bytes are identical across all three runs. Cassette stays at 3 entries — replay never re-records. Reproduce it yourself:\n\n```bash\ngit clone https://github.com/tathagat22/reel \u0026\u0026 cd reel\n./opus-demo.sh\n```\n\n---\n\n## What Reel is\n\nA local HTTP proxy that sits between your code and the LLM provider. On the first call to a given prompt, Reel forwards the request upstream and writes the wire-level request/response pair to a JSONL \"cassette\" file. On every subsequent call with the same fingerprint, the cassette serves the recorded response from disk — about 250× faster than the network and ~$0 to your billing.\n\nIt works with **any OpenAI-compatible endpoint** (OpenAI, Ollama, NVIDIA NIM, vLLM, Groq, Together, OpenRouter, LM Studio), Anthropic's `/v1/messages` API, and Google's Gemini `/v1beta/models/...:generateContent` API. Streaming is captured chunk-by-chunk with millisecond timing fidelity — replays reproduce the original TTFT and inter-chunk gaps within ~20 ms.\n\n## Real scenarios that motivated this\n\n### Your pytest suite bills you on every push\n\nTests that call `OpenAI().chat.completions.create(...)` cost real money on every CI run — maybe $0.10, maybe $5, multiplied by every PR push by every team member by every retry. Add Reel:\n\n```python\n# tests/test_summarize.py\ndef test_summarize(reel_cassette):           # fixture auto-registered by the pytest plugin\n    from openai import OpenAI\n    resp = OpenAI().chat.completions.create(\n        model=\"gpt-5\",\n        messages=[{\"role\": \"user\", \"content\": \"Summarize this...\"}],\n    )\n    assert \"TL;DR\" in resp.choices[0].message.content\n```\n\nFirst run records to `tests/cassettes/test_summarize.jsonl`. Every run after replays from disk. In CI: `pytest --reel-mode replay` — any uncaptured request 404s, so the test suite cannot accidentally hit a real API. CI runs without an API key.\n\n### A user reported a weird LLM answer; you can't reproduce it locally\n\nIn production you have logs but no way to replay the exact call. Hand the affected cassette to a developer on a separate machine — they replay your production traffic byte-for-byte, with no API key and no risk of mutating live data. Reel records the actual request body and the upstream's actual response, not a model card.\n\n### You're iterating on a prompt; every tweak re-spends tokens\n\nA prompt-engineering session hits the API dozens of times per evening. With Reel in `auto` mode, each unique prompt costs real money once; every variation you've already tried replays free. Combine with `reel diff -l old.jsonl -r new.jsonl` to see exactly what changed between two prompt versions.\n\n### Your AI coding agent is slow\n\nAider, opencode, Claude Code, Cursor, Codex CLI — most coding agents send the same file content, the same chunk embeddings, the same tool definitions to the LLM many times during a session. Reel caches the deterministic parts so re-asking similar questions hits disk instead of the network. Verified to work transparently with Aider, opencode (NVIDIA NIM upstream), and Claude Code (via `ANTHROPIC_BASE_URL`).\n\n## The pytest plugin\n\nAuto-registered via the `pytest11` entry point. No `conftest.py` edits.\n\n```python\n# Fixture form — cassette path is inferred from the test name:\ndef test_chat(reel_cassette):\n    OpenAI().chat.completions.create(...)\n\n# Decorator form — custom cassette path:\nfrom reel.sdk import cassette\n\n@cassette(\"tests/cassettes/golden_chat.jsonl\")\ndef test_chat_golden():\n    OpenAI().chat.completions.create(...)\n\n# Marker form — for pytest-marker fans:\nimport pytest\n\n@pytest.mark.cassette(\"tests/cassettes/chat.jsonl\", mode=\"record\")\ndef test_chat_marker(reel_cassette):\n    OpenAI().chat.completions.create(...)\n```\n\n`pytest --reel-mode replay` forces every cassette into replay mode regardless of what the test source code says — useful for CI where you want to fail loud on any uncaptured request.\n\nFull walkthrough: [docs/guides/pytest.md](docs/guides/pytest.md).\n\n## Provider support\n\nSingle proxy, three first-class providers, plus anything OpenAI-compatible:\n\n| Provider | Endpoint Reel routes | Streaming |\n|---|---|---|\n| **OpenAI** | `/v1/chat/completions`, `/v1/embeddings`, `/v1/responses`, ... | ✅ SSE |\n| **Anthropic** | `/v1/messages` | ✅ SSE with `event:` lines |\n| **Google Gemini** | `/v1beta/models/\u003cmodel\u003e:generateContent`, `:streamGenerateContent` | ✅ SSE |\n| **NVIDIA NIM** | OpenAI-compatible — `--upstream https://integrate.api.nvidia.com` | ✅ |\n| **Ollama** | OpenAI-compatible at `http://localhost:11434/v1` | ✅ |\n| **vLLM, LM Studio, Groq, Together, OpenRouter** | OpenAI-compatible — set `--upstream` to whatever they serve | ✅ |\n| **Azure OpenAI, AWS Bedrock, GCP Vertex** | Proxied generically; first-class adapters on the roadmap | ✅ |\n\nFor multi-provider apps in a single session, use the explicit URL prefix: `http://127.0.0.1:7878/openai/v1/...`, `/anthropic/v1/...`, `/gemini/v1beta/...`. The same proxy handles all three at once.\n\n## How Reel compares\n\n| Tool | Where it sits | Works with non-Python? | SSE streaming? | Breaks when SDKs change transport? |\n|---|---|---|---|---|\n| **Reel** | HTTP proxy | Yes (any language) | ✅ with timing fidelity | No — it's language-agnostic |\n| **VCR.py / pytest-recording / pytest-vcr** | Monkey-patches urllib3 / requests | No (Python only) | Partial | Yes — breaks on transport changes |\n| **respx / pytest-httpx** | Mocks httpx at the client layer | Python only | Limited | Yes |\n| **llm-test-harness** | Wraps the SDK in Python (`harness.wrap(client)`) | No (Python only) | Limited | Yes — coupled to specific SDK clients |\n| **agent-vcr** | Records JSON-RPC for **MCP** servers (different layer entirely) | n/a — MCP, not LLM HTTP | n/a | n/a |\n| **Hand-rolled mocks** | Inside your code | No | Whatever you implement | Whenever you forget to update them |\n| **WireMock / MockServer** | HTTP proxy (Java) | Yes | Manual fixtures | Generic, not LLM-aware |\n\nThe trade-off: **VCR.py is easier to drop into a single test in a single file. Reel is easier to use across a whole project and any client that respects the standard `OPENAI_BASE_URL` / `ANTHROPIC_BASE_URL` env-var convention.**\n\nReel also ships the things you'd build yourself anyway: a smart matcher for prompts that drift slightly between runs (`exact / normalized / ignore-fields / fuzzy`), capture-time secret + PII scrubbing, a pre-commit hook that refuses to commit cassettes still containing detectable keys, and an analytics CLI for browsing, costing, and diffing recorded sessions.\n\n## CLI commands\n\n| Command | What it does |\n|---|---|\n| `reel auto -c \u003cpath\u003e` | Replay if cached, else record (the dev-loop default) |\n| `reel record -c \u003cpath\u003e` | Always forward to upstream + capture |\n| `reel replay -c \u003cpath\u003e` | Cassette only; 404 on miss. **The CI mode.** |\n| `reel ui -c \u003cpath\u003e` | Local web inspector — browse and search cassettes in a browser |\n| `reel inspect -c \u003cpath\u003e` | Rich-table view with composable filters (provider, model, status, has-tool-call, regex) |\n| `reel cost -c \u003cpath\u003e` | Token totals × current OpenAI/Anthropic/Gemini pricing → $ summary |\n| `reel diff -l A -r B` | Aligned diff of two cassettes — surface what regressed |\n| `reel stats -c \u003cpath\u003e` | Counts, error rate, token totals, TTFT distribution |\n| `reel redact -c \u003cpath\u003e` | Post-hoc scrub of secrets/PII from an existing cassette |\n| `reel doctor` | Sanity check: ports, upstreams, write perms, optional dep state |\n| `reel version` | Print installed version |\n\nRun `reel \u003ccommand\u003e --help` for flags.\n\n## Architecture\n\n```\nsrc/reel/\n├── proxy/        HTTP + SSE core — forwarder, mode handlers, stream capture, structured logs\n├── adapters/     ProviderAdapter interface — openai.py · anthropic.py · gemini.py\n├── cassette/     Schema · writer · reader · matcher · body codec · in-memory store\n├── redact/       Secret + PII pattern scrubbing\n├── analytics/    Pure functions over CassetteEntry — filters · cost · diff · stats\n├── inspector/    `reel ui` — Starlette + HTMX + Pico.css, no JavaScript build step\n├── cli/          Typer commands wired into the `reel` console_script\n└── sdk/          @cassette decorator + pytest plugin\n```\n\nDeeper dive: [docs/architecture.md](docs/architecture.md). Roadmap: [docs/roadmap.md](docs/roadmap.md).\n\n## FAQ\n\n### Is this just VCR.py with extra steps?\n\nNo. VCR.py and pytest-recording monkey-patch Python HTTP clients (`urllib3`, `requests`, `httpx`). When OpenAI or Anthropic ship a new SDK with a different transport, those tools silently break. Reel is an HTTP proxy — it sees the actual bytes on the wire, no matter what client sent them, no matter what language. It also works with non-Python clients (Cursor, Aider's Go bits, your TypeScript scripts).\n\n### How is Reel different from `llm-test-harness` and `agent-vcr`?\n\nBoth are recent (early 2026) and in adjacent territory; neither does the same thing as Reel:\n\n- **`llm-test-harness`** wraps the SDK at the Python client level (`harness.wrap(anthropic.Anthropic())`) and bundles eval scoring + regression metrics. Same fundamental shape as VCR.py — Python-only, coupled to specific SDK clients, breaks if a vendor changes its transport. Reel sits one layer below that as a language-agnostic HTTP proxy, and stays out of eval/scoring on purpose (use whatever eval framework you already like).\n- **`agent-vcr`** records JSON-RPC for **MCP** (Model Context Protocol) servers — a different layer entirely from LLM HTTP APIs. Complementary to Reel rather than competing: agent-vcr cassettes your MCP tool servers, Reel cassettes the actual LLM calls underneath.\n\n### Will it work with my coding agent — Claude Code, Aider, opencode, Cursor?\n\n- **Aider** — yes, transparently. Aider respects `OPENAI_API_BASE`.\n- **opencode** — yes, transparently. Verified with the NVIDIA NIM upstream.\n- **Claude Code** — yes; verified that the Claude Code binary respects `ANTHROPIC_BASE_URL`. Run `ANTHROPIC_BASE_URL=http://127.0.0.1:7878 claude ...`.\n- **Cursor** — yes with a one-line config change. Cursor's settings expose a \"Custom OpenAI Base URL\" field.\n- **Codex CLI** — yes, transparently. Uses the standard OpenAI env vars.\n- **Continue.dev** — yes via the custom-endpoint configuration.\n\nA `reel run -- \u003ccommand\u003e` zero-config wrapper is on the roadmap for the next minor release.\n\n### What about non-deterministic LLM responses?\n\nThe cassette stores the actual response the upstream returned on the first call. On replay, that exact response is served back. **Determinism is at the cassette level, not at the LLM level — that's the point.** If you genuinely need the model to think fresh every test run, don't put that test under Reel. For the in-between case (same prompt, different context window injected by your code), Reel's `ignore-fields` matcher mode lets you drop the parts you don't want fingerprinted.\n\n### Does it handle SSE streaming?\n\nYes, and that's the hardest thing Reel does. Each SSE chunk is captured with its wall-clock offset from the first byte. On replay, chunks are emitted with `asyncio.sleep` based on those offsets, so timing-sensitive client code (live token-by-token UIs, latency assertions) behaves the same way against a cassette as it does against the real upstream. Three timing modes: `--timing realtime | fast | slow=\u003cN\u003e`.\n\n### What about API keys in cassettes?\n\nReel **never** captures request headers — that's where API keys live in every supported provider. The cassette only stores method, path, body, and the fingerprint. Response bodies are also scanned for embedded keys (sk-*, sk-ant-*, AIza*, ghp_*, AKIA*, etc.) and Bearer tokens, with detected matches replaced by a redaction marker before the entry hits disk. A bundled pre-commit hook (`hooks/pre-commit-cassette-check.py`) refuses to commit any cassette that still contains a detectable secret pattern.\n\n### Does it work with local models? Ollama, vLLM, LM Studio?\n\nYes. Anything serving an OpenAI-compatible `/v1/chat/completions` endpoint works as an upstream. For Ollama:\n\n```bash\nreel auto -c tape.jsonl --upstream http://localhost:11434\n```\n\nThen any OpenAI SDK pointed at Reel routes through Ollama. The benefit even with local models: replay is ~3 ms while inference is 200-2000 ms, so iterating on prompts gets dramatically faster.\n\n### Why is the PyPI package named `reel-vcr` but I `import reel`?\n\nThe bare `reel` name on PyPI was already registered by an unrelated async-subprocess library. Rather than rename the whole project, Reel follows the same convention as Pillow (`pip install pillow` → `import PIL`) and scikit-learn (`pip install scikit-learn` → `import sklearn`). The PyPI distribution name is `reel-vcr`; the CLI binary, GitHub repo, and Python import path are all just `reel`.\n\n### Can I use this with tool calls / function calling?\n\nYes. Tool definitions are part of the request body and participate in the fingerprint, so a request with different `tools` content gets a different cassette key. The `tool_calls` field in responses is captured and replayed normally. The case Reel doesn't yet handle elegantly is multi-turn agentic loops where the LLM picks a tool, your code runs it, and the loop continues for many rounds — each individual call works, but order-of-operations matching across long loops is a roadmap item.\n\n### Is there a hosted version? Reel Cloud?\n\nNo, and there isn't a plan to build one until there's clear pull for it. Reel runs entirely on `127.0.0.1`, writes only to files you explicitly point it at, ships no telemetry, and makes no network calls of its own besides forwarding requests to whatever upstream you configured.\n\n### Does it work in CI — GitHub Actions, GitLab, CircleCI?\n\nYes. The recommended CI pattern is:\n\n```yaml\n- run: pip install reel-vcr\n- run: pytest --reel-mode replay\n```\n\nReplay mode fails loud on any uncaptured request, so you can't accidentally hit a real API from CI even if a teammate forgot to commit a cassette. See [docs/guides/ci.md](docs/guides/ci.md).\n\n### How do I uninstall?\n\n```bash\npip uninstall reel-vcr\n```\n\nThat's it. No daemon survives, no shell config edits, no `~/.config` directories created. The only state Reel writes is the cassette files you explicitly named.\n\n## Status\n\n**v0.1.0 is live on PyPI.** Verified clean install on Python 3.11, 3.13, and 3.14 (102 install/import/runtime checks). 344 tests in the project's own suite, green on every CI run. Apache 2.0. Issues and PRs welcome — see [CONTRIBUTING.md](CONTRIBUTING.md).\n\n## Install\n\n```bash\npip install reel-vcr           # CLI binary + import path stay `reel`\nreel --help\n```\n\nOr from source:\n\n```bash\ngit clone https://github.com/tathagat22/reel \u0026\u0026 cd reel\nuv sync\nuv run reel --help\n```\n\n## Development\n\n```bash\nuv sync\nmake check         # ruff + ruff format + pyright (strict) + pytest — must pass before every commit\nuv run reel auto -c ./scratch/test.jsonl\n```\n\nCI runs lint, format check, type check (pyright strict), and the full test suite on Python 3.11 / 3.12 / 3.13 — see [.github/workflows/ci.yml](.github/workflows/ci.yml). The mkdocs-material docs site auto-deploys to GitHub Pages on every push to `main` — see [.github/workflows/docs.yml](.github/workflows/docs.yml).\n\n## License\n\n[Apache 2.0](LICENSE) — use, fork, vendor, embed, ship.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftathagat22%2Freel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftathagat22%2Freel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftathagat22%2Freel/lists"}