https://github.com/mxyhi/token_proxy
Local AI API gateway for OpenAI / Gemini / Anthropic. Runs on your machine, keeps tokens counted (SQLite), offers priority-based load balancing, optional OpenAI Chat↔Responses format conversion, and one-click setup for Claude Code / Codex.
https://github.com/mxyhi/token_proxy
api-key-managenment claude-api claude-code codex openai openai-api openai-responses-api opencode provider-management rust tauri typescript
Last synced: about 1 month ago
JSON representation
Local AI API gateway for OpenAI / Gemini / Anthropic. Runs on your machine, keeps tokens counted (SQLite), offers priority-based load balancing, optional OpenAI Chat↔Responses format conversion, and one-click setup for Claude Code / Codex.
- Host: GitHub
- URL: https://github.com/mxyhi/token_proxy
- Owner: mxyhi
- License: apache-2.0
- Created: 2026-01-09T16:14:54.000Z (5 months ago)
- Default Branch: main
- Last Pushed: 2026-03-24T14:27:57.000Z (2 months ago)
- Last Synced: 2026-03-25T18:06:09.139Z (2 months ago)
- Topics: api-key-managenment, claude-api, claude-code, codex, openai, openai-api, openai-responses-api, opencode, provider-management, rust, tauri, typescript
- Language: Rust
- Homepage:
- Size: 13.6 MB
- Stars: 28
- Watchers: 1
- Forks: 4
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Agents: AGENTS.md
Awesome Lists containing this project
README
# Token Proxy
English | [中文](README.zh-CN.md)
Local AI API gateway for OpenAI / Gemini / Anthropic. Runs on your machine, keeps tokens counted (SQLite), offers priority-based load balancing, optional API format conversion (OpenAI Chat/Responses ↔ Anthropic Messages, plus Gemini ↔ OpenAI/Anthropic, including SSE/tools/images), and one-click setup for Claude Code / Codex.
> Default listen port: **9208** (release) / **19208** (debug builds).
---
## What you get
- Multiple providers: `openai`, `openai-response`, `anthropic`, `gemini`, `kiro`, `codex`
- Built-in routing + optional format conversion (OpenAI Chat ⇄ Responses; Anthropic Messages ↔ OpenAI; Gemini ↔ OpenAI/Anthropic; SSE supported)
- Per-upstream priority + two balancing strategies (fill-first / round-robin)
- Model alias mapping (exact / prefix* / wildcard*) and response model rewrite
- Local access key (Authorization) + upstream key injection
- SQLite-powered dashboard (requests, tokens, cached tokens, latency, recent)
- macOS tray live token rate (optional)
## Screenshots
| | |
| --- | --- |
| **Dashboard**
 | **Core**
 |
| **Upstreams**
 | **Add upstream**
 |
## Quick start (macOS)
1) Install: move `Token Proxy.app` to `/Applications`. If blocked: `xattr -cr /Applications/Token\ Proxy.app`.
2) Launch the app. The proxy starts automatically.
3) Open **Config File** tab, edit and save (writes `config.jsonc` in the Tauri config dir). Defaults are usable; just paste your upstream API keys. Running proxies auto-apply the new config via reload or restart when needed.
4) Call via curl (example with local auth):
```bash
curl -X POST \
-H "Authorization: Bearer YOUR_LOCAL_KEY" \
-H "Content-Type: application/json" \
http://127.0.0.1:9208/v1/chat/completions \
-d '{"model":"gpt-4.1-mini","messages":[{"role":"user","content":"hi"}]}'
```
You can also call using the Anthropic Messages format (useful for Claude Code clients):
```bash
curl -X POST \
-H "x-api-key: YOUR_LOCAL_KEY" \
-H "Content-Type: application/json" \
http://127.0.0.1:9208/v1/messages \
-d '{"model":"claude-3-5-sonnet-20241022","max_tokens":256,"messages":[{"role":"user","content":[{"type":"text","text":"hi"}]}]}'
```
## Workspace & CLI (Rust)
- This repo is now a Cargo workspace; the Tauri app still lives in `src-tauri/`.
- CLI crate: `crates/token_proxy_cli` (binary `token-proxy`).
- Default config path: `./config.jsonc` (override with `--config`).
- GitHub Releases also publish packaged CLI archives per target:
- Unix: `token-proxy_cli__.tar.gz`
- Windows: `token-proxy_cli__.zip`
```bash
# start proxy
cargo run -p token_proxy_cli -- serve
# start with custom config path
cargo run -p token_proxy_cli -- --config ./config.jsonc serve
# config helpers
cargo run -p token_proxy_cli -- config init
cargo run -p token_proxy_cli -- --config ./config.jsonc config path
```
## Frontend tests
```bash
# watch mode
pnpm test
# run once (CI-friendly)
pnpm test:run
# coverage (optional)
pnpm test:coverage
# TypeScript typecheck
pnpm exec tsc --noEmit
```
Notes:
- Test files live in `src/**/*.test.{ts,tsx}`.
- Global test setup (Tauri mocks + jsdom polyfills) is in `src/test/setup.ts`.
- Vitest config is in `vitest.config.ts`.
## Configuration reference
- File: `config.jsonc` (comments + trailing commas allowed)
- Location:
- CLI: `--config` (default: `./config.jsonc`)
- Tauri: **AppConfig** directory (resolved automatically by the app)
### Core fields
| Field | Default | Notes |
| --- | --- | --- |
| `host` | `127.0.0.1` | Listen address (IPv6 allowed; will be bracketed in URLs) |
| `port` | `9208` release / `19208` debug | Change if the port is taken |
| `local_api_key` | `null` | When set: local auth uses format-specific headers (see Auth rules); local auth inputs are **not** forwarded upstream. |
| `app_proxy_url` | `null` | Proxy for app updater & as placeholder for upstreams (`"$app_proxy_url"`). Supports `http/https/socks5/socks5h`. |
| `log_level` | `silent` | `silent|error|warn|info|debug|trace`; debug/trace log request headers (auth redacted) and small bodies (≤64KiB). Release builds force `silent`. |
| `max_request_body_bytes` | `20971520` (20 MiB) | 0 = fallback to default. Protects inbound body size. |
| `retryable_failure_cooldown_secs` | `15` | Cooldown window after retryable failures that should temporarily sideline an upstream. `0` disables cooldown. Reloading or restarting the running proxy resets current cooldown state. |
| `tray_token_rate.enabled` | `true` | macOS tray live rate; harmless elsewhere. |
| `tray_token_rate.format` | `split` | `combined` (`total`), `split` (`↑in ↓out`), `both` (`total | ↑in ↓out`). |
| `upstream_strategy` | `{ "order": "fill_first", "dispatch": { "type": "serial" } }` | Structured strategy object. `order` controls candidate ordering inside one priority group; `dispatch` controls serial / hedged / race execution. |
### Upstream entries (`upstreams[]`)
| Field | Default | Notes |
| --- | --- | --- |
| `id` | required | Unique per upstream. |
| `providers` | required | One upstream can serve multiple providers. Special providers `kiro/codex` cannot be mixed with others. |
| `base_url` | required | Full base; overlapping path parts are de-duplicated. (`providers=["kiro"]` / `["codex"]` can be empty.) |
| `api_key` | `null` | Provider-specific bearer/key; overrides request headers. |
| `kiro_account_id` | `null` | Required when `providers=["kiro"]`. |
| `preferred_endpoint` | `null` | `kiro` only (`providers=["kiro"]`): `ide` or `cli`. |
| `proxy_url` | `null` | Per-upstream proxy; supports `http/https/socks5/socks5h`; default is **no system proxy**. `$app_proxy_url` placeholder allowed. |
| `priority` | `0` | Higher = tried earlier. Grouped by priority then by order (or round-robin). |
| `enabled` | `true` | Disabled upstreams are skipped. |
| `model_mappings` | `{}` | Exact / `prefix*` / `*`. Priority: exact > longest prefix > wildcard. Response echoes original alias. |
| `convert_from_map` | `{}` | Explicitly allow inbound format conversion per provider. Example: `{ "openai-response": ["openai_chat", "anthropic_messages"] }`. |
| `overrides.header` | `{}` | Set/remove headers (null removes). Hop-by-hop/Host/Content-Length are always ignored. |
## Routing & format conversion
- Gemini native API: `/v1beta/models/*` (including `:generateContent`, `:streamGenerateContent`, `:countTokens`, `:embedContent`, `:batchEmbedContents`), model catalog/detail, `/v1beta/files*`, `/upload/v1beta/files*`, `/v1beta/cachedContents*`, `/v1beta/tunedModels*` → `gemini`.
- Anthropic: `/v1/messages` (and subpaths) and `/v1/complete` → `anthropic` (Kiro shares the same format).
- OpenAI create routes: `/v1/chat/completions` → `openai`; `/v1/responses` → `openai-response`.
- OpenAI native pass-through routes are explicitly pinned to OpenAI-compatible providers and won't fall through to `anthropic`: `chat/completions/*`, `responses/*`, `assistants*`, `threads*`, `conversations*`, `chatkit*`, `containers*`, `evals*`, `files*`, `uploads*`, `batches*`, `vector_stores*`, `images/*`, `audio/*`, `embeddings`, `moderations`, `completions`, `fine_tuning/*`, `realtime/*`, `skills*`, `videos*`.
- For `responses/*` resources, provider preference is `openai-response` → `openai`; for other OpenAI native resources, provider preference is `openai` → `openai-response`.
- Other paths: choose the provider with the highest configured priority; tie-break is `openai` > `openai-response` > `anthropic`.
- Cross-format fallback/conversion is controlled by `upstreams[].convert_from_map` (no global switch). If a provider has no eligible upstream for the inbound format, it won't be selected.
- If `openai` is missing for `/v1/chat/completions`: fallback can be `openai-response`, `anthropic`, or `gemini` (priority-based; tie-break prefers `openai-response`).
- For `/v1/messages`: choose between `anthropic` and `kiro` by priority; tie-break uses upstream id. If the chosen provider returns a retryable error, the proxy will fall back to the other native provider (Anthropic ↔ Kiro) when configured.
- If neither `anthropic` nor `kiro` exists for `/v1/messages`: other providers can be selected only when allowed for `anthropic_messages` via `convert_from_map` (e.g. `openai-response`, `openai`, `gemini`).
- If `openai-response` is missing for `/v1/responses`: fallback can be `openai`, `anthropic`, or `gemini` (priority-based; tie-break prefers `openai`).
- If `gemini` is missing for `/v1beta/models/*:generateContent` or `*:streamGenerateContent`: fallback can be `openai-response`, `openai`, or `anthropic` (priority-based; tie-break prefers `openai-response`).
- Other Gemini native endpoints are pass-through only and require a configured `gemini` upstream.
## Auth rules (important)
- Local access: `local_api_key` enabled → require format-specific key. These local auth inputs are stripped and **not** forwarded upstream.
- OpenAI / Responses: `Authorization: Bearer `
- Anthropic `/v1/messages`: `x-api-key` or `x-anthropic-api-key`
- Gemini native API: `x-goog-api-key` or `?key=...`
- When `local_api_key` is enabled, request headers are **not** used for upstream auth; configure `upstreams[].api_key` instead.
- Upstream auth resolution (per request):
- **OpenAI**: `upstream.api_key` → `x-openai-api-key` → `Authorization` (only if `local_api_key` is **not** set) → error.
- **Anthropic**: `upstream.api_key` → `x-api-key` / `x-anthropic-api-key` → error. Missing `anthropic-version` is auto-filled with `2023-06-01`.
- **Gemini**: `upstream.api_key` → `x-goog-api-key` → query `?key=...` → error.
## Load balancing & retries
- Priorities: higher `priority` groups first.
- `upstream_strategy.order` controls selection inside the same priority group:
- `fill_first`: keep the configured list order.
- `round_robin`: rotate the starting point across requests.
- `upstream_strategy.dispatch` controls how requests are launched inside one priority group:
- `{"type":"serial"}`: try one candidate at a time.
- `{"type":"hedged","delay_ms":2000,"max_parallel":2}`: launch the first candidate immediately, then add one more attempt after `delay_ms` if the prior attempt is still unresolved, up to `max_parallel`.
- `{"type":"race","max_parallel":3}`: launch up to `max_parallel` candidates immediately and take the first successful result.
- Retryable conditions: network timeout/connect errors, or status 400/401/403/404/408/422/429/307/5xx (including 504/524). Retries stay within the same provider's priority groups.
- Cooldown conditions: `401/403/408/429/5xx` will temporarily move the failed upstream behind ready peers for `retryable_failure_cooldown_secs` (default `15`); `400/404/422/307` stay retryable but do not trigger cross-request cooldown.
- `/v1/messages` only: after the chosen native provider is exhausted (retryable errors), the proxy can fall back to the other native provider (`anthropic` ↔ `kiro`) if it is configured.
## Observability
- SQLite log: `data.db` in config dir. Stores per-request stats (tokens, cached tokens, latency, model, upstream).
- Token rate: macOS tray shows live total or split rates (configurable via `tray_token_rate`).
- Debug/trace log bodies capped at 64KiB.
## Dashboard
- In-app **Dashboard** page visualizes totals, per-provider stats, time series, and recent requests (page size 50, offset supported).
- The Logs panel supports a 30-second request-detail capture window: when enabled it stores request headers/bodies during that window, always keeps error responses for failed requests, and turns off automatically afterward.
## One-click CLI setup
- Claude Code: writes `~/.claude/settings.json` `env` (`ANTHROPIC_BASE_URL`, `ANTHROPIC_AUTH_TOKEN` when local key is set).
- Codex: writes `~/.codex/config.toml` `model_provider="token_proxy"` and `[model_providers.token_proxy].base_url` → `http://127.0.0.1:/v1`; writes `~/.codex/auth.json` `OPENAI_API_KEY`.
- OpenCode: writes `~/.config/opencode/opencode.json[c]` `provider.token_proxy` and `options.baseURL` → `http://127.0.0.1:/v1`; writes `~/.local/share/opencode/auth.json` `token_proxy.key`. This requires at least one **exact model mapping** in Upstreams (no `*`).
- A `.token_proxy.bak` file is created before overwriting; restart the CLI to apply.
## FAQ
- **Port already in use?** Change `port` in `config.jsonc`; remember to update your client base URL.
- **Got 401?** If `local_api_key` is set, you must send the format-specific local key (OpenAI/Responses: `Authorization`, Anthropic: `x-api-key`, Gemini: `x-goog-api-key` or `?key=`). With local auth enabled, configure upstream keys in `upstreams[].api_key`.
- **Got 504?** Upstream did not send response headers or the first body chunk within 120s. For streaming responses, a 120s idle timeout between chunks may also close the connection.
- **413 Payload Too Large?** Body exceeded `max_request_body_bytes` (default 20 MiB) or the 4 MiB transform limit for format-conversion requests.
- **Why no system proxy?** By design, `reqwest` is built with `.no_proxy()`; set per-upstream `proxy_url` if needed.