{"id":51449312,"url":"https://github.com/leeguooooo/cloudflare-llm-gateway","last_synced_at":"2026-07-05T19:01:20.363Z","repository":{"id":367674260,"uuid":"1281823570","full_name":"leeguooooo/cloudflare-llm-gateway","owner":"leeguooooo","description":"A serverless, OpenAI-compatible LLM API gateway for Cloudflare Workers — a new-api / one-api alternative that runs on the free tier (Workers + D1, no server).","archived":false,"fork":false,"pushed_at":"2026-06-27T03:05:02.000Z","size":3748,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-27T04:19:36.325Z","etag":null,"topics":["ai","cloudflare-workers","key-pool","llm-gateway","new-api","one-api","openai-api","serverless"],"latest_commit_sha":null,"homepage":null,"language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/leeguooooo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-27T01:06:47.000Z","updated_at":"2026-06-27T03:05:06.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/leeguooooo/cloudflare-llm-gateway","commit_stats":null,"previous_names":["leeguooooo/cloudflare-llm-gateway"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/leeguooooo/cloudflare-llm-gateway","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leeguooooo%2Fcloudflare-llm-gateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leeguooooo%2Fcloudflare-llm-gateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leeguooooo%2Fcloudflare-llm-gateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leeguooooo%2Fcloudflare-llm-gateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/leeguooooo","download_url":"https://codeload.github.com/leeguooooo/cloudflare-llm-gateway/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/leeguooooo%2Fcloudflare-llm-gateway/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35165568,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-05T02:00:06.290Z","response_time":100,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","cloudflare-workers","key-pool","llm-gateway","new-api","one-api","openai-api","serverless"],"created_at":"2026-07-05T19:01:19.381Z","updated_at":"2026-07-05T19:01:20.319Z","avatar_url":"https://github.com/leeguooooo.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# cloudflare-llm-gateway\n\nA serverless, **OpenAI-compatible LLM API gateway** for **Cloudflare Workers** — a\n[new-api](https://github.com/QuantumNous/new-api) / [one-api](https://github.com/songquanpeng/one-api)\nstyle gateway that runs entirely on the **free tier** (Workers + D1, no server, no Redis).\n\n![cloudflare-llm-gateway — pool many AI keys behind one OpenAI-compatible API on Cloudflare](assets/hero.png)\n\nPool many upstream AI API keys behind one stable OpenAI-compatible endpoint.\nDead keys auto-disable, rate-limited keys cool down with progressive backoff and\nauto-revive, requests fall back across providers so callers never see a 503, and\nyou get a built-in admin/consumer console — all on the free tier.\n\n## Features\n\n- **9 providers, OpenAI-compatible** — Gemini, Mistral, OpenRouter, OpenAI, DeepSeek, Groq, Moonshot (Kimi), GLM (Zhipu), Qwen (DashScope). One `POST /v1/chat/completions` + `GET /v1/models`; route by model name or an explicit `provider:model` prefix. Native passthrough at `/gemini/*`, `/mistral/*`, … too.\n- **Self-healing key pool** — round-robin across active keys; `401/403`/arrears → auto-disable, `429`/quota → cooldown with **progressive backoff**, success → revive. Liveness probes use a real 1-token call (not a `/models` endpoint that lies about suspended/throttled accounts).\n- **Auto-fallback** — if the chosen provider is down/throttled, the request is transparently served by another provider with live keys (response carries `X-KeyPool-Provider/Model/Fallback`); opt out per request with `{\"fallback\": false}`. Callers never get a 503.\n- **Model-level availability** — models that a key can't actually serve (paid-only, not-found) are probed and hidden from `GET /v1/models`, so users only pick models that work.\n- **Per-key console** — every key (masked) with status, stats, last error, and a live \"检测全部\" batch check (OpenRouter/DeepSeek show real balance); enable/disable/delete; disabled keys hidden by default.\n- **Roles + SSO** — optional OIDC SSO (Authorization Code + PKCE) with an admin-approval gate, or simple bearer-token auth. Admins manage the pool and aren't metered; consumers self-serve their own tokens.\n- **Metering \u0026 billing** — per-request + per-token usage with daily charts and logs; optional credit billing (per-model input/output pricing, a global discount, per-token deduction) with Stripe top-up. Off by default.\n- **Token controls** — per-token quota, RPM limit, and expiry.\n- **Built-in chat** — consumer chat playground (streaming) and an admin per-key debug chat.\n- **$0 to run** — Cloudflare Workers + D1 free tier; unattended health-check via cron or an included GitHub Actions schedule.\n\n## Architecture\n\n```\nclient ──▶ Worker (Hono) ──▶ pick active key ──▶ upstream provider\n                │                                    (Gemini/Mistral/OpenRouter)\n                ├─ D1: keys, tokens, users, logs\n                └─ cron / POST /admin/probe: revive cooldowns, probe disabled keys\n```\n\nKeys self-heal: a `429`/quota error cools a key down, a `401`/`403` disables it,\nand both auto-revive (on a successful retry, or via the cron / `/admin/probe`).\n\n![self-healing key pool: active → cooldown on 429, active → disabled on 401, both auto-revive back to active](assets/self-healing.png)\n\n## Setup\n\n```bash\nnpm install\n\n# 1. create the D1 database, then paste its id into wrangler.toml\nnpx wrangler d1 create llm-gateway\ncp wrangler.toml.example wrangler.toml      # edit database_id (+ optional SSO vars)\n\n# 2. apply the schema\nnpx wrangler d1 execute llm-gateway --remote --file=./schema.sql\n\n# 3. set secrets\nnpx wrangler secret put ADMIN_TOKEN         # bearer token for admin/API\nnpx wrangler secret put SESSION_SECRET      # random 32+ bytes (for SSO sessions)\n\n# 4. deploy\nnpx wrangler deploy\n```\n\nOpen the deployed URL to reach the admin console.\n\n## API\n\n| Method \u0026 path | Auth | Purpose |\n|---|---|---|\n| `POST /v1/chat/completions` | user token / session | OpenAI-compatible chat |\n| `GET /v1/models` | user token / session | list available models |\n| `/gemini/*` `/mistral/*` `/openrouter/*` | user token / session | native passthrough |\n| `POST /admin/keys/import` | admin | bulk-import `provider:key` lines |\n| `GET /admin/keys/list` | admin | every key + stats |\n| `POST /admin/keys/:id/check` | admin | live liveness / balance probe |\n| `POST /admin/keys/:id/enable\\|disable`, `DELETE /admin/keys/:id` | admin | per-key ops |\n| `GET /admin/users`, `POST /admin/users/:id/approve\\|block` | admin | approve consumers |\n| `GET/POST/DELETE /me/tokens` | session | consumers manage their own tokens |\n| `POST /admin/check-all-keys` | admin | batch probe: revive healthy, disable dead |\n| `POST /admin/probe-models` / `GET /admin/models-status` | admin | model-level availability |\n| `GET /admin/usage` `/admin/logs` `/me/usage` `/me/logs` | admin / session | metering |\n| `GET /admin/balances` `POST /admin/balances/:sub/topup` `/admin/prices` | admin | billing |\n| `POST /me/checkout` `POST /stripe/webhook` | session / none | Stripe credit top-up |\n\n### Import keys\n\n```bash\ncurl https://YOUR_WORKER/admin/keys/import \\\n  -H \"Authorization: Bearer $ADMIN_TOKEN\" \\\n  -H 'content-type: application/json' \\\n  -d '{\"keys\":\"gemini:AIza...\\nmistral:...\\nopenrouter:sk-or-v1-...\"}'\n```\n\n### Use it\n\n```bash\ncurl https://YOUR_WORKER/v1/chat/completions \\\n  -H \"Authorization: Bearer \u003cuser-token\u003e\" \\\n  -H 'content-type: application/json' \\\n  -d '{\"model\":\"mistral-small-latest\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}'\n```\n\n## Notes\n\n- **Terms of service:** aggregating / sharing / reselling provider API keys may\n  violate the upstream providers' terms. Use it to pool **your own** keys, and\n  review each provider's ToS before exposing it to others.\n- SSO is optional — leave `OIDC_*` unset to run with `ADMIN_TOKEN` + minted\n  tokens only.\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleeguooooo%2Fcloudflare-llm-gateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fleeguooooo%2Fcloudflare-llm-gateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fleeguooooo%2Fcloudflare-llm-gateway/lists"}