{"id":50881476,"url":"https://github.com/wangjiehu/llm-api-pool","last_synced_at":"2026-06-15T13:03:09.468Z","repository":{"id":362412663,"uuid":"1258566581","full_name":"wangjiehu/llm-api-pool","owner":"wangjiehu","description":"OpenAI + Anthropic compatible LLM API pool/proxy with web session support (Gemini/Claude/ChatGPT/Codex), smart routing, real-time quota monitoring, high-freq agent ready. Local exe or self-host.","archived":false,"fork":false,"pushed_at":"2026-06-04T04:22:59.000Z","size":159,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-04T06:11:17.216Z","etag":null,"topics":["agent","ai-pool","anthropic","claude","codex","fastapi","gemini","llm","openai","playwright","proxy"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wangjiehu.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-03T17:47:27.000Z","updated_at":"2026-06-04T04:23:03.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/wangjiehu/llm-api-pool","commit_stats":null,"previous_names":["wangjiehu/llm-api-pool"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/wangjiehu/llm-api-pool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangjiehu%2Fllm-api-pool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangjiehu%2Fllm-api-pool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangjiehu%2Fllm-api-pool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangjiehu%2Fllm-api-pool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wangjiehu","download_url":"https://codeload.github.com/wangjiehu/llm-api-pool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wangjiehu%2Fllm-api-pool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34363557,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-15T02:00:07.085Z","response_time":63,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agent","ai-pool","anthropic","claude","codex","fastapi","gemini","llm","openai","playwright","proxy"],"created_at":"2026-06-15T13:03:09.357Z","updated_at":"2026-06-15T13:03:09.446Z","avatar_url":"https://github.com/wangjiehu.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM API Pool\n\nLocal Windows software gateway for multiple LLM API keys and account sessions.\n\nThe app runs a FastAPI server, opens a desktop-style dashboard, and exposes OpenAI-compatible and Anthropic-compatible endpoints for tools such as Cursor, Continue, Aider, Claude Code, Cline, and custom scripts.\n\nRepository: https://github.com/wangjiehu/llm-api-pool\n\n## What It Provides\n\n- Local API server at `http://localhost:8080`.\n- OpenAI-compatible endpoint: `POST /v1/chat/completions`.\n- Anthropic-compatible endpoint: `POST /v1/messages`.\n- Model list endpoint: `GET /v1/models`.\n- Official API key channels for OpenAI, Anthropic, and Gemini.\n- Advanced web-session channels for ChatGPT, Codex-style GPT usage, Claude, and Gemini through Playwright browser automation.\n- Smart routing by provider compatibility, health, quota estimate, latency, in-flight load, priority, and cooldown state.\n- Self-contained dashboard with account health, quota estimate, latency, in-flight load, and a playground.\n- Admin diagnostics export for sanitized runtime, channel, router, browser, and recent event state.\n- Portable Windows `--onedir` build for faster startup than PyInstaller onefile extraction.\n\nOfficial API channels are the recommended production path. Web-session channels are useful for personal quota pooling, but they are inherently more fragile because provider pages, cookies, 2FA, captcha, and browser automation can change.\n\n## Release Packages\n\nUse the package that matches your Windows device:\n\n- `llm-pool-windows-x64.zip`: primary Windows x64 package.\n- Each zip is published with a `.sha256` file.\n\nWindows on Arm users should use the x64 package under Windows' built-in x64 emulation. A native ARM64 package is not published yet because the full native dependency stack is not currently installable in the `windows-11-arm` runner with this dependency set; the current probe fails before packaging while building native dependencies such as `cryptography`, and Playwright is therefore not available either.\n\nDo not use a 32-bit x86 package; this project depends on Playwright/Chromium and modern Python packages, so 32-bit Windows is not a sensible support target.\n\n## Quick Start: Windows App\n\n1. Download the release zip for your architecture.\n2. Extract it.\n3. Run `llm-pool.exe` inside the extracted `llm-pool` folder.\n4. The dashboard opens automatically.\n5. Add an official API key first; add web-session channels only if you need them.\n6. Use `http://localhost:8080/v1` as the base URL in your tool.\n\nWhen no `ADMIN_TOKEN` is configured, the app generates a random local admin token at startup and injects it only into the loopback dashboard. This keeps double-click local usage smooth while still requiring a token for admin API calls.\n\nRuntime data such as `channels.json` and Playwright profiles is stored next to the executable. API keys and cookies in `channels.json` are encrypted with Windows DPAPI for the current Windows user. Do not share that app folder if it contains personal accounts or browser profiles.\n\n## Quick Start: Source\n\n```powershell\npip install -r requirements.txt\npython main.py\n```\n\nUseful options:\n\n```powershell\npython main.py --host 127.0.0.1 --port 8080\npython main.py --no-open\npython main.py --install-browser\n```\n\n## API Usage\n\nOpenAI-compatible:\n\n```bash\ncurl http://localhost:8080/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"auto\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello from the pool\"}],\n    \"stream\": false\n  }'\n```\n\nAnthropic-compatible:\n\n```bash\ncurl http://localhost:8080/v1/messages \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"auto\",\n    \"max_tokens\": 1024,\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello from the pool\"}],\n    \"stream\": false\n  }'\n```\n\nIf `API_TOKEN` is configured, send it as either:\n\n```text\nAuthorization: Bearer \u003ctoken\u003e\n```\n\nor:\n\n```text\nX-Api-Key: \u003ctoken\u003e\n```\n\n## Accounts\n\nOfficial channels:\n\n- `official_openai`\n- `official_claude`\n- `official_gemini`\n\nAdvanced web-session channels:\n\n- `web_chatgpt`\n- `web_codex`\n- `web_claude`\n- `web_gemini`\n\nFor web-session channels, pasted cookies are more reliable than password login when an account uses 2FA, captcha, SSO, or device verification. Password login is best-effort browser automation and may fail when the provider changes its login UI.\n\n## Remote Hosting\n\nRemote hosting must be explicit. The app refuses non-local binds unless both `ADMIN_TOKEN` and `API_TOKEN` are set:\n\n```powershell\n$env:HOST=\"0.0.0.0\"\n$env:PORT=\"8080\"\n$env:ADMIN_TOKEN=\"replace-with-strong-admin-token\"\n$env:API_TOKEN=\"replace-with-strong-api-token\"\n$env:CORS_ORIGINS=\"https://your-dashboard.example\"\npython main.py\n```\n\nSecurity defaults:\n\n- `/admin/*` always requires `ADMIN_TOKEN`.\n- `/v1/*` requires `API_TOKEN` when configured, and remote mode requires it.\n- Tokens are accepted in headers, not URL query strings.\n- API requests are rate-limited by client/token/path. Default: `RATE_LIMIT_PER_MINUTE=120`.\n- CORS defaults to localhost only. Add exact external origins through `CORS_ORIGINS`.\n- `/admin/diagnostics` returns a sanitized JSON snapshot for debugging. It redacts API keys, passwords, cookies, and tokens.\n- Set `DEBUG_ERRORS=1` only while debugging packaging issues locally.\n\n## Diagnostics\n\nUse the dashboard `Diagnostics` button or call:\n\n```powershell\nInvoke-RestMethod http://localhost:8080/admin/diagnostics -Headers @{\"X-Admin-Token\"=\"\u003ctoken\u003e\"}\n```\n\nThe export includes runtime information, security posture, data-file location, channel health, router state, browser context count, and recent sanitized events. It is intended for local debugging and issue reports; it does not include raw prompts, API keys, cookies, passwords, or tokens.\n\n## Dashboard and GitHub Pages\n\n`dashboard.html` is self-contained and has no external CSS or JavaScript dependency. It works in the packaged app and can also be hosted as a static page.\n\nFor a static dashboard:\n\n1. Publish `dashboard.html` as `index.html`.\n2. Open the page.\n3. Set the backend URL to your running local or remote server.\n4. Enter the admin token.\n5. Configure `CORS_ORIGINS` on the backend if the page is not served from localhost.\n\nThe static page does not store accounts by itself. API keys, cookies, and channel data stay on the backend.\n\n## Build Windows Portable App\n\nRun:\n\n```powershell\n.\\build_exe.bat\n```\n\nThe script discovers Conda dynamically. If a `happy` environment exists, it uses it; otherwise it falls back to `base`, then to `python` on PATH.\n\nOutput:\n\n```text\ndist\\llm-pool\\llm-pool.exe\ndist\\llm-pool-windows-x64.zip\ndist\\llm-pool-windows-x64.zip.sha256\nllm-pool-launch.bat\n```\n\nThe GitHub Actions workflow builds the x64 package, launches the frozen exe, checks `/health`, verifies the dashboard HTML, local admin-token bootstrap, and `/admin/diagnostics`, zips the build, and uploads SHA256 files on releases. A separate ARM64 compatibility probe tracks whether native ARM64 packaging has become feasible.\n\n`Dependency Probe` runs weekly and on demand with the current unlocked `requirements.txt` set. It validates imports, web-channel contracts, `pip check`, and Playwright Chromium installability so upstream dependency drift is visible before a release rebuild.\n\n## Files\n\n- `main.py`: FastAPI server, routing, provider backends, Playwright web sessions, security, CLI.\n- `dashboard.html`: self-contained dashboard and playground.\n- `requirements.txt`: runtime dependencies.\n- `requirements-lock.txt`: resolved dependency lock used for reproducible CI builds when present.\n- `build_exe.bat`: local Windows build script.\n- `.github/workflows/build-exe.yml`: release/manual Windows x64 build workflow plus ARM64 compatibility probe.\n- `.github/workflows/dependency-probe.yml`: scheduled unlocked dependency drift check.\n- `scripts/validate_web_contracts.py`: local/CI contract check for channel mappings, routes, and dashboard hooks.\n- `examples/usage.md`: short integration examples.\n\n## Known Limits\n\n- Web UI selectors may need updates when providers change their pages.\n- Web streaming is best-effort because it polls rendered browser output.\n- DPAPI-encrypted `channels.json` secrets are tied to the current Windows user.\n- Official Gemini support uses Google's current `google-genai` SDK.\n- Tool calls, vision, and file inputs are normalized but not fully implemented for every backend.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwangjiehu%2Fllm-api-pool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwangjiehu%2Fllm-api-pool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwangjiehu%2Fllm-api-pool/lists"}