{"id":37290630,"url":"https://github.com/sxueck/llm-gateway","last_synced_at":"2026-04-12T14:15:20.922Z","repository":{"id":318802641,"uuid":"1070124414","full_name":"sxueck/llm-gateway","owner":"sxueck","description":"Lightweight distributed LLM gateway w/ web UI for model mgmt \u0026 routing. Supports vibe programming, prompt opt., \u0026 optimized OpenAI API/Anthropic calls","archived":false,"fork":false,"pushed_at":"2026-02-01T14:46:08.000Z","size":40793,"stargazers_count":32,"open_issues_count":1,"forks_count":6,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-01T23:40:45.626Z","etag":null,"topics":["anthropic-claude","llm-gateway","llmops","openai","vibe-coding"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sxueck.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-10-05T10:16:04.000Z","updated_at":"2026-02-01T14:23:53.000Z","dependencies_parsed_at":"2026-01-19T04:01:00.464Z","dependency_job_id":null,"html_url":"https://github.com/sxueck/llm-gateway","commit_stats":null,"previous_names":["sxueck/llm-gateway"],"tags_count":27,"template":false,"template_full_name":null,"purl":"pkg:github/sxueck/llm-gateway","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxueck%2Fllm-gateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxueck%2Fllm-gateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxueck%2Fllm-gateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxueck%2Fllm-gateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sxueck","download_url":"https://codeload.github.com/sxueck/llm-gateway/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sxueck%2Fllm-gateway/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29290477,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-10T03:42:42.660Z","status":"ssl_error","status_checked_at":"2026-02-10T03:42:41.897Z","response_time":65,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["anthropic-claude","llm-gateway","llmops","openai","vibe-coding"],"created_at":"2026-01-16T02:30:12.855Z","updated_at":"2026-04-12T14:15:20.896Z","avatar_url":"https://github.com/sxueck.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LLM Gateway\n\n\u003cdiv align=\"center\"\u003e\n\n[![GitHub Wiki](https://img.shields.io/badge/docs-wiki-blue?style=flat-square)](https://github.com/sxueck/llm-gateway/wiki)\n[![License: MIT](https://img.shields.io/badge/license-MIT-green.svg?style=flat-square)](LICENSE)\n[![Node.js](https://img.shields.io/badge/node-\u003e=v22-339933?style=flat-square\u0026logo=node.js)](https://nodejs.org/)\n[![Bun](https://img.shields.io/badge/bun-\u003e=v1.0-black?style=flat-square\u0026logo=bun)](https://bun.sh/)\n\n[English](./README_EN.md) | **中文**\n\n\u003c/div\u003e\n\n\u003e 生产级别 LLM 网关管理系统，部署三个月已稳定负载超过 **50亿 Token** 的任务处理（持续累积中）\n\u003e\n\u003e 提供直观的 Web UI 界面，用于管理多个 LLM 提供商、虚拟密钥、路由配置和模型管理\n\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"80%\" alt=\"Dashboard\" src=\"https://github.com/user-attachments/assets/a69d7e89-5225-4c2e-bae3-d11faddc9b56\" /\u003e\n\u003c/p\u003e\n\u003cp align=\"center\"\u003e\n  \u003cimg width=\"80%\" alt=\"Health Monitoring\" src=\"https://github.com/user-attachments/assets/196adf78-2346-41f9-903b-a18920464486\" /\u003e\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"./docs/screenshot.md\"\u003e更多截图\u003c/a\u003e\n\u003c/p\u003e\n\n---\n\n## 目录\n\n- [特性](#特性)\n- [快速开始](#快速开始)\n- [健康监控](#健康监控)\n- [贡献](#贡献)\n- [许可证](#许可证)\n- [致谢](#致谢)\n\n---\n\n## 特性\n\n| 功能 | 描述 |\n|------|------|\n| **提供商管理** | 支持 20+ 主流 LLM 提供商：OpenAI、Anthropic、Google、DeepSeek 等 |\n| **虚拟密钥** | 创建和管理虚拟 API 密钥，支持速率限制和访问控制 |\n| **路由配置** | 负载均衡和故障转移策略，提高服务可用性 |\n| **模型管理** | 统一管理所有提供商的模型，支持批量导入和自定义配置 |\n| **多端点支持** | 兼容 `/v1/chat/completions`、`/v1/responses`、`/v1/messages` 等端点 |\n| **健康监控** | 免登录健康检查页面，实时展示模型可用率、延迟分位数（P50/P95）和错误分布 |\n| **用户认证** | 基于 JWT 的安全认证机制 |\n| **实时监控** | 仪表盘展示系统状态和配置信息 |\n| **中转站支持** | 隔离 Codex 等上游强制注入的提示词，使下游应用对 Prompt 遵循更规范 |\n| **内置 PII 保护** | 自动检测和脱敏请求中的个人身份信息，支持流式响应还原 |\n\n---\n\n## 快速开始\n\n### 前置要求\n\n| 依赖 | 版本要求 | 说明 |\n|------|----------|------|\n| Node.js | \u003e= v22 | 运行时环境 |\n| Bun | \u003e= v1.0 | Monorepo 脚本基于 Bun workspaces |\n| MySQL | 8.x | 数据库（或兼容 MySQL 协议） |\n| Docker | - | 可选，用于容器化部署 |\n| 最低配置 | 1C2G | 开启 PII 隐私保护等计算密集功能需提升配置 |\n\n### 安装\n\n```bash\n# 克隆仓库\ngit clone https://github.com/sxueck/llm-gateway.git\ncd llm-gateway\n\n# 安装依赖（包含 packages/backend 与 packages/web）\nbun install\n```\n\n### 配置\n\n创建 `.env` 文件并配置环境变量：\n\n```bash\ncp .env.example .env\n```\n\n编辑 `.env` 文件（至少需要配置 MySQL 与 `JWT_SECRET`）：\n\n```env\nPORT=3000\nNODE_ENV=development\nLOG_LEVEL=info\nJWT_SECRET=your-secret-key-change-this-in-production\n\n# MySQL 数据库配置\nMYSQL_HOST=localhost\nMYSQL_PORT=3306\nMYSQL_USER=root\nMYSQL_PASSWORD=your-mysql-password\nMYSQL_DATABASE=llm_gateway\n```\n\n\u003e **重要**: 生产环境请务必修改 `JWT_SECRET` 为一个强随机字符串（至少 32 字符）\n\n### 启动服务\n\n```bash\n# 同时启动后端(3000)与前端(5173)\nbun run dev:all\n```\n\n| 服务 | 地址 |\n|------|------|\n| Web UI | http://localhost:5173 |\n| Backend API | http://localhost:3000 |\n\n**单独启动：**\n\n```bash\n# 仅后端\nbun run dev:backend\n\n# 仅前端\nbun run dev:web\n```\n\n**生产构建与启动（前后端分离部署）：**\n\n```bash\n# 构建前后端\nbun run build\n\n# 启动后端（生产模式）\nbun run start\n```\n\n\u003e 提示：前端产物位于 `packages/web/dist`，请使用 Nginx/静态文件服务单独部署\n\n### Docker Compose 部署\n\n请参考 [Docker 部署指南](./docs/docker-deployment.md)\n\n### 快速使用\n\n1. **添加供应商** - 添加类似 DeepSeek 的 AI 服务商，并填入供应商密钥\n2. **添加模型** - 添加供应商提供的 AI 模型（如 DeepSeek 的 `deepseek-chat`）\n3. **创建虚拟密钥** - 用于访问 LLM Gateway 的 API\n4. **(可选) 配置 Prompt 管理规则** - 实现 prompt 的动态修改和增强\n5. **使用虚拟密钥访问 API** - 在应用中调用 LLM Gateway\n\n---\n\n## 健康监控\n\nLLM Gateway 提供公开的健康监控页面，无需登录即可访问，实时展示各模型的可用性和性能指标。\n\n### 启用持久监控\n\n1. 在\"系统设置 -\u003e 监控设置\"中开启\"启用持久监控\"\n2. 首次开启时系统会自动创建一个\"监控专用虚拟密钥\"：\n   - 具备访问全部模型的能力\n   - 仅用于健康检查\n   - 默认禁用请求体/响应体日志\n\n**只有当\"持久监控\"为开启状态时，以下能力才会生效：**\n- 后端健康检查调度器运行（周期性对目标执行健康检查）\n- 公开监控页面与相关免鉴权 API 可访问\n\n\u003e 提示：可随时关闭\"持久监控\"，系统将停止调度器并关闭公开访问（公开端点返回 404）\n\n### 访问健康监控页面\n\n访问 `http://your-gateway-url/status` 即可查看健康监控页面。\n\n### 功能特性\n\n- **实时状态监控** - 显示所有配置的模型目标的当前状态（健康/降级/宕机）\n- **可用率统计** - 展示 1小时 和 24小时 的可用率\n- **延迟指标** - P50 和 P95 延迟统计\n- **错误追踪** - 记录并展示错误类型和错误信息\n- **自动刷新** - 默认每 60 秒自动刷新数据\n- **详细历史** - 点击目标可查看详细的检查历史记录\n\n### API 端点\n\n健康监控提供以下公开 API 端点（免鉴权）：\n\n| 端点 | 描述 |\n|------|------|\n| `GET /public/health/summary` | 获取所有目标的汇总信息 |\n| `GET /public/health/targets` | 获取目标清单 |\n| `GET /public/health/detail?target_id=xxx` | 获取单个目标的详细信息 |\n| `GET /public/health/runs?target_id=xxx\u0026window=24h\u0026page=1\u0026page_size=50` | 获取检查历史记录 |\n\n### 配置健康检查目标\n\n健康检查目标通过数据库配置，可以为任何模型或虚拟模型设置健康检查：\n\n```sql\n-- 示例：为模型添加健康检查\nINSERT INTO health_targets (id, name, type, target_id, enabled, check_interval_seconds, check_prompt)\nVALUES ('target-1', 'DeepSeek Chat', 'model', 'model-id-here', 1, 300, 'Say \"OK\"');\n```\n\n**主要配置参数：**\n\n| 参数 | 说明 | 默认值 |\n|------|------|--------|\n| `check_interval_seconds` | 检查频率（秒） | 300 秒（5分钟） |\n| `check_prompt` | 健康检查使用的提示词 | `\"Say 'OK'\"` |\n| `check_config` | JSON 配置，可设置超时、重试等参数 | - |\n\n### 限流保护\n\n健康监控 API 默认启用限流保护：\n- 每个 IP 每分钟最多 60 个请求\n- 超过限制将返回 429 错误\n\n---\n\n## 贡献\n\n欢迎提交 Issue 和 Pull Request！\n\n---\n\n## 许可证\n\n[MIT License](./LICENSE) - LLM Gateway\n\n---\n\n## 致谢\n\n- [Naive UI](https://www.naiveui.com/) - UI 组件库\n- [Fastify](https://www.fastify.io/) - 高性能 Web 框架\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsxueck%2Fllm-gateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsxueck%2Fllm-gateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsxueck%2Fllm-gateway/lists"}