{"id":28600903,"url":"https://github.com/labring/aiproxy","last_synced_at":"2026-06-06T09:01:46.709Z","repository":{"id":281621348,"uuid":"945849668","full_name":"labring/aiproxy","owner":"labring","description":"AI Proxy is a high performance AI gateway using OpenAI / Claude / Gemini protocol as the entry point. It features intelligent error handling, multi-channel management, and comprehensive monitoring. With support for multiple models, rate limiting, and multi-tenant isolation.","archived":false,"fork":false,"pushed_at":"2026-05-25T11:24:38.000Z","size":6087,"stargazers_count":464,"open_issues_count":11,"forks_count":94,"subscribers_count":6,"default_branch":"main","last_synced_at":"2026-05-25T13:22:56.444Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/labring.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-10T08:19:55.000Z","updated_at":"2026-05-25T11:24:43.000Z","dependencies_parsed_at":"2026-03-11T06:04:38.807Z","dependency_job_id":null,"html_url":"https://github.com/labring/aiproxy","commit_stats":null,"previous_names":["zijiren233/aiproxy","labring/aiproxy"],"tags_count":65,"template":false,"template_full_name":null,"purl":"pkg:github/labring/aiproxy","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labring%2Faiproxy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labring%2Faiproxy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labring%2Faiproxy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labring%2Faiproxy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/labring","download_url":"https://codeload.github.com/labring/aiproxy/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/labring%2Faiproxy/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33975476,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-06T02:00:07.033Z","response_time":107,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-11T14:39:14.798Z","updated_at":"2026-06-06T09:01:46.691Z","avatar_url":"https://github.com/labring.png","language":"Go","funding_links":[],"categories":["Openai"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n  \u003ch1\u003eAI Proxy\u003c/h1\u003e\n  \u003cp\u003eNext-generation AI gateway with OpenAI-compatible protocol\u003c/p\u003e\n  \n  [![Release](https://img.shields.io/github/release/labring/aiproxy)](https://github.com/labring/aiproxy/releases)\n  [![License](https://img.shields.io/github/license/labring/aiproxy)](https://github.com/labring/aiproxy/blob/main/LICENSE)\n  [![Go Version](https://img.shields.io/github/go-mod/go-version/labring/aiproxy?filename=core%2Fgo.mod)](https://github.com/labring/aiproxy/blob/main/core/go.mod)\n  [![Build Status](https://img.shields.io/github/actions/workflow/status/labring/aiproxy/release.yml?branch=main)](https://github.com/labring/aiproxy/actions)\n  \n  [English](./README.md) | [简体中文](./README.zh.md)\n\u003c/div\u003e\n\n---\n\n## 🚀 Overview\n\nAI Proxy is a powerful, production-ready AI gateway that provides intelligent request routing, comprehensive monitoring, and seamless multi-tenant management. Built with OpenAI-compatible, Anthropic and Gemini protocols, it serves as the perfect middleware for AI applications requiring reliability, scalability, and advanced features.\n\n## ✨ Key Features\n\n### 🔄 **Intelligent Request Management**\n\n- **Smart Retry Logic**: Intelligent retry strategies with automatic error recovery\n- **Priority-based Channel Selection**: Route requests based on channel priority and error rates\n- **Load Balancing**: Efficiently distribute traffic across multiple AI providers\n- **Protocol Conversion**: Seamless protocol conversion between OpenAI Chat Completions, Claude Messages, Gemini, and OpenAI Responses API\n  - Chat/Claude/Gemini → Responses API: Use responses-only models with any protocol\n\n### 📊 **Comprehensive Monitoring \u0026 Analytics**\n\n- **Real-time Alerts**: Proactive notifications for balance warnings, error rates, and anomalies\n- **Detailed Logging**: Complete request/response tracking with audit trails\n- **Advanced Analytics**: Request volume, error statistics, RPM/TPM metrics, and cost analysis\n- **Channel Performance**: Error rate analysis and performance monitoring\n\n### 🏢 **Multi-tenant Architecture**\n\n- **Organization Isolation**: Complete separation between different organizations\n- **Flexible Access Control**: Token-based authentication with subnet restrictions\n- **Resource Quotas**: RPM/TPM limits and usage quotas per group\n- **Custom Pricing**: Per-group model pricing and billing configuration\n\n### 🤖 **MCP (Model Context Protocol) Support**\n\n- **Public MCP Servers**: Ready-to-use MCP integrations\n- **Organization MCP Servers**: Private MCP servers for organizations\n- **Embedded MCP**: Built-in MCP servers with configuration templates\n- **OpenAPI to MCP**: Automatic conversion of OpenAPI specs to MCP tools\n\n### 🔌 **Plugin System**\n\n- **Cache Plugin**: High-performance caching for identical requests with Redis/memory storage\n- **Web Search Plugin**: Real-time web search capabilities with support for Google, Bing, and Arxiv\n- **Think Split Plugin**: Support for reasoning models with content splitting, automatically handling `\u003cthink\u003e` tags\n- **Stream Fake Plugin**: Avoid non-streaming request timeouts through internal streaming transmission\n- **Extensible Architecture**: Easy to add custom plugins for additional functionality\n\n### 🔧 **Advanced Capabilities**\n\n- **Multi-format Support**: Text, image, audio, and document processing\n- **Model Mapping**: Flexible model aliasing and routing\n- **Prompt Caching**: Intelligent caching with billing support\n- **Think Mode**: Support for reasoning models with content splitting\n- **Built-in Tokenizer**: No external tiktoken dependencies\n\n## 📊 Management Panel\n\nAI Proxy provides a management panel for managing AI Proxy's configuration and monitoring.\n\n![Dashboard](./docs/images/dashboard.png)\n![Logs](./docs/images/logs.png)\n\n## 🏗️ Architecture\n\n```mermaid\ngraph TB\n    Client[Client Applications] --\u003e Gateway[AI Proxy Gateway]\n    Gateway --\u003e Auth[Authentication \u0026 Authorization]\n    Gateway --\u003e Router[Intelligent Router]\n    Gateway --\u003e Monitor[Monitoring \u0026 Analytics]\n    Gateway --\u003e Plugins[Plugin System]\n\n    Plugins --\u003e CachePlugin[Cache Plugin]\n    Plugins --\u003e SearchPlugin[Web Search Plugin]\n    Plugins --\u003e ThinkSplitPlugin[Think Split Plugin]\n    Plugins --\u003e StreamFakePlugin[Stream Fake Plugin]\n\n    Router --\u003e Provider1[OpenAI]\n    Router --\u003e Provider2[Anthropic]\n    Router --\u003e Provider3[Azure OpenAI]\n    Router --\u003e ProviderN[Other Providers]\n\n    Gateway --\u003e MCP[MCP Servers]\n    MCP --\u003e PublicMCP[Public MCP]\n    MCP --\u003e GroupMCP[Organization MCP]\n    MCP --\u003e EmbedMCP[Embedded MCP]\n\n    Monitor --\u003e Alerts[Alert System]\n    Monitor --\u003e Analytics[Analytics Dashboard]\n    Monitor --\u003e Logs[Audit Logs]\n```\n\n## 🚀 Quick Start\n\n### Docker (Recommended)\n\n```bash\n# Quick start with default configuration\ndocker run -d \\\n  --name aiproxy \\\n  -p 3000:3000 \\\n  -v $(pwd)/aiproxy:/aiproxy \\\n  -e ADMIN_KEY=your-admin-key \\\n  ghcr.io/labring/aiproxy:latest\n\n# Nightly build\ndocker run -d \\\n  --name aiproxy \\\n  -p 3000:3000 \\\n  -v $(pwd)/aiproxy:/aiproxy \\\n  -e ADMIN_KEY=your-admin-key \\\n  ghcr.io/labring/aiproxy:main\n```\n\n### Docker Compose\n\n```bash\n# Download docker-compose.yaml\ncurl -O https://raw.githubusercontent.com/labring/aiproxy/main/docker-compose.yaml\n\n# Start services\ndocker-compose up -d\n```\n\n## 🔧 Configuration\n\n### Environment Variables\n\n#### **Core Settings**\n\n```bash\nLISTEN=:3000                    # Server listen address\nADMIN_KEY=your-admin-key        # Admin API key\nDISABLE_WEB_ROOT=true           # Redirect only `/` to GitHub, keep other web routes available\n```\n\n#### **Database Configuration**\n\n```bash\nSQL_DSN=postgres://user:pass@host:5432/db    # Primary database\nLOG_SQL_DSN=postgres://user:pass@host:5432/log_db  # Log database (optional)\nREDIS=redis://localhost:6379     # Redis for caching\n```\n\n#### **Feature Toggles**\n\n```bash\nBILLING_ENABLED=true           # Enable billing features\nSAVE_ALL_LOG_DETAIL=true     # Log all request details\n```\n\n### Advanced Configuration\n\n\u003cdetails\u003e\n\u003csummary\u003eClick to expand advanced configuration options\u003c/summary\u003e\n\n#### **Quotas**\n\n```bash\nGROUP_MAX_TOKEN_NUM=100        # Max tokens per group\n```\n\n#### **Logging \u0026 Retention**\n\n```bash\nLOG_STORAGE_HOURS=168          # Log retention (0 = unlimited)\nLOG_DETAIL_STORAGE_HOURS=72    # Detail log retention\nCLEAN_LOG_BATCH_SIZE=5000      # Log cleanup batch size\n```\n\n#### **Security \u0026 Access Control**\n\n```bash\nIP_GROUPS_THRESHOLD=5          # IP sharing alert threshold\nIP_GROUPS_BAN_THRESHOLD=10     # IP sharing ban threshold\n```\n\n\u003c/details\u003e\n\n## 🔌 Plugins\n\nAI Proxy supports a plugin system that extends its functionality. Currently available plugins:\n\n### Cache Plugin\n\nThe Cache Plugin provides high-performance caching for AI API requests:\n\n- **Dual Storage**: Supports both Redis and in-memory caching\n- **Content-based Keys**: Uses SHA256 hash of request body\n- **Configurable TTL**: Custom time-to-live for cached items\n- **Size Limits**: Prevents memory issues with configurable limits\n\n[View Cache Plugin Documentation](./core/relay/plugin/cache/README.md)\n\n### Web Search Plugin\n\nThe Web Search Plugin adds real-time web search capabilities:\n\n- **Multiple Search Engines**: Supports Google, Bing, and Arxiv\n- **Smart Query Rewriting**: AI-powered query optimization\n- **Reference Management**: Automatic citation formatting\n- **Dynamic Control**: User-controllable search depth\n\n[View Web Search Plugin Documentation](./core/relay/plugin/web-search/README.md)\n\n### Think Split Plugin\n\nThe Think Split Plugin supports content splitting for reasoning models:\n\n- **Automatic Recognition**: Automatically detects `\u003cthink\u003e...\u003c/think\u003e` tags in responses\n- **Content Separation**: Extracts thinking content to `reasoning_content` field\n- **Streaming Support**: Supports both streaming and non-streaming responses\n\n[View Think Split Plugin Documentation](./core/relay/plugin/thinksplit/README.md)\n\n### Stream Fake Plugin\n\nThe Stream Fake Plugin solves timeout issues with non-streaming requests:\n\n- **Timeout Avoidance**: Prevents request timeouts through internal streaming transmission\n- **Transparent Conversion**: Automatically converts non-streaming requests to streaming format, transparent to clients\n- **Response Reconstruction**: Collects all streaming data chunks and reconstructs them into complete non-streaming responses\n- **Connection Keep-Alive**: Maintains active connections through streaming transmission to avoid network timeouts\n\n[View Stream Fake Plugin Documentation](./core/relay/plugin/streamfake/README.md)\n\n## 📚 API Documentation\n\n### Interactive API Explorer\n\nVisit `http://localhost:3000/swagger/index.html` for the complete API documentation with interactive examples.\n\n### Quick API Examples\n\n#### **List Available Models**\n\n```bash\ncurl -H \"Authorization: Bearer your-token\" \\\n  http://localhost:3000/v1/models\n```\n\n#### **Chat Completion**\n\n```bash\ncurl -X POST http://localhost:3000/v1/chat/completions \\\n  -H \"Authorization: Bearer your-token\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"gpt-4\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n  }'\n```\n\n#### **Claude API**\n\n```bash\n# Use Claude models through OpenAI API format\ncurl -X POST http://localhost:3000/v1/messages \\\n  -H \"X-Api-Key: Bearer your-token\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"gpt-5\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello Claude!\"}]\n  }'\n```\n\n## 🔌 Integrations\n\n### Sealos Platform\n\nDeploy instantly on Sealos with built-in model capabilities:\n[Deploy to Sealos](https://hzh.sealos.run/?openapp=system-aiproxy)\n\n### FastGPT Integration\n\nSeamlessly integrate with FastGPT for enhanced AI workflows:\n[FastGPT Documentation](https://doc.fastgpt.cn/docs/introduction/development/modelConfig/ai-proxy)\n\n### Claude Code Integration\n\nUse AI Proxy with Claude Code by configuring these environment variables:\n\n```bash\nexport ANTHROPIC_BASE_URL=http://127.0.0.1:3000\nexport ANTHROPIC_AUTH_TOKEN=sk-xxx\nexport ANTHROPIC_MODEL=gpt-5\nexport ANTHROPIC_SMALL_FAST_MODEL=gpt-5-nano\n```\n\n### Gemini CLI Integration\n\nUse AI Proxy with Gemini CLI by configuring these environment variables:\n\n```bash\nexport GOOGLE_GEMINI_BASE_URL=http://127.0.0.1:3000\nexport GEMINI_API_KEY=sk-xxx\n```\n\nAlternatively, you can use the `/auth` command in the Gemini CLI to output the `GEMINI_API_KEY`.\n\n### Codex Integration\n\nUse AI Proxy with Codex by configuring `~/.codex/config.toml`:\n\n```toml\n# Recall that in TOML, root keys must be listed before tables.\nmodel = \"gpt-4o\"\nmodel_provider = \"aiproxy\"\n\n[model_providers.aiproxy]\n# Name of the provider that will be displayed in the Codex UI.\nname = \"AIProxy\"\n# The path `/chat/completions` will be amended to this URL to make the POST\n# request for the chat completions.\nbase_url = \"http://127.0.0.1:3000/v1\"\n# If `env_key` is set, identifies an environment variable that must be set when\n# using Codex with this provider. The value of the environment variable must be\n# non-empty and will be used in the `Bearer TOKEN` HTTP header for the POST request.\nenv_key = \"AIPROXY_API_KEY\"\n# Valid values for wire_api are \"chat\" and \"responses\". Defaults to \"chat\" if omitted.\nwire_api = \"chat\"\n```\n\n**Protocol Conversion Support**:\n\n- **Responses-only models**: AI Proxy automatically converts Chat/Claude/Gemini requests to Responses API format for models that only support the Responses API\n- **Multi-protocol access**: Use any protocol (Chat Completions, Claude Messages, or Gemini) to access responses-only models\n- **Transparent conversion**: No client-side changes needed - AI Proxy handles protocol translation automatically\n\n**Reasoning / Thinking Compatibility Docs**:\n\n- [Thinking / Reasoning Compatibility](./docs/REASONING_COMPATIBILITY.md)\n\n### MCP (Model Context Protocol)\n\nAI Proxy provides comprehensive MCP support for extending AI capabilities:\n\n- **Public MCP Servers**: Community-maintained integrations\n- **Organization MCP Servers**: Private organizational tools\n- **Embedded MCP**: Easy-to-configure built-in functionality\n- **OpenAPI to MCP**: Automatic tool generation from API specifications\n\n## 🛠️ Development\n\n### Prerequisites\n\n- Go 1.24+\n- Node.js 22+ (for frontend development)\n- PostgreSQL (optional, SQLite by default)\n- Redis (optional, for caching)\n\n### Building from Source\n\n```bash\n# Clone repository\ngit clone https://github.com/labring/aiproxy.git\ncd aiproxy\n\n# Build frontend (optional)\ncd web \u0026\u0026 npm install -g pnpm \u0026\u0026 pnpm install \u0026\u0026 pnpm run build \u0026\u0026 cp -r dist ../core/public/dist/ \u0026\u0026 cd ..\n\n# Build backend\ncd core \u0026\u0026 go build -o aiproxy .\n\n# Run\n./aiproxy\n```\n\n## 📄 License\n\nThis project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.\n\n## 🙏 Acknowledgments\n\n- OpenAI for the API specification\n- The open-source community for various integrations\n- All contributors and users of AI Proxy\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flabring%2Faiproxy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flabring%2Faiproxy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flabring%2Faiproxy/lists"}