{"id":26334347,"url":"https://github.com/em-geeklab/nexusgate","last_synced_at":"2026-02-06T18:16:51.524Z","repository":{"id":281334973,"uuid":"944943983","full_name":"EM-GeekLab/NexusGate","owner":"EM-GeekLab","description":"Smart LLM/Agent Management in One Line of Code","archived":false,"fork":false,"pushed_at":"2025-03-15T10:18:50.000Z","size":5452,"stargazers_count":21,"open_issues_count":9,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-15T11:24:11.995Z","etag":null,"topics":["bun","drizzle-orm","elysiajs","fumadocs","react","shadcn-ui","tailwindcss"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/EM-GeekLab.png","metadata":{"files":{"readme":"README.en.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2025-03-08T09:40:42.000Z","updated_at":"2025-03-15T10:18:53.000Z","dependencies_parsed_at":"2025-03-15T11:35:14.888Z","dependency_job_id":null,"html_url":"https://github.com/EM-GeekLab/NexusGate","commit_stats":null,"previous_names":["geekchange/nexusgate","geektechx/nexusgate","em-geeklab/nexusgate"],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EM-GeekLab%2FNexusGate","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EM-GeekLab%2FNexusGate/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EM-GeekLab%2FNexusGate/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/EM-GeekLab%2FNexusGate/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/EM-GeekLab","download_url":"https://codeload.github.com/EM-GeekLab/NexusGate/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243806362,"owners_count":20350824,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bun","drizzle-orm","elysiajs","fumadocs","react","shadcn-ui","tailwindcss"],"created_at":"2025-03-16T00:18:24.997Z","updated_at":"2026-01-31T13:00:55.895Z","avatar_url":"https://github.com/EM-GeekLab.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n![Banner](./assets/img/banner-en.svg)\n\n# NexusGate\n\nMonitor and manage your Agent applications with just one line of code\n\n[![GitHub license](https://img.shields.io/github/license/em-geeklab/nexusgate)](https://github.com/em-geeklab/nexusgate/blob/main/LICENSE)\n[![GitHub stars](https://img.shields.io/github/stars/em-geeklab/nexusgate)](https://github.com/em-geeklab/nexusgate/stargazers)\n[![GitHub issues](https://img.shields.io/github/issues/em-geeklab/nexusgate)](https://github.com/em-geeklab/nexusgate/issues)\n[![Free Use](https://img.shields.io/badge/free-pricing?logo=free\u0026color=%20%23155EEF\u0026label=pricing\u0026labelColor=%20%23528bff)](https://img.shields.io/badge/free-pricing?logo=free\u0026color=%20%23155EEF\u0026label=pricing\u0026labelColor=%20%23528bff)\n\n\u003c/div\u003e\n\n\u003cdiv align=\"right\"\u003e\n  \u003ca href=\"README.md\"\u003e中文\u003c/a\u003e\n\u003c/div\u003e\n\n---\n\n## 🚀 Introduction\n\nNexusGate is an open-source, lightweight **AI Agent and LLM application** monitoring and management tool that enables you to build a \"management panel\" and \"task manager\" for your enterprise's \"intelligent infrastructure\" with one click.\n\nWith the rapid development of Large Language Model (LLM) technology, intelligent Agents and AI applications built on LLMs are being widely deployed in enterprises. NexusGate provides a comprehensive platform for managing, tracking, debugging, analyzing, and optimizing these **LLM-based intelligent Agent applications**. Whether it's simple ChatBots, complex multi-step Agent workflows, or knowledge base applications integrated with RAG (Retrieval Augmented Generation), NexusGate offers a unified monitoring and management experience.\n\nDesigned for AI developers and enterprise administrators, NexusGate features an intuitive management interface and developer-friendly REST APIs, providing functionalities such as **LLM call logging**, **Agent runtime performance tracking**, **Token cost analysis**, **LLM request traffic control**, **multi-platform model unified management**, **enterprise-grade AI application governance**, and **inference service monitoring**. On NexusGate, developers can optimize Agent configurations based on detailed LLM interaction history, while enterprise administrators can use a centralized monitoring dashboard to manage all LLM-based intelligent applications within the enterprise from a single point, improving overall management and development efficiency.\n\n## ✨ Key Features\n\n- **Comprehensive LLM Management**: Focus on your AI applications through a unified management system to improve quality, reduce costs, decrease latency, and ensure security. Compatible with all mainstream large language model services and inference frameworks.\n- **Multi-Provider Support**: Supports multiple LLM providers including OpenAI, Anthropic, Azure OpenAI, Ollama, and the new OpenAI Responses API format.\n- **Function Calling Support**: Full support for tool definitions, function calls (tool_calls), and tool results logging and display, making it easy to debug Agent applications.\n- **Intelligent Load Balancing**: Weighted random load balancing to distribute traffic across multiple providers.\n- **Automatic Failover**: Automatically switches to backup providers when one fails, with exponential backoff retry strategy.\n- **Production Monitoring**: Record all production interactions for monitoring, analysis, debugging, and optimization.\n- **Enterprise-Grade Management**: Manage applications or business systems with one click, providing metering and auditing of LLM content.\n- **Kubernetes Native**: Provides K8s Operator for automatic API key management in Kubernetes environments.\n\n## 🐳 Quick Start\n\n### Prerequisites\n\nNexusGate uses Docker for deployment. Please ensure your system has Docker and Docker Compose installed.\n\n#### Install Docker\n\nIf Docker is not installed on your system, please visit the [Docker official website](https://www.docker.com/) to download and install the version suitable for your operating system:\n\n- **Windows/macOS users**: Download and install [Docker Desktop](https://www.docker.com/products/docker-desktop/)\n- **Linux users**: Refer to [Docker Engine installation guide](https://docs.docker.com/engine/install/) or use the get.docker.com script for installation.\n\n```bash\ncurl -fsSL https://get.docker.com | sudo sh\n```\n\n#### Verify Installation\n\nAfter installation, run the following commands in terminal or CMD to verify:\n\n```bash\ndocker --version\ndocker compose version\n```\n\nIf the commands output version information normally, the installation is successful.\n\n### One-Click Deployment\n\n**Method 1: One-Click Script Deployment (Recommended)**\n\nIf you are using Linux or macOS, you can run the following command to quickly deploy NexusGate:\n\n```bash\ncurl -fsSL https://github.com/EM-GeekLab/NexusGate/raw/refs/heads/main/scripts/quick-start.sh -o quick-start.sh\nbash quick-start.sh\n```\n\nIf you are using Windows, please download [quick-start.bat](https://github.com/EM-GeekLab/NexusGate/raw/refs/heads/main/scripts/quick-start.bat) and run it in CMD:\n\n```\n.\\quick-start.bat\n```\n\n💡 **Interactive Configuration**: The one-click script supports interactive configuration, you can:\n\n- **Custom Password**: Enter your own database password and admin key\n- **Auto-Generate**: Press Enter directly to use secure random passwords (recommended)\n- **Port Configuration**: Customize web service port (default 8080)\n- **Input Validation**: The script validates password strength and port validity\n\n🔒 **Security Features**:\n\n- Password input is hidden in terminal\n- Auto-generated passwords are 16-character strong passwords\n- Supports password length validation (minimum 8 characters)\n- Port range validation (1024-65535)\n\n❗ **Note**: The script will automatically create a `.env` file, please do not delete this file.\n\n**Method 2: Manual Configuration**\n\n1. **Download Configuration File**\n\n   ```bash\n   wget https://github.com/EM-GeekLab/NexusGate/raw/refs/heads/main/docker-compose.yaml\n   ```\n\n2. **Configure Environment Variables (Important)**\n\n   Create environment variable configuration file:\n\n   ```bash\n   cat \u003e .env \u003c\u003c 'EOF'\n   # ======================\n   # Database Configuration\n   # ======================\n   # PostgreSQL database password (must change)\n   POSTGRES_PASSWORD=your_secure_database_password_here\n\n   # ======================\n   # Admin Configuration\n   # ======================\n   # Admin key for accessing admin interface (must change)\n   ADMIN_SUPER_SECRET=your_admin_secret_key_here\n\n   # ======================\n   # Service Configuration\n   # ======================\n   # Web service external port (optional, default 3000 if not set, recommended 8080)\n   WEB_PORT=8080\n   EOF\n   ```\n\n   **Important Parameter Description:**\n\n   | Parameter            | Required | Description                                  | Example Value           |\n   | -------------------- | -------- | -------------------------------------------- | ----------------------- |\n   | `POSTGRES_PASSWORD`  | ✅       | Database password, recommend strong password | `MySecurePass123!`      |\n   | `ADMIN_SUPER_SECRET` | ✅       | Admin login key                              | `admin_key_2024_secure` |\n   | `WEB_PORT`           | ❌       | Web service port (default 3000 if not set)   | `8080`                  |\n\n   \u003e ⚠️ **Security Warning**:\n   \u003e\n   \u003e - Please make sure to change `POSTGRES_PASSWORD` and `ADMIN_SUPER_SECRET`!\n   \u003e - Passwords should include uppercase and lowercase letters, numbers, and special characters\n   \u003e - Length should be at least 12 characters\n   \u003e - Use more complex passwords in production environment\n\n3. **Start Services**\n\n   ```bash\n   docker compose up -d\n   ```\n\n4. **Access System**\n\n   After startup, access in browser: `http://localhost:8080` (if you changed the port, use the corresponding port).\n\n   Use the `ADMIN_SUPER_SECRET` set in your `.env` file as the admin key to login. After login, please refresh the page to ensure configuration takes effect.\n\n### Troubleshooting\n\n**Common Issue Solutions:**\n\n1. **Port Conflict**\n\n   ```bash\n   # Modify port in .env file\n   WEB_PORT=9090  # Change to another port\n   ```\n\n2. **Permission Issues (Linux/macOS)**\n\n   ```bash\n   # Ensure current user is in docker group\n   sudo usermod -aG docker $USER\n   newgrp docker\n   ```\n\n3. **Service Status Check**\n\n   ```bash\n   # View all service status\n   docker compose ps\n\n   # View service logs\n   docker compose logs -f\n\n   # Restart services\n   docker compose restart\n   ```\n\n4. **Complete Reset**\n\n   ```bash\n   # Stop and remove all containers and data\n   docker compose down -v\n\n   # Restart\n   docker compose up -d\n   ```\n\n**Get Help:**\n\n- If you encounter problems, please check [GitHub Issues](https://github.com/EM-GeekLab/NexusGate/issues)\n- Or submit a new Issue in the project repository\n\n## 🔍 System Features\n\n### 1. Model Layer Management\n\nConnect and manage multiple LLM services,such as:\n\n- Public Cloud Services: OpenAI, DeepSeek, Alibaba Qwen\n- Enterprise Private Models: Large Model All-in-One Machine\n\nNexusGate supports over 20 tested model services and deployment frameworks, while supporting multiple integrable client applications, giving you flexibility and choice.\n![Create Model Layer Configuration](./assets/img/upstream-config.webp)\n_Figure 1.1: Creating Model Layer Configuration_\n\n\u003e _You can also view the example below for more details._\n\n\u003cdetails\u003e\n \u003csummary\u003e\u003cmark\u003eClick to view example video:Configure a Model\u003c/mark\u003e\u003c/summary\u003e\n \u003cvideo controls src=\"https://github.com/user-attachments/assets/7c3aec03-c288-494d-a08c-aec5c92c509a\" title=\"Configure a Model\"\u003e\u003c/video\u003e\n\u003c/details\u003e\n\n### 2. Comprehensive Logging\n\nMonitor detailed information of all interactions, including:\n\n- Request timestamps and status.\n- Input prompts and generated content.\n- Model information and token usage.\n- Latency metrics and user feedback.\n\nThe system provides an admin view of all API key chat logs and history records for specific API keys, with a detailed sidebar view of request details and conversation context.\n\n![History with Conversation Details Sidebar](./assets/img/history-log-details.webp)\n_Figure 2.1: Conversation Details Sidebar_\n\n![History Display](./assets/img/history-table.webp)\n_Figure 2.2: Historical Display_\n\n### 3. Application Management\n\nControl and configure platform-integrated applications:\n\n- API key creation and management.\n- user-friendly naming conventions.\n- expiration settings and visibility controls.\n\n![Create Application with API Key Settings](./assets/img/create-application.webp)\n_Figure 3.1: Application with API Key Settings_\n\n\u003e _You can also view the example below for more details._\n\n\u003cdetails\u003e\n \u003csummary\u003e\u003cmark\u003eClick to view example video:Create and Manage Application\u003c/mark\u003e\u003c/summary\u003e\n\n\u003cvideo controls src=\"https://github.com/user-attachments/assets/a8a2f0a9-f4c0-43b9-a604-29167c439386\" title=\"Create and Manage API Keys\"\u003e\u003c/video\u003e\n\n \u003c/details\u003e\n\n## 👨‍💻 For Developers\n\n### 1.One-Line Code Integration\n\nIntegrate NexusGate into your existing LLM applications with just one line of code modification:\n\n#### Python (Using OpenAI library)\n\n```python\n# Before:\nfrom openai import OpenAI\nclient = OpenAI(api_key=\"your-openai-api-key\")\n\n# After:\nfrom openai import OpenAI\nclient = OpenAI(api_key=\"your-nexusgate-api-key\", base_url=\"https://your-nexusgate-server/v1\")\n```\n\n#### JavaScript/TypeScript\n\n```javascript\n// Before:\nimport OpenAI from \"openai\";\nconst openai = new OpenAI({ apiKey: \"your-openai-api-key\" });\n\n// After:\nimport OpenAI from \"openai\";\nconst openai = new OpenAI({\n  apiKey: \"your-nexusgate-api-key\",\n  baseURL: \"https://your-nexusgate-server/v1\",\n});\n```\n\n### 2.API Documentation\n\nNexusGate provides comprehensive OpenAPI documentation for easy integration with your existing systems and workflows. You can access the OpenAPI specification at:\n\n```\nhttps://your-nexusgate-server/swagger\n```\n\nThe documentation includes all available endpoints, request/response formats, and authentication requirements, enabling developers to quickly understand and utilize all of NexusGate's capabilities.\n\n## 👨‍💼 For Administrators\n\n### 1.Centralized LLM Management\n\nNexusGate provides a unified dashboard for managing all LLM applications in your organization:\n\n- **Cost Control**: Track token usage across all applications and providers\n- **Security Oversight**: Monitor all prompts and completions to ensure compliance and data protection\n- **Performance Optimization**: Identify bottlenecks and optimize response times\n- **Usage Analytics**: Understand how different teams and applications are utilizing LLM resources\n\n### 2.Application Management\n\nManage platform-integrated applications,offering flexible expiration settings for enhanced security, rate limits and usage limits for cost control, and granular permissions for different models and features.\n\n## 🗺️ Roadmap\n\nWe're constantly adding new features and capabilities to NexusGate. Here's what we're working on next:\n\n- [x] 🌐 Internationalization: Complete i18n support with official Chinese language support.\n- [ ] 📊 Enhanced Analytics: Expand our monitoring metrics including success rates, request volumes, token usage statistics, request completion rates, Agent usage rankings, model usage rankings, error analysis, full-chain latency, inference latency, and throughput measurements.\n- [x] 🔄 Prometheus Integration: Create comprehensive overview dashboards by integrating with external Prometheus instances to monitor server hardware, inference frameworks, and other information sources.\n- [x] 🚦 Traffic Control: Implement fine-grained traffic management for each API key, including quotas and priorities for specific models, enabling administrators to precisely control resource allocation.\n- [x] ⚖️ Intelligent Load Balancing: Weighted random load balancing to distribute traffic across multiple providers.\n- [x] 🔁 Automatic Failover: Automatically switches to backup providers when one fails, with exponential backoff retry strategy.\n- [x] 🛠️ Function Calling Support: Full support for OpenAI-format tool definitions, function calls (tool_calls), and tool results logging and display.\n- [x] ☸️ Kubernetes Operator: Provides K8s Operator for automatic API key management in Kubernetes environments.\n- [ ] 💡 Manual Reporting SDK: Develop SDKs for more granular tracking that can be embedded directly in developer code, enabling more detailed monitoring such as end-user analytics.\n- [ ] 👍 Feedback System: Build robust feedback mechanisms to collect and analyze user responses to AI-generated content.\n- [ ] 💬 Prompt Management: Create tools for prompt creation, optimization, and batch testing, helping developers craft more effective interactions with LLMs.\n  - [ ] 🧠 Automated Evaluation: Leverage LLMs to automatically evaluate outputs and provide quality metrics without human intervention.\n  - [ ] 📚 Dataset Creation and Fine-tuning: Implement dataset management and model fine-tuning pipelines, one-click import to [LLaMa Factory](https://github.com/hiyouga/LLaMA-Factory) for fine-tuning, and monitoring using [SwanLab](https://github.com/SwanHubX/SwanLab).\n- [ ] 🌐 Gateway-Layer Tool Integration: Add capabilities to models without built-in tools (like web search) by implementing functionality at the gateway layer and exposing it through standard API interfaces.\n\n## 📝 License\n\n[Apache License 2.0](LICENSE)\n\n## 🤝 Contributing\n\nWe welcome developers of all skill levels to contribute! Whether it's fixing bugs, adding features, or improving documentation, your contributions are valuable.\n\nPlease check out [CONTRIBUTING.md](CONTRIBUTING.md) to learn how to get started.\n\n**Contributors**\n\n\u003cimg src=\"https://contrib.rocks/image?repo=EM-GeekLab/NexusGate\" /\u003e\n\n## 📚 Documentation\n\nFor more detailed information, visit our [official documentation](https://geektechstudio.feishu.cn/wiki/RNWDwS8NyiZZdokTV4IcmXm5nBg).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fem-geeklab%2Fnexusgate","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fem-geeklab%2Fnexusgate","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fem-geeklab%2Fnexusgate/lists"}