{"id":31890565,"url":"https://github.com/luispater/mini-router","last_synced_at":"2025-10-13T07:50:03.834Z","repository":{"id":302236809,"uuid":"1011713537","full_name":"luispater/mini-router","owner":"luispater","description":null,"archived":false,"fork":false,"pushed_at":"2025-07-01T09:15:42.000Z","size":34,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-01T09:37:11.855Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/luispater.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-01T08:28:00.000Z","updated_at":"2025-07-01T09:15:24.000Z","dependencies_parsed_at":"2025-07-01T09:49:06.422Z","dependency_job_id":null,"html_url":"https://github.com/luispater/mini-router","commit_stats":null,"previous_names":["luispater/mini-router"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/luispater/mini-router","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispater%2Fmini-router","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispater%2Fmini-router/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispater%2Fmini-router/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispater%2Fmini-router/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/luispater","download_url":"https://codeload.github.com/luispater/mini-router/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/luispater%2Fmini-router/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279014159,"owners_count":26085464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-13T02:00:06.723Z","response_time":61,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-13T07:49:45.453Z","updated_at":"2025-10-13T07:50:03.819Z","avatar_url":"https://github.com/luispater.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Mini Router - AI Model Router\n\n## Overview\n\nMini Router is a lightweight, high-performance AI model router built with Go. It is designed to provide a unified API endpoint for various large language models (LLMs), offering OpenAI API compatibility. This allows clients to interact with different AI providers through a single, consistent interface. The router supports features like model load balancing, rate limiting, and dynamic configuration via a YAML file.\n\n## Features\n\n*   **OpenAI API Compatibility**: Exposes a standard OpenAI-compatible endpoint (`/v1/chat/completions`), allowing seamless integration with existing tools and libraries that support the OpenAI API.\n*   **Multi-Model Support**: Configure and manage multiple AI models from different providers within a single instance.\n*   **Load Balancing**: Implements round-robin load balancing for models that have multiple provider API keys or configurations, enhancing reliability and distributing the load.\n*   **Dynamic Configuration**: All settings, including server configuration, models, and API keys, are managed through a single `config.yaml` file, which is loaded at startup.\n*   **Streaming and Non-Streaming Support**: Handles both streaming (`text/event-stream`) and standard JSON responses for chat completions.\n*   **Health Check**: A `/health` endpoint to monitor the status of the router.\n*   **Authentication**: Secures the chat completion endpoint with API key authentication.\n*   **Extensible Provider System**: Designed to be easily extensible with new AI providers.\n\n## Getting Started\n\n### Prerequisites\n\n*   [Go](https://golang.org/dl/) (version 1.24 or later is recommended)\n\n### Installation \u0026 Running\n\n1.  **Clone the repository:**\n    ```bash\n    git clone https://github.com/luispater/mini-router.git\n    cd mini-router\n    ```\n\n2.  **Configure the application:**\n    Create and edit the `config.yaml` file based on the instructions in the **Configuration** section below.\n\n3.  **Build the application:**\n    ```bash\n    go build -o mini-router\n    ```\n\n4.  **Run the application:**\n    ```bash\n    ./mini-router\n    ```\n    The server will start on the port specified in your `config.yaml`.\n\n## Configuration (`config.yaml`)\n\nThe application is configured using the `config.yaml` file. Here is a detailed breakdown of the configuration options.\n\n### `server`\n\nThis section configures the HTTP server settings.\n\n| Parameter         | Type     | Description                                            | Example |\n| ----------------- | -------- | ------------------------------------------------------ | ------- |\n| `port`            | `string` | The port the server will listen on.                    | `\"8316\"`  |\n| `shutdown_timeout`| `string` | The graceful shutdown timeout (e.g., `10s`, `1m`).     | `10s`   |\n\n**Example:**\n```yaml\nserver:\n  port: \"8316\"\n  shutdown_timeout: 10s\n```\n\n### `models`\n\nThis is a list of AI models that the router will manage.\n\n| Parameter                 | Type        | Description                                                                        |\n| ------------------------- | ----------- |------------------------------------------------------------------------------------|\n| `id`                      | `integer`   | A unique identifier for the model configuration entry.                             |\n| `name`                    | `string`    | The model name used in the API request (e.g., `gemini-2.5-pro`).                   |\n| `provider_model_name`     | `string`    | The actual model name that the provider expects (e.g., `gemini-2.5-pro`).          |\n| `display_name`            | `string`    | A user-friendly name for display purposes.                                         |\n| `description`             | `string`    | A brief description of the model.                                                  |\n| `supports_chat`           | `boolean`   | Whether the model supports chat completions.                                       |\n| `supports_completion`     | `boolean`   | Whether the model supports standard text completions.                              |\n| `supports_input_image`    | `boolean`   | Whether the model supports image inputs.                                           |\n| `support_google_thinking` | `boolean`   | Enables special handling for Google's \"thinking\" or reasoning features.            |\n| `rpm`                     | `integer`   | Requests Per Minute limit for this model.                                          |\n| `tpm`                     | `integer`   | Tokens Per Minute limit for this model.                                            |\n| `rpd`                     | `integer`   | Requests Per Day limit for this model.                                             |\n| `input_price_per_token`   | `float`     | The cost per input token.                                                          |\n| `output_price_per_token`  | `float`     | The cost per output token.                                                         |\n| `max_tokens`              | `integer`   | The maximum number of tokens the model can generate in a single response.          |\n| `context_length`          | `integer`   | The maximum context length (in tokens) the model supports.                         |\n| `supported_parameters`    | `[]string`  | A list of API parameters supported by this model (e.g., `tools`, `temperature`).   |\n| `provider_api_key`        | `[]string`  | A list of API keys for the backend provider. The router will use these in a round-robin fashion. |\n| `is_openai_compatibility` | `boolean`   | Set to `true` if the provider's API is OpenAI-compatible.                          |\n| `base_url`                | `string`    | The base URL of the provider's API endpoint.                                       |\n| `enabled`                 | `boolean`   | If `true`, this model configuration is active and can be used.                     |\n| `visible`                 | `boolean`   | If `true`, this model will be listed in the `/v1/models` endpoint.                 |\n\n**Example:**\n```yaml\nmodels:\n  - id: 4\n    name: \"gemini-2.5-pro\"\n    provider_model_name: \"gemini-2.5-pro\"\n    display_name: \"Gemini 2.5 Pro\"\n    description: \"Google's Gemini 2.5 Pro\"\n    supports_chat: true\n    supports_completion: true\n    supports_input_image: true\n    support_google_thinking: true\n    rpm: 5\n    tpm: 250000\n    rpd: 100\n    input_price_per_token: 0.0\n    output_price_per_token: 0.0\n    max_tokens: 65536\n    context_length: 1048576\n    supported_parameters: [\"tools\", \"tool_choice\", \"max_tokens\", \"temperature\", \"top_p\", \"stop\", \"frequency_penalty\", \"presence_penalty\", \"seed\", \"response_format\", \"structured_outputs\"]\n    provider_api_key:\n      - \"AIzaSy...S6cBqxM4\"\n      - \"AIzaSy...J7amzb2w\"\n    is_openai_compatibility: true\n    base_url: \"https://generativelanguage.googleapis.com/v1beta/openai\"\n    enabled: true\n    visible: true\n```\n\n### `api_keys`\n\nThis section defines the API keys that clients will use to authenticate with the Mini Router itself.\n\n| Parameter | Type      | Description                                                              |\n| --------- | --------- | ------------------------------------------------------------------------ |\n| `id`      | `integer` | A unique identifier for the API key.                                     |\n| `key`     | `string`  | The API key string. Should be prefixed (e.g., `sk-or-v1-...`).           |\n| `name`    | `string`  | A descriptive name for the key.                                          |\n| `is_active`| `boolean` | If `true`, the key is active and can be used for authentication.         |\n| `user_id` | `integer` | An associated user ID.                                                   |\n| `rps`, `rpm`, `rph`, `rpd` | `integer` | Rate limits for this key (requests per second/minute/hour/day). `0` means no limit. |\n| `tps`, `tpm`, `tph`, `tpd` | `integer` | Token limits for this key (tokens per second/minute/hour/day). `0` means no limit. |\n\n**Example:**\n```yaml\napi_keys:\n  - id: 1\n    key: \"sk-or-v1-...\"\n    name: \"Default\"\n    is_active: true\n    user_id: 1\n    rpm: 0 # No limit\n```\n\n## API Endpoints\n\n### Health Check\n\n*   **Endpoint**: `GET /health`\n*   **Description**: Returns the operational status of the server.\n*   **Authentication**: None\n*   **Success Response (200 OK)**:\n    ```json\n    {\n      \"status\": \"ok\",\n      \"time\": \"2025-07-01T16:05:29+08:00\"\n    }\n    ```\n\n### List Models\n\n*   **Endpoint**: `GET /v1/models`\n*   **Description**: Retrieves a list of all `visible` and `enabled` models from the configuration.\n*   **Authentication**: None\n*   **Success Response (200 OK)**:\n    ```json\n    {\n      \"object\": \"list\",\n      \"data\": [\n        {\n          \"id\": \"gemini-2.5-pro\",\n          \"object\": \"model\",\n          \"name\": \"Gemini 2.5 Pro\",\n          \"description\": \"Google's Gemini 2.5 Pro\",\n          \"supported_parameters\": [\n            \"tools\",\n            \"tool_choice\",\n            \"max_tokens\",\n            // ... other parameters\n          ],\n          \"context_length\": 1048576,\n          \"max_completion_tokens\": 65536,\n          // ... other model details\n        }\n      ]\n    }\n    ```\n\n### Chat Completions\n\n*   **Endpoint**: `POST /v1/chat/completions`\n*   **Description**: Creates a model response for the given chat conversation. This endpoint is compatible with the OpenAI Chat Completions API.\n*   **Authentication**: Required. Provide an API key from the `api_keys` configuration in the `Authorization` header as a Bearer token.\n    ```\n    Authorization: Bearer sk-or-v1-...\n    ```\n*   **Request Body**: Standard OpenAI chat completion request body.\n    ```json\n    {\n      \"model\": \"gemini-2.5-pro:free\",\n      \"messages\": [\n        {\n          \"role\": \"user\",\n          \"content\": \"Hello!\"\n        }\n      ],\n      \"stream\": false\n    }\n    ```\n*   **Success Response**: Standard OpenAI chat completion response (or a `text/event-stream` if `stream: true`).\n\n## Dependencies\n\nThis project relies on several open-source libraries, including:\n\n*   [Gin](https://github.com/gin-gonic/gin): A high-performance HTTP web framework.\n*   [gjson](https://github.com/tidwall/gjson): A fast JSON parser.\n*   [sjson](https://github.com/tidwall/sjson): A library to set JSON values.\n*   [gojsonschema](https://github.com/xeipuuv/gojsonschema): A JSON Schema validator.\n*   [yaml.v3](https://gopkg.in/yaml.v3): A YAML support library for Go.\n\n## License\n\nThis project is licensed under the MIT License. Refer to the [LICENSE](LICENSE) file for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluispater%2Fmini-router","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fluispater%2Fmini-router","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fluispater%2Fmini-router/lists"}