{"id":30891349,"url":"https://github.com/isaced/ai-router","last_synced_at":"2026-01-20T17:37:16.021Z","repository":{"id":308820321,"uuid":"1033759380","full_name":"isaced/ai-router","owner":"isaced","description":"A lightweight, framework-agnostic router for AI/LLM API requests.","archived":false,"fork":false,"pushed_at":"2025-08-16T13:10:55.000Z","size":131,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-08-29T07:54:35.663Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/isaced.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-08-07T09:53:58.000Z","updated_at":"2025-08-16T13:10:58.000Z","dependencies_parsed_at":"2025-08-08T06:16:44.350Z","dependency_job_id":null,"html_url":"https://github.com/isaced/ai-router","commit_stats":null,"previous_names":["isaced/ai-router"],"tags_count":11,"template":false,"template_full_name":null,"purl":"pkg:github/isaced/ai-router","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isaced%2Fai-router","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isaced%2Fai-router/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isaced%2Fai-router/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isaced%2Fai-router/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/isaced","download_url":"https://codeload.github.com/isaced/ai-router/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/isaced%2Fai-router/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274229377,"owners_count":25245189,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-08T02:00:09.813Z","response_time":121,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-08T18:43:19.842Z","updated_at":"2026-01-20T17:37:16.009Z","avatar_url":"https://github.com/isaced.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ai-router 🤖🔄\n\n[![NPM Version](https://img.shields.io/npm/v/%40isaced%2Fai-router)](https://www.npmjs.com/package/@isaced/ai-router) [![JSR](https://jsr.io/badges/@isaced/ai-router)](https://jsr.io/@isaced/ai-router)\n\n[English](README.md) | [简体中文](README.zh-CN.md)\n\n**A lightweight, framework-agnostic router for AI/LLM API requests.**\n\nDistribute traffic across multiple providers (OpenAI, Anthropic, Gemini, etc.), accounts, and models with built-in **load balancing**, **failover**, and **easy extensibility**.\n\nPerfect for developers building resilient, scalable AI applications without vendor lock-in.\n\n\u003e 🚀 One interface. Multiple backends. Zero downtime.\n\n## ✨ Features\n\n- ✅ **Multi-provider support**: OpenAI, Anthropic, Google Gemini, Azure, and more (or your own)\n- 🔁 **Load balancing**: Random, or least-loaded distribution across keys/accounts\n- 🛟 **Failover \u0026 retry**: Automatically switch to backup providers on error or timeout\n- 🧩 **Pluggable middleware**: Add auth, logging, rate limiting, caching, etc.\n- ⚡ **Lightweight \u0026 dependency-free**: Works in Node.js, serverless, and edge runtimes\n- 📦 **Framework-agnostic**: Use with Hono, Express, Fastify or standalone\n- 💻 **TypeScript ready**: Full type definitions included\n- 🔄 **Zero dependencies**: 0 dependencies\n\n## 📦 Installation\n\n```bash\nnpm install @isaced/ai-router\n```\n\n## 🚀 Quick Start\n\n```ts\nimport { AIRouter } from '@isaced/ai-router';\n\n// Define your providers and API keys\nconst router = new AIRouter({\n  providers: [\n    {\n      name: 'openai-primary',\n      type: 'openai',\n      endpoint: 'https://api.openai.com/v1',\n      accounts: [\n        {\n          apiKey: 'sk-xxx',\n          models: [\n            // Simple model configuration (no rate limits)\n            'gpt-4-turbo',\n            // Advanced model configuration with per-model rate limits\n            {\n              name: 'gpt-4',\n              rateLimit: {\n                rpm: 100,  // 100 requests per minute\n                tpm: 80000, // 80k tokens per minute\n                rpd: 2000   // 2000 requests per day\n              }\n            },\n            {\n              name: 'gpt-3.5-turbo',\n              rateLimit: {\n                rpm: 500,   // Higher rate limit for cheaper model\n                tpm: 200000\n              }\n            }\n          ]\n        },\n        {\n          apiKey: 'sk-yyy',\n          models: ['gpt-3.5-turbo'] // No rate limits for this account\n        }\n      ],\n    },\n    {\n      name: 'custom-provider',\n      type: 'custom',\n      endpoint: 'https://your-custom-api.com/v1',\n      accounts: [\n        {\n          apiKey: 'custom-key-1',\n          models: [\n            {\n              name: 'custom-model-1',\n              rateLimit: {\n                rpm: 50,\n                tpm: 30000\n              }\n            },\n            'custom-model-2' // No rate limits\n          ]\n        }\n      ]\n    }\n  ],\n  strategy: 'rate-limit-aware' // Use rate-limit aware strategy for automatic load balancing\n});\n\n// Route a chat completion request\nconst response = await router.chat({\n  model: 'gpt-4',\n  messages: [{ role: 'user', content: 'Hello!' }]\n});\n\nconsole.log(response);\n```\n\n## 🎯 Per-Model Rate Limiting\n\nAI Router now supports per-model rate limiting, allowing you to set different rate limits for different models:\n\n```ts\nconst router = new AIRouter({\n  providers: [\n    {\n      name: 'provider',\n      accounts: [\n        {\n          apiKey: 'your-key',\n          models: [\n            // High-end model with strict limits\n            {\n              name: 'gpt-4',\n              rateLimit: {\n                rpm: 50,    // 50 requests per minute\n                tpm: 40000, // 40k tokens per minute\n                rpd: 1000   // 1000 requests per day\n              }\n            },\n            // Cheaper model with relaxed limits\n            {\n              name: 'gpt-3.5-turbo',\n              rateLimit: {\n                rpm: 200,\n                tpm: 150000\n              }\n            },\n            // No rate limits for this model\n            'claude-instant'\n          ]\n        }\n      ]\n    }\n  ],\n  strategy: 'rate-limit-aware'\n});\n```\n\n### Rate Limit Types\n\n- **`rpm`**: Requests Per Minute - Maximum number of API calls per minute\n- **`tpm`**: Tokens Per Minute - Maximum number of input tokens per minute  \n- **`rpd`**: Requests Per Day - Maximum number of API calls per day\n\nThe router will automatically select the best available model that can handle your request without exceeding rate limits.\n\n## ⚙️ Advanced: Middleware\n\nExtend behavior with middleware:\n\n```ts\nrouter.use(async (req, next) =\u003e {\n  console.log('Outgoing request:', req.url);\n  const start = Date.now();\n  const res = await next(req);\n  console.log('Response time:', Date.now() - start, 'ms');\n  return res;\n});\n\n// Add rate limiting\nrouter.use(rateLimit({ max: 1000 / 60 })); // 1000 RPM\n```\n\nBuild your own for caching, tracing, or authentication.\n\n## 🔁 Load Balancing Strategies\n\n```ts\nnew AIRouter({\n  providers: [...],\n  strategy: 'random' // or 'least-loaded'\n})\n```\n\n- `random`: Random pick (fast)\n- `least-loaded`: Pick least busy (requires health tracking)\n\n## 🧪 Running in Serverless / Edge\n\nWorks seamlessly in Vercel, Cloudflare Workers, Netlify, etc.:\n\n```ts\n// api/chat.js (Vercel Function)\nexport default async function handler(req, res) {\n  const { messages } = req.body;\n  const response = await router.chat({ model: 'gpt-4', messages });\n  res.json(response);\n}\n```\n\n## 📚 Examples\n\nExplore examples to see how to integrate `ai-router` with different frameworks and use cases:\n\n- [Hono AI Router Example](./examples/hono-ai-router/): Demonstrates how to use `ai-router` with the Hono web framework for managing multiple AI providers with rate limiting and load balancing. This example also supports OpenAI-compatible APIs.\n\n## 🛠️ Development\n\n```bash\ngit clone https://github.com/isaced/ai-router.git\ncd ai-router\nnpm install\nnpm run build\nnpm test\n```\n\nContributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for details.\n\n\n## 📄 License\n\nMIT\n\n---\n\n\u003e 🌐 Route your AI. Balance your load. Avoid your limits.\n\u003e\n\u003e Made with ❤️ for developers building the future of AI.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fisaced%2Fai-router","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fisaced%2Fai-router","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fisaced%2Fai-router/lists"}