{"id":49841988,"url":"https://github.com/reaatech/llm-router","last_synced_at":"2026-06-16T12:00:43.411Z","repository":{"id":352534231,"uuid":"1215369951","full_name":"reaatech/llm-router","owner":"reaatech","description":"Cost/latency/judgment-based model router — pluggable routing strategies, fallback chains, per-model cost telemetry, OTel spans, eval hooks. Ships a sample config for the frontier-judge + code-workhorse + local-inference pattern.","archived":false,"fork":false,"pushed_at":"2026-06-04T02:56:50.000Z","size":401,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-04T03:11:56.803Z","etag":null,"topics":["agentic-ai","ai-infrastructure","anthropic","bedrock","cost-optimization","fallback","gemini","latency","llm","llm-router","llmops","model-routing","multi-model","ollama","openai","opentelemetry","routing-engine","typescript","vllm"],"latest_commit_sha":null,"homepage":"https://reaatech.com/products/orchestration-protocols/llm-router","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reaatech.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-04-19T20:31:17.000Z","updated_at":"2026-06-04T02:56:05.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/reaatech/llm-router","commit_stats":null,"previous_names":["reaatech/llm-router"],"tags_count":2,"template":false,"template_full_name":null,"purl":"pkg:github/reaatech/llm-router","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reaatech%2Fllm-router","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reaatech%2Fllm-router/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reaatech%2Fllm-router/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reaatech%2Fllm-router/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reaatech","download_url":"https://codeload.github.com/reaatech/llm-router/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reaatech%2Fllm-router/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34404748,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-16T02:00:06.860Z","response_time":126,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["agentic-ai","ai-infrastructure","anthropic","bedrock","cost-optimization","fallback","gemini","latency","llm","llm-router","llmops","model-routing","multi-model","ollama","openai","opentelemetry","routing-engine","typescript","vllm"],"created_at":"2026-05-14T07:01:36.527Z","updated_at":"2026-06-16T12:00:43.405Z","avatar_url":"https://github.com/reaatech.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# llm-router\n\n[![CI](https://github.com/reaatech/llm-router/actions/workflows/ci.yml/badge.svg)](https://github.com/reaatech/llm-router/actions/workflows/ci.yml)\n[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)\n[![TypeScript](https://img.shields.io/badge/TypeScript-5.8-blue)](https://www.typescriptlang.org/)\n\n\u003e Cost-aware, multi-model LLM routing with pluggable strategies, fallback chains, cost telemetry, eval hooks, and MCP integration.\n\nThis monorepo provides an intelligent routing engine for LLM requests across multiple providers, balancing cost, latency, and quality. It includes strategy orchestration, circuit breakers, budget enforcement, quality scoring, A/B testing, and an MCP server for agent integration.\n\n## Features\n\n- **Pluggable routing strategies** — cost-optimized, latency-optimized, judgment-based (two-tier escalation), and capability-based routing\n- **Fallback chains** — ordered degradation paths with circuit breakers and exponential-backoff retry logic\n- **Cost telemetry** — real-time cost tracking, daily budget enforcement (soft/hard limits), alert thresholds, and structured cost reports\n- **Eval hooks** — pre-routing, post-routing, and post-execution hooks for quality monitoring and A/B testing\n- **Quality scoring** — rule-based, LLM-as-judge, and human-feedback scoring with multi-criteria evaluation\n- **MCP server** — exposes `route_request`, `get_model_info`, and `get_cost_report` tools via the Model Context Protocol\n- **Observability** — structured logging (Pino), OpenTelemetry tracing, metrics collection, and real-time dashboard snapshots\n- **CLI** — route, benchmark, cost-report, and validate-config commands\n- **Fail-closed security** — no live provider SDK integrations shipped; requires explicit `executeModel` callback or provider client registration\n\n## Installation\n\n### Using the packages\n\nPackages are published under the `@reaatech` scope and can be installed individually:\n\n```bash\n# Core types and schemas\npnpm add @reaatech/llm-router-core\n\n# Routing engine (includes observability, eval, config loading)\npnpm add @reaatech/llm-router-engine\n\n# Pluggable routing strategies\npnpm add @reaatech/llm-router-strategies\n\n# Fallback chains and circuit breakers\npnpm add @reaatech/llm-router-fallback\n\n# Cost telemetry and budget management\npnpm add @reaatech/llm-router-telemetry\n\n# MCP server integration\npnpm add @reaatech/llm-router-mcp\n\n# CLI tool\npnpm add @reaatech/llm-router-cli\n```\n\n### Contributing\n\n```bash\n# Clone the repository\ngit clone https://github.com/reaatech/llm-router.git\ncd llm-router\n\n# Install dependencies\npnpm install\n\n# Build all packages\npnpm build\n\n# Run the test suite\npnpm test\n\n# Run linting\npnpm lint\n```\n\n## Quick Start\n\nCreate a router from a YAML config and route a request:\n\n```typescript\nimport { LLMRouter, parseRouterConfig } from \"@reaatech/llm-router-engine\";\n\nconst router = LLMRouter.fromConfig(\n  parseRouterConfig(`\nmodels:\n  workhorses:\n    - id: glm-edge\n      provider: zhipu\n      cost_per_million_input: 0.30\n      cost_per_million_output: 0.60\n      max_tokens: 128000\n      capabilities: [general]\nstrategies:\n  default:\n    type: cost-optimized\n    workhorse_pool: [glm-edge]\nbudgets:\n  default:\n    daily_limit: 25\n    alert_thresholds: [0.5, 0.75, 0.9]\n    hard_limit: true\n  `),\n  {\n    executeModel: async (model, request) =\u003e ({\n      content: `Response from ${model.id}`,\n      inputTokens: 10,\n      outputTokens: 20,\n    }),\n  },\n);\n\nconst result = await router.route({\n  prompt: \"Explain the Observer pattern.\",\n  strategy: \"cost-optimized\",\n});\n\nconsole.log(result.model.id, result.cost);\n```\n\nSee the [`config/examples/`](./config/examples/) directory for complete configuration samples including cost-optimized, low-latency, and workhorse-judge setups.\n\n## Packages\n\n| Package | Description |\n| ------- | ----------- |\n| [`@reaatech/llm-router-core`](./packages/core) | Core types, Zod schemas, and input validation |\n| [`@reaatech/llm-router-engine`](./packages/engine) | Main routing engine with registry, eval, observability, and config loading |\n| [`@reaatech/llm-router-strategies`](./packages/strategies) | Pluggable routing strategies with priority-based orchestrator |\n| [`@reaatech/llm-router-fallback`](./packages/fallback) | Fallback chains, circuit breakers, and retry logic |\n| [`@reaatech/llm-router-telemetry`](./packages/telemetry) | Cost tracking, budget management, and metrics collection |\n| [`@reaatech/llm-router-mcp`](./packages/mcp) | MCP server exposing routing tools to AI agents |\n| [`@reaatech/llm-router-cli`](./packages/cli) | Command-line interface for routing, benchmarking, and reporting |\n\n## Documentation\n\n- [`ARCHITECTURE.md`](./ARCHITECTURE.md) — System design, package relationships, and data flows\n- [`AGENTS.md`](./AGENTS.md) — Agent development guide with strategy configs and security checklist\n- [`CONTRIBUTING.md`](./CONTRIBUTING.md) — Contribution workflow and quality gate commands\n- [`skills/`](./skills/) — Domain-specific guides for each routing capability\n\n## License\n\n[MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freaatech%2Fllm-router","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freaatech%2Fllm-router","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freaatech%2Fllm-router/lists"}