{"id":40273473,"url":"https://github.com/bluewave-labs/langroute","last_synced_at":"2026-01-20T03:02:48.726Z","repository":{"id":278986810,"uuid":"929908438","full_name":"bluewave-labs/LangRoute","owner":"bluewave-labs","description":"This is a robust and configurable LLM proxy server built with Node.js, Express, and PostgreSQL. It acts as an intermediary between your applications and various Large Language Model (LLM) providers","archived":false,"fork":false,"pushed_at":"2026-01-02T12:43:26.000Z","size":19428,"stargazers_count":30,"open_issues_count":14,"forks_count":9,"subscribers_count":1,"default_branch":"master","last_synced_at":"2026-01-14T01:37:07.169Z","etag":null,"topics":["llm","llm-gateway","llm-proxy","llmproxy","proxy"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bluewave-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-02-09T17:19:17.000Z","updated_at":"2026-01-02T12:43:38.000Z","dependencies_parsed_at":"2025-06-18T13:24:14.510Z","dependency_job_id":"b98b71c2-d002-4064-b9ab-891db0060836","html_url":"https://github.com/bluewave-labs/LangRoute","commit_stats":null,"previous_names":["bluewave-labs/langroute"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/bluewave-labs/LangRoute","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluewave-labs%2FLangRoute","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluewave-labs%2FLangRoute/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluewave-labs%2FLangRoute/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluewave-labs%2FLangRoute/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bluewave-labs","download_url":"https://codeload.github.com/bluewave-labs/LangRoute/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bluewave-labs%2FLangRoute/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28594958,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-20T02:08:49.799Z","status":"ssl_error","status_checked_at":"2026-01-20T02:08:44.148Z","response_time":117,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["llm","llm-gateway","llm-proxy","llmproxy","proxy"],"created_at":"2026-01-20T03:02:44.518Z","updated_at":"2026-01-20T03:02:48.715Z","avatar_url":"https://github.com/bluewave-labs.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\u003ca href=\"https://bluewavelabs.ca\" target=\"_blank\"\u003eLangRoute\u003c/a\u003e\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\u003cstrong\u003eAn LLM proxy server built with Next.js, shadcn/ui, Node.js, Express and PostgreSQL. \u003c/strong\u003e\u003c/p\u003e\n\n\u003cimg width=\"1463\" alt=\"image\" src=\"https://github.com/user-attachments/assets/e5df3a9c-c948-42d4-b260-d7086ccb4650\" /\u003e\n\n## Table of Contents\n- [Table of Contents](#table-of-contents)\n- [Overview](#overview)\n- [Features](#features)\n- [Architecture](#architecture)\n- [Tech stack](#tech-stack)\n- [Quick start](#quick-start)\n\t- [Prerequisites](#prerequisites)\n\t- [Environment](#environment)\n\t- [Commands](#commands)\n- [Project structure](#project-structure)\n- [Configuration and scripts](#configuration-and-scripts)\n- [Testing](#testing)\n- [CI/CD and deployment](#cicd-and-deployment)\n- [Conventions](#conventions)\n- [Community](#community)\n- [Security](#security)\n- [Troubleshooting / FAQ](#troubleshooting--faq)\n- [License](#license)\n\n## Overview\nLangRoute is a flexible LLM proxy that routes, throttles, and logs calls across multiple model providers. Core logic runs on Node.js, Express, and PostgreSQL, while a Next.js + shadcn UI lets admins tweak configs and watch metrics. One endpoint in, the best model out.\n\nSee the [architecture document](docs/architecture.md) for a deeper dive into system design.\n\n## Features\n- **Multi-model support**\n  - Connect OpenAI, Anthropic, Google, Azure OpenAI, or any local model via LLM adapters.\n- **Smart routing**\n  - Pick the fastest, cheapest, or most accurate model on every request.\n  - Auto-retry with multi-provider fallback when a vendor is unavailable.\n- **Access Keys \u0026 rate limits**\n  - Generate Access Keys in Key Management to call LangRoute from your applications.\n  - Enforce per-key and per-tenant rate limits and budgets.\n  - Provider API Keys (Secrets) are encrypted at rest and configured per model.\n- **Logging \u0026 monitoring**\n  - Track latency, token spend, and full request metadata (prompts redacted in privacy mode).\n  - Ship metrics to Prometheus or Grafana.\n- **Security \u0026 privacy**\n  - Tenant isolation with row-level security.\n  - Access Keys shown in full only once at creation; UI displays secure previews.\n  - Provider API Keys encrypted at rest and never logged.\n- **Admin \u0026 dev tools**\n  - Web playground for side-by-side model tests.\n  - Hot-reload configs without downtime.\n  - Web dashboard for managing Access Keys, configuring providers, and viewing analytics.\n  - Toggle cache, rate limits, and routing rules from the dashboard.\n\n## Architecture\n1. **Unified API** – Send requests to `/api/completions` (canonical) or `/api/v1/chat/completions` (OpenAI-compatible alias).\n2. **Routing engine** – Reads model registry, weights, and provider health, then selects the appropriate LLM adapter.\n3. **LLM adapters** – Translate requests to provider-specific formats (OpenAI, Anthropic, Google, Azure, etc.).\n4. **Middleware chain** – Handles Access Key validation, request validation, caching, and token counting.\n5. **Async workers** – Push logs and metrics to PostgreSQL and observability backends.\n\n## Tech stack\n| Layer        | Tooling                                        |\n| ------------ | ---------------------------------------------- |\n| HTTP server  | Node.js, Express                               |\n| Admin UI     | Next.js, React, shadcn/ui, Radix UI components |\n| Styling      | Tailwind CSS                                   |\n| Language     | TypeScript                                     |\n| Data store   | PostgreSQL (Redis optional)                    |\n| Auth         | next-auth with Prisma adapter                  |\n| State \u0026 data | @tanstack/react-query, Zustand                 |\n\n## Quick start\n### Prerequisites\n- Node.js 20+\n- npm (bundled with Node)\n\n### Environment\n- Copy and adjust [`env/.env.example`](env/.env.example) before running the app. Critical variables include:\n  - `DATABASE_URL` – PostgreSQL connection string.\n  - `AUTH_SECRET` – Secret for NextAuth.\n  - `AUTH_URL` – Base URL for auth callbacks.\n  - `REDIS_URL` – Optional Redis connection.\n\n### Commands\n```bash\nnpm install               # Install dependencies\nnpm run env               # Generate .env from env/ templates\nnpm run db boot           # (Optional) start local Postgres via Docker\nnpm run dev               # Start Next.js dev server with Turbopack\nnpm run build             # Build for production\nnpm start                 # Start production server\nnpm run lint              # Run ESLint\nnpm run check             # Type-check and lint\n```\n\nSee [docs/getting-started.md](docs/getting-started.md) for workflows and additional setup notes.\n\n## Project structure\n```\n.\n├─ docs/                  # Additional project documentation\n├─ env/                   # Environment templates\n├─ prisma/                # Prisma schema \u0026 seeds\n├─ scripts/               # Helper scripts (dev, env, db)\n├─ src/\n│  ├─ app/                # Next.js routes and API handlers\n│  ├─ db/                 # Prisma client wrapper\n│  ├─ lib/                # Shared models, config, utils, middleware\n│  └─ shadcn-ui/          # Reusable UI components\n├─ docker/                # Docker compose for Postgres\n├─ public/                # Static assets\n└─ ...\n```\n\n## Configuration and scripts\nKey configuration files:\n- [`next.config.ts`](next.config.ts) – Next.js configuration.\n- [`tailwind.config.js`](tailwind.config.js) – Tailwind CSS settings.\n- [`tsconfig.json`](tsconfig.json) – TypeScript compiler options.\n- [`eslint.config.mjs`](eslint.config.mjs) – ESLint setup.\n- [`.prettierrc`](.prettierrc) – Prettier rules.\n\nCommon npm scripts:\n\n| Script     | Description                                        |\n| ---------- | -------------------------------------------------- |\n| `dev`      | Start the development server (selects target env)  |\n| `dev:boot` | Run checks, prepare env, boot DB, start dev server |\n| `build`    | Build the app for production                       |\n| `start`    | Start the built app                                |\n| `lint`     | Run ESLint                                         |\n| `check`    | Run TypeScript `tsc --noEmit` and lint             |\n| `db`       | Manage Dockerized Postgres (`npm run db help`)     |\n| `env`      | Generate a consolidated `.env` file                |\n\n## Testing\nNo test suite is currently defined. Running `npm test` yields no tests (TODO). Linting and type checks can be run with `npm run lint` and `npm run check`.\n\n## CI/CD and deployment\nNo GitHub workflows or deployment configuration are present yet (TODO).\n\n## Conventions\n- TypeScript with strict compiler options.\n- ESLint (`next/core-web-vitals`) and Prettier enforce style.\n- Import sorting and Tailwind class ordering via Prettier plugins.\n\n## Community\n- Review the [Contributing guidelines](CONTRIBUTING.md)\n- Follow our [Code of Conduct](CODE_OF_CONDUCT.md)\n\n## Security\nSecurity issues can be reported via our [Security Policy](SECURITY.md).\n\n## Troubleshooting / FAQ\n- Use Node.js 20+ to match the TypeScript and Next.js setup.\n- Run `npm run env` if `.env` is missing or outdated.\n- Start the local database with `npm run db boot` before running the app.\n- Ensure required environment variables like `DATABASE_URL` and `AUTH_SECRET` are set.\n- Set `REDIS_URL` if enabling Redis-backed rate limiting.\n- Docker must be running for database scripts to succeed.\n\n## License\nNo license file is currently included (TODO). Ownership and licensing terms are undecided.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluewave-labs%2Flangroute","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbluewave-labs%2Flangroute","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbluewave-labs%2Flangroute/lists"}