{"id":50831930,"url":"https://github.com/reallyartificial/freeport","last_synced_at":"2026-06-14T00:01:47.971Z","repository":{"id":349173810,"uuid":"1199622338","full_name":"ReallyArtificial/freeport","owner":"ReallyArtificial","description":"Open-source LLM Gateway. Multi-provider routing, fallback, semantic caching, cost tracking, guardrails. Drop-in OpenAI API replacement. Self-hosted.","archived":false,"fork":false,"pushed_at":"2026-06-01T20:08:05.000Z","size":236,"stargazers_count":0,"open_issues_count":6,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-01T22:09:14.584Z","etag":null,"topics":["ai","ai-agents","ai-infrastructure","anthropic","developer-tools","gateway","llm","llm-gateway","open-source","openai","self-hosted","semantic-cache","typescript"],"latest_commit_sha":null,"homepage":"https://github.com/ReallyArtificial","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ReallyArtificial.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-02T14:34:35.000Z","updated_at":"2026-06-01T20:08:09.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ReallyArtificial/freeport","commit_stats":null,"previous_names":["reallyartificial/freeport"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/ReallyArtificial/freeport","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReallyArtificial%2Ffreeport","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReallyArtificial%2Ffreeport/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReallyArtificial%2Ffreeport/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReallyArtificial%2Ffreeport/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ReallyArtificial","download_url":"https://codeload.github.com/ReallyArtificial/freeport/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ReallyArtificial%2Ffreeport/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34304629,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-13T02:00:06.617Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","ai-agents","ai-infrastructure","anthropic","developer-tools","gateway","llm","llm-gateway","open-source","openai","self-hosted","semantic-cache","typescript"],"created_at":"2026-06-14T00:01:19.409Z","updated_at":"2026-06-14T00:01:47.915Z","avatar_url":"https://github.com/ReallyArtificial.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Freeport\n\n**Open-source LLM Gateway** — self-hosted, single Docker container.\n\nPrompt management, model fallback, semantic caching, cost tracking, guardrails, A/B testing, and an admin UI. Drop-in replacement for the OpenAI API — works with any OpenAI SDK.\n\n## Quick Start\n\n```bash\nnpm install\nnpm run build:ui   # build the admin dashboard\nnpm run dev        # starts on http://localhost:4000\n```\n\nOpen `http://localhost:4000/ui/` and add your API keys through the Providers page. No environment variables or config files required to get started.\n\nAlternatively, run with Docker:\n\n```bash\ndocker-compose up\n```\n\nThe gateway starts on `http://localhost:4000`. Admin UI at `http://localhost:4000/ui/`.\n\n## Usage\n\nPoint your OpenAI SDK at Freeport:\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:4000/v1\",\n    api_key=\"any-key\",  # or your configured API key\n)\n\nresponse = client.chat.completions.create(\n    model=\"gpt-4o-mini\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n)\n```\n\n```typescript\nimport OpenAI from 'openai';\n\nconst client = new OpenAI({\n  baseURL: 'http://localhost:4000/v1',\n  apiKey: 'any-key',\n});\n\nconst response = await client.chat.completions.create({\n  model: 'gpt-4o-mini',\n  messages: [{ role: 'user', content: 'Hello!' }],\n});\n```\n\nStreaming works identically — set `stream: true`.\n\n## Features\n\n### Multi-Provider Support\nRoute requests to OpenAI, Anthropic, and Google Gemini through a unified OpenAI-compatible API.\n\n```yaml\nproviders:\n  - name: openai\n    type: openai\n    keys:\n      - key: \"${OPENAI_API_KEY}\"\n  - name: anthropic\n    type: anthropic\n    keys:\n      - key: \"${ANTHROPIC_API_KEY}\"\n```\n\n### Fallback Chains + Circuit Breaker\nAutomatic failover across providers. If OpenAI is down, fall back to Anthropic.\n\n```yaml\nfallbackChains:\n  - name: primary\n    providers: [openai, anthropic, google]\n    circuitBreaker:\n      failureThreshold: 3\n      resetTimeoutMs: 60000\n```\n\n### Prompt Management\nVersion prompts externally. Update them without redeploying your app.\n\n```bash\n# Create a prompt\ncurl -X POST http://localhost:4000/api/prompts \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"slug\": \"summarize\", \"name\": \"Summarizer\"}'\n\n# Add a version and publish it\ncurl -X POST http://localhost:4000/api/prompts/{id}/versions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"content\": \"Summarize this: {{text}}\", \"tag\": \"published\"}'\n\n# Use it in requests (via freeport metadata)\ncurl -X POST http://localhost:4000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"gpt-4o-mini\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"placeholder\"}],\n    \"freeport\": {\"prompt\": \"summarize\", \"variables\": {\"text\": \"...\"}}\n  }'\n```\n\n### Semantic Caching\nSimilar prompts return cached responses. Uses local embeddings (all-MiniLM-L6-v2) — no external API calls.\n\n```yaml\ncache:\n  enabled: true\n  similarityThreshold: 0.95\n  ttlSeconds: 3600\n```\n\n### Spend Tracking \u0026 Budgets\nPer-project cost tracking with hard budget caps and kill switches.\n\n```bash\n# Create a project with a budget\ncurl -X POST http://localhost:4000/api/projects \\\n  -d '{\"name\": \"my-app\", \"budgetLimit\": 50}'\n\n# Set budget limits\ncurl -X POST http://localhost:4000/api/budgets/{projectId} \\\n  -d '{\"monthlyLimit\": 100, \"dailyLimit\": 10}'\n\n# Emergency kill switch\ncurl -X POST http://localhost:4000/api/budgets/{projectId}/kill \\\n  -d '{\"killed\": true}'\n```\n\n### Input/Output Guardrails\nPII detection (SSN, credit card, email, phone), content filtering, token limits. Plugin architecture for custom guardrails.\n\n```yaml\nguardrails:\n  enabled: true\n  piiDetection: true\n  contentFilter: true\n  maxTokens: 128000\n  customPlugins:\n    - my-custom-guardrail.js\n```\n\n### A/B Testing\nSplit traffic between prompt variants and track metrics.\n\n### Rate Limiting\nToken bucket rate limiter with per-key limits.\n\n```yaml\nrateLimit:\n  enabled: true\n  requestsPerMinute: 60\n```\n\n### Load Balancing\nRound-robin across multiple API keys per provider.\n\n```yaml\nproviders:\n  - name: openai\n    type: openai\n    keys:\n      - key: \"${OPENAI_KEY_1}\"\n      - key: \"${OPENAI_KEY_2}\"\n      - key: \"${OPENAI_KEY_3}\"\n```\n\n## API Endpoints\n\n### Proxy (OpenAI-compatible)\n| Method | Path | Description |\n|--------|------|-------------|\n| POST | `/v1/chat/completions` | Chat completion (streaming supported) |\n| POST | `/v1/completions` | Legacy completion |\n| POST | `/v1/embeddings` | Embedding passthrough |\n| GET | `/v1/models` | List available models |\n\n### Admin API\n| Method | Path | Description |\n|--------|------|-------------|\n| GET/POST | `/api/providers` | List/create LLM providers |\n| PUT/DELETE | `/api/providers/:id` | Update/delete provider |\n| GET/POST | `/api/prompts` | List/create prompts |\n| GET/PUT/DELETE | `/api/prompts/:id` | Get/update/delete prompt |\n| POST | `/api/prompts/:id/versions` | Create prompt version |\n| POST | `/api/prompts/resolve` | Resolve prompt with variables |\n| GET/POST | `/api/projects` | List/create projects |\n| GET/POST | `/api/budgets/:projectId` | Get/set budget |\n| POST | `/api/budgets/:projectId/kill` | Kill switch |\n| GET | `/api/logs` | Query request logs |\n| GET | `/api/logs/stats` | Usage analytics |\n| GET/POST | `/api/ab-tests` | List/create A/B tests |\n| GET | `/api/system/status` | System status |\n| POST | `/api/system/cache/clear` | Clear cache |\n| GET | `/health` | Health check |\n\n## Configuration\n\nProviders can be configured in three ways (any combination works):\n\n### Option 1: Admin UI (recommended for local dev)\n\nStart the server and open `http://localhost:4000/ui/`. Go to **Providers** and add your API keys. They are stored in the local SQLite database and persist across restarts.\n\n### Option 2: Environment Variables\n\n```bash\nFREEPORT_OPENAI_API_KEY=sk-xxx npm run dev\n```\n\n| Env Var | Description |\n|---------|-------------|\n| `FREEPORT_OPENAI_API_KEY` | OpenAI API key |\n| `FREEPORT_ANTHROPIC_API_KEY` | Anthropic API key |\n| `FREEPORT_GOOGLE_API_KEY` | Google API key |\n| `FREEPORT_ADMIN_API_KEY` | Admin API authentication key |\n| `FREEPORT_API_KEY` | Proxy API authentication key |\n| `FREEPORT_PORT` | Server port (default: 4000) |\n| `FREEPORT_HOST` | Server host (default: 0.0.0.0) |\n| `FREEPORT_CONFIG` | Path to config file |\n\n### Option 3: YAML Config File\n\n```bash\ncp config/freeport.example.yaml config/freeport.yaml\n# Edit with your API keys, then:\nnpm run dev\n```\n\nYAML values support `${ENV_VAR}` interpolation with `${VAR:-default}` syntax.\n\n## Architecture\n\n```\nClient (OpenAI SDK) --\u003e Freeport (Fastify)\n                          |\n                    Pre-Processing:\n                      Auth -\u003e Rate Limit -\u003e Budget Check -\u003e\n                      Prompt Resolution -\u003e Input Guardrails -\u003e\n                      Semantic Cache Lookup\n                          |\n                    Routing:\n                      A/B Router -\u003e Fallback Chain -\u003e Load Balancer\n                          |\n                    LLM Provider (OpenAI / Anthropic / Google)\n                          |\n                    Post-Processing:\n                      Output Guardrails -\u003e Cost Tracking -\u003e\n                      Budget Update -\u003e Cache Store -\u003e Log\n                          |\nClient \u003c-------------- Response\n```\n\n## Tech Stack\n\n- **Runtime**: Node.js + TypeScript + Fastify v5\n- **Database**: SQLite (better-sqlite3) — zero external dependencies\n- **Embeddings**: Local all-MiniLM-L6-v2 (optional, for semantic cache)\n- **Admin UI**: Preact + Vite\n- **Deployment**: Single Docker container\n\n## Custom Guardrail Plugins\n\nCreate a `.js` file in the `plugins/` directory:\n\n```javascript\nexport default {\n  name: 'my-guardrail',\n  checkInput(text) {\n    // Return { passed: true/false, guardrail: 'name', message: '...' }\n    if (text.includes('forbidden')) {\n      return { passed: false, guardrail: 'my-guardrail', message: 'Forbidden content' };\n    }\n    return { passed: true, guardrail: 'my-guardrail' };\n  },\n  checkOutput(text) {\n    return { passed: true, guardrail: 'my-guardrail' };\n  },\n};\n```\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freallyartificial%2Ffreeport","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freallyartificial%2Ffreeport","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freallyartificial%2Ffreeport/lists"}