{"id":51022996,"url":"https://github.com/thomas-illiet/cerbai","last_synced_at":"2026-06-21T17:30:29.708Z","repository":{"id":352423152,"uuid":"1215089801","full_name":"thomas-illiet/cerbai","owner":"thomas-illiet","description":"CerbAI is a lightweight HTTP reverse proxy written in Go, built for corporate environments where access to LLM services is secured via JWT tokens obtained through an OAuth2 client_credentials flow over mutual TLS (mTLS).","archived":false,"fork":false,"pushed_at":"2026-04-19T14:58:55.000Z","size":19,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-19T15:31:00.642Z","etag":null,"topics":["jwt","mtls","oauth","openai","proxy"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thomas-illiet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-04-19T13:19:10.000Z","updated_at":"2026-04-19T14:58:59.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/thomas-illiet/cerbai","commit_stats":null,"previous_names":["thomas-illiet/cerbai"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/thomas-illiet/cerbai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fcerbai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fcerbai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fcerbai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fcerbai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thomas-illiet","download_url":"https://codeload.github.com/thomas-illiet/cerbai/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fcerbai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34620358,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-21T02:00:05.568Z","response_time":54,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jwt","mtls","oauth","openai","proxy"],"created_at":"2026-06-21T17:30:25.080Z","updated_at":"2026-06-21T17:30:29.703Z","avatar_url":"https://github.com/thomas-illiet.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# CerbAI\n\n![banner](assets/banner.png)\n\n[![CI](https://github.com/thomas-illiet/cerbai/actions/workflows/ci.yml/badge.svg)](https://github.com/thomas-illiet/cerbai/actions/workflows/ci.yml)\n[![Release](https://github.com/thomas-illiet/cerbai/actions/workflows/release.yml/badge.svg)](https://github.com/thomas-illiet/cerbai/actions/workflows/release.yml)\n\nCerbAI is a lightweight HTTP reverse proxy written in Go, designed for corporate environments where LLM access is protected by a JWT obtained through an OAuth2 `client_credentials` flow over mTLS. It is Kubernetes-native: configured entirely through environment variables or CLI flags — no config file required.\n\n## How it works\n\n```none\nClient (OpenAI SDK / curl)\n        │\n        ▼\n   ┌─────────┐    client_credentials   ┌───────────────┐\n   │ CerbAI  │ ─────── mTLS ─────────► │ Token Service │\n   │  :8085  │ ◄──── JWT token ─────── └───────────────┘\n   └────┬────┘    (memory or Redis)\n        │  Authorization: Bearer \u003ctoken\u003e\n        ▼\n   ┌──────────┐\n   │ LLM API  │\n   └──────────┘\n```\n\n1. On each incoming request, CerbAI retrieves a valid JWT from its cache (memory or Redis).\n2. On a cache miss, it refreshes the token via `client_credentials` over mTLS.\n3. It injects the token into the `Authorization` header and forwards the request to the LLM.\n4. SSE streaming responses (`text/event-stream`) are forwarded chunk by chunk with no buffering.\n\n## Requirements\n\n- Go 1.23+ (or Docker)\n- A client certificate (cert + key) for mTLS\n- Access to an internal token service and LLM endpoint\n- Redis (optional, recommended for multi-replica deployments)\n\n## Installation\n\n### From source\n\n```bash\ngit clone https://github.com/thomas-illiet/cerbai\ncd cerbai\ngo build -o cerbai .\n```\n\n### Docker\n\n```bash\ndocker pull ghcr.io/thomas-illiet/cerbai:latest\n```\n\n## Configuration\n\nCerbAI is configured exclusively through **CLI flags** or **environment variables** — no config file needed.\n\n### CLI flags\n\n```\nUsage:\n  cerbai [flags]\n\nFlags:\n      --client-id string          OAuth2 client ID (env: CERBAI_CLIENT_ID)\n      --client-secret string      OAuth2 client secret (env: CERBAI_CLIENT_SECRET)\n  -h, --help                      help for cerbai\n      --listen-addr string        Address to listen on (env: CERBAI_LISTEN_ADDR) (default \":8085\")\n      --llm-url string            Upstream LLM base URL (env: CERBAI_LLM_URL)\n      --proxy-token string        Bearer token required to use the proxy, optional (env: CERBAI_PROXY_TOKEN)\n      --redis-url string          Redis URL for shared token cache, optional (env: CERBAI_REDIS_URL)\n      --tls-ca-file string        Custom CA certificate file, optional (env: CERBAI_TLS_CA_FILE)\n      --tls-cert-file string      mTLS client certificate file path (env: CERBAI_TLS_CERT_FILE)\n      --tls-key-file string       mTLS client key file path (env: CERBAI_TLS_KEY_FILE)\n      --token-cache-ttl duration  Token cache TTL (env: CERBAI_TOKEN_CACHE_TTL) (default 5m0s)\n      --token-endpoint string     OAuth2 token endpoint URL (env: CERBAI_TOKEN_ENDPOINT)\n      --token-header string       Header name to inject the token into (env: CERBAI_TOKEN_HEADER) (default \"Authorization\")\n      --token-prefix string       Token value prefix (env: CERBAI_TOKEN_PREFIX) (default \"Bearer \")\n      --log-level string          Log level: debug, info, warn, error (env: CERBAI_LOG_LEVEL) (default \"info\")\n```\n\n### Environment variables\n\n| Variable                 | Default         | Description                                   |\n| ------------------------ | --------------- | --------------------------------------------- |\n| `CERBAI_LISTEN_ADDR`     | `:8085`         | Address to listen on                          |\n| `CERBAI_LLM_URL`         | —               | Upstream LLM base URL                         |\n| `CERBAI_TOKEN_ENDPOINT`  | —               | OAuth2 token endpoint URL                     |\n| `CERBAI_CLIENT_ID`       | —               | OAuth2 client ID                              |\n| `CERBAI_CLIENT_SECRET`   | —               | OAuth2 client secret                          |\n| `CERBAI_TLS_CERT_FILE`   | —               | mTLS client certificate file path             |\n| `CERBAI_TLS_KEY_FILE`    | —               | mTLS client key file path                     |\n| `CERBAI_TLS_CA_FILE`     | —               | Custom CA file (optional, uses system CAs)    |\n| `CERBAI_TOKEN_CACHE_TTL` | `5m`            | Token cache TTL                               |\n| `CERBAI_TOKEN_HEADER`    | `Authorization` | Header name for token injection               |\n| `CERBAI_TOKEN_PREFIX`    | `Bearer `       | Token value prefix                            |\n| `CERBAI_PROXY_TOKEN`     | —               | Bearer token to access the proxy (optional)   |\n| `CERBAI_REDIS_URL`       | —               | Redis URL (optional, see Token cache section) |\n| `CERBAI_LOG_LEVEL`       | `info`          | Log level: debug, info, warn, error           |\n\n## Running\n\n### Minimal example\n\n```bash\n./cerbai \\\n  --llm-url https://llm.internal.example.com \\\n  --token-endpoint https://auth.internal.example.com/oauth2/token \\\n  --client-id my-client \\\n  --client-secret my-secret \\\n  --tls-cert-file /etc/certs/client.crt \\\n  --tls-key-file  /etc/certs/client.key\n```\n\n### With Docker Compose\n\n```bash\ncp .env.example .env\n# Edit .env with your values\n\n# Place your mTLS certificates in ./certs/\n# certs/client.crt  certs/client.key  certs/ca.crt\n\ndocker compose up -d\n```\n\nThis starts CerbAI alongside a Redis instance used for token caching.\n\n## Token cache\n\nCerbAI supports two cache backends:\n\n### In-memory (default)\n\nLocal to each instance. Simple, no external dependency.\n\n```bash\n./cerbai --token-cache-ttl 10m ...\n```\n\n### Redis (optional — recommended for multi-replica)\n\nAll replicas share the same cached token. Avoids N simultaneous requests to the token service during a scale-out event.\n\n```bash\n./cerbai --redis-url redis://redis:6379/0 ...\n# With TLS:\n./cerbai --redis-url rediss://redis:6379/0 ...\n```\n\n| Behaviour             | Detail                                              |\n| --------------------- | --------------------------------------------------- |\n| Configurable TTL      | `--token-cache-ttl` (default: 5m)                   |\n| Respects `expires_in` | Uses `min(configured TTL, expires_in - 30s)`        |\n| Thread-safe (memory)  | `sync.RWMutex` with double-checked locking          |\n| Atomic (Redis)        | Native `SET EX`, no mutex required                  |\n| Startup warm-up       | Token pre-fetched at boot; non-fatal if unavailable |\n\n## Proxy authentication\n\nWhen `--proxy-token` (or `CERBAI_PROXY_TOKEN`) is set, all requests to the proxy must include a matching `Authorization` header. Omit the flag to disable auth entirely.\n\n```bash\n./cerbai --proxy-token my-secret-key ...\n```\n\n```bash\ncurl http://localhost:8085/v1/chat/completions \\\n  -H \"Authorization: Bearer my-secret-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"gpt-4\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello!\"}]}'\n```\n\nRequests without a valid token receive `401 Unauthorized`. The `/healthz` endpoint is always public.\n\n## Health check\n\nCerbAI exposes a health endpoint at `GET /healthz` that returns `200 ok` when the process is running.\n\n```bash\ncurl http://localhost:8085/healthz\n```\n\n## Usage\n\nPoint your OpenAI-compatible client at `http://localhost:8085`:\n\n```bash\n# Non-streaming\ncurl http://localhost:8085/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"gpt-4\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello!\"}]}'\n\n# SSE streaming\ncurl http://localhost:8085/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"gpt-4\",\"stream\":true,\"messages\":[{\"role\":\"user\",\"content\":\"Hello!\"}]}'\n```\n\nPython (OpenAI SDK):\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:8085/v1\",\n    api_key=\"not-used\",  # token is managed by CerbAI\n)\n\nstream = client.chat.completions.create(\n    model=\"gpt-4\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n    stream=True,\n)\nfor chunk in stream:\n    print(chunk.choices[0].delta.content, end=\"\", flush=True)\n```\n\n## Logs\n\nStructured JSON logs on stdout. Set log level with `--log-level` or `CERBAI_LOG_LEVEL`:\n\n```bash\n./cerbai --log-level debug ...\n# or\nexport CERBAI_LOG_LEVEL=debug\n```\n\nAvailable levels: `debug`, `info`, `warn`, `error` (default: `info`)\n\n### Example logs\n\n```json\n{\"time\":\"2026-04-19T10:00:00Z\",\"level\":\"INFO\",\"msg\":\"starting CerbAI\",\"version\":\"v1.2.0\",\"commit\":\"abc1234\",\"build_date\":\"2026-04-19T10:00:00Z\"}\n{\"time\":\"2026-04-19T10:00:00Z\",\"level\":\"INFO\",\"msg\":\"config loaded\",\"listen_addr\":\":8085\",\"llm_url\":\"https://llm.internal.example.com\",\"token_cache_ttl\":\"5m0s\",\"redis\":true}\n{\"time\":\"2026-04-19T10:00:00Z\",\"level\":\"INFO\",\"msg\":\"token refreshed\",\"ttl\":\"4m30s\",\"backend\":\"redis\",\"duration_ms\":45}\n{\"time\":\"2026-04-19T10:00:00Z\",\"level\":\"INFO\",\"msg\":\"starting proxy server\",\"addr\":\":8085\"}\n```\n\n### Debug level logs\n\nAt `debug` level, additional request-level logging:\n\n```json\n{\"time\":\"2026-04-19T10:00:01Z\",\"level\":\"DEBUG\",\"msg\":\"incoming request\",\"path\":\"/v1/chat/completions\",\"method\":\"POST\",\"remote_addr\":\"127.0.0.1:54321\"}\n{\"time\":\"2026-04-19T10:00:01Z\",\"level\":\"DEBUG\",\"msg\":\"token fetched\",\"duration_ms\":2}\n{\"time\":\"2026-04-19T10:00:01Z\",\"level\":\"DEBUG\",\"msg\":\"request completed\",\"path\":\"/v1/chat/completions\",\"method\":\"POST\",\"duration_ms\":1250}\n{\"time\":\"2026-04-19T10:00:02Z\",\"level\":\"WARN\",\"msg\":\"auth failed\",\"path\":\"/v1/chat/completions\",\"method\":\"POST\",\"remote_addr\":\"127.0.0.1:54322\"}\n```\n\n## Performance testing\n\nCerbAI ships with a [k6](https://k6.io) performance test suite. Tests run entirely locally via Docker Compose against a lightweight Go mock server that simulates the OAuth2 token endpoint and the LLM API — no real credentials or upstream required.\n\n### Architecture\n\n```none\nk6 ──► cerbai :8085 ──► mock-server :9090\n                │              │\n                │         POST /token          (fake OAuth2 response)\n                └────────► POST /v1/chat/completions  (fake LLM response)\n```\n\n### Prerequisites\n\n- Docker with Compose plugin\n\n### Quick start\n\n```bash\n# Build the mock server image (once)\nmake perf-build\n\n# Smoke test — 2 VUs for 1 minute\nmake perf-smoke\n\n# Load test — 50 VUs for 2 minutes\nmake perf-load\n\n# Stress test — ramp up to 200 VUs to find the saturation point\nmake perf-stress\n\n# Soak test — 20 VUs for 30 minutes (detects memory leaks and degradation)\nmake perf-soak\n\n# Smoke → load → stress in sequence\nmake perf-all\n\n# Tear down the perf environment\nmake perf-down\n```\n\n### Test scenarios\n\n| Scenario | VUs            | Duration | p95 threshold | Error threshold |\n| -------- | -------------- | -------- | ------------- | --------------- |\n| Smoke    | 2              | 1 min    | \u003c 500 ms      | \u003c 1%            |\n| Load     | 50             | 2 min    | \u003c 1 s         | \u003c 2%            |\n| Stress   | 0 → 200 (ramp) | 8 min    | \u003c 2 s         | \u003c 5%            |\n| Soak     | 20             | 30 min   | \u003c 1 s         | \u003c 1%            |\n\n### Optional environment variables\n\n| Variable      | Default              | Description                                  |\n| ------------- | -------------------- | -------------------------------------------- |\n| `PROXY_URL`   | `http://cerbai:8085` | CerbAI address as seen from the k6 container |\n| `PROXY_TOKEN` | _(empty)_            | Bearer token if `--proxy-token` is set       |\n\nPass them inline to override:\n\n```bash\nPROXY_TOKEN=my-secret make perf-smoke\n```\n\n### File layout\n\n```none\ntests/perf/\n├── mock_server/\n│   └── main.go          — Go mock server (OAuth2 token + LLM endpoints)\n└── k6/\n    ├── smoke.js\n    ├── load.js\n    ├── stress.js\n    └── soak.js\ndocker-compose.perf.yml  — Override: points cerbai at mock-server (no TLS)\n```\n\n## CI / CD\n\n| Workflow      | Trigger                          | Action                                 |\n| ------------- | -------------------------------- | -------------------------------------- |\n| `ci.yml`      | Push to any branch, PR to `main` | Build, vet, test, Dockerfile lint      |\n| `release.yml` | Push tag `v*`                    | Multi-arch Docker build \u0026 push to GHCR |\n\nTo release a new version:\n\n```bash\ngit tag v1.0.0\ngit push origin v1.0.0\n```\n\nThe release workflow builds `linux/amd64` and `linux/arm64` images and publishes them to `ghcr.io/thomas-illiet/cerbai` with tags `v1.0.0`, `v1.0`, `v1`, and `sha-\u003cshort\u003e`.\n\n## Architecture\n\n```none\ninternal/\n├── config/\n│   └── config.go       — Viper config, Cobra flags, TLS config builder\n├── middleware/\n│   └── auth.go         — Bearer token auth middleware for proxy access\n├── proxy/\n│   └── handler.go      — httputil.ReverseProxy, token injection, SSE streaming\n└── token/\n    ├── cache.go        — In-memory cache (RWMutex double-check) + OAuth2 fetcher\n    └── redis.go        — Redis cache (atomic SET EX, multi-replica safe)\nmain.go                 — Cobra CLI, cache selection, token warmup, graceful shutdown\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomas-illiet%2Fcerbai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthomas-illiet%2Fcerbai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomas-illiet%2Fcerbai/lists"}