{"id":48291636,"url":"https://github.com/palenaai/litellm-operator","last_synced_at":"2026-07-10T12:00:39.130Z","repository":{"id":348665199,"uuid":"1198801850","full_name":"PalenaAI/litellm-operator","owner":"PalenaAI","description":"Kubernetes operator for deploying and managing LiteLLM AI Gateway. Declarative CRDs for models, teams, users, and virtual keys with bidirectional config sync, SSO/SCIM user provisioning, and OLM support. Built with Operator SDK.","archived":false,"fork":false,"pushed_at":"2026-07-04T19:40:19.000Z","size":1348,"stargazers_count":6,"open_issues_count":1,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-04T21:12:14.613Z","etag":null,"topics":["ai-gateway","cloud-native","gitops","kubernetes","kubernetes-operator","litellm","llm","openai","openshift","operator"],"latest_commit_sha":null,"homepage":"https://litellm-operator.palena.ai/","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/PalenaAI.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":"CLA.md"}},"created_at":"2026-04-01T19:17:22.000Z","updated_at":"2026-07-04T19:40:21.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/PalenaAI/litellm-operator","commit_stats":null,"previous_names":["bitkaio/litellm-operator"],"tags_count":16,"template":false,"template_full_name":null,"purl":"pkg:github/PalenaAI/litellm-operator","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PalenaAI%2Flitellm-operator","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PalenaAI%2Flitellm-operator/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PalenaAI%2Flitellm-operator/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PalenaAI%2Flitellm-operator/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/PalenaAI","download_url":"https://codeload.github.com/PalenaAI/litellm-operator/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/PalenaAI%2Flitellm-operator/sbom","scorecard":{"id":1246987,"data":{"date":"2026-05-05T08:44:23Z","repo":{"name":"github.com/PalenaAI/litellm-operator","commit":"c3e046fb33023d0e0d83628f2eee8dac3e5162ca"},"scorecard":{"version":"v5.0.0","commit":"ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4"},"score":6,"checks":[{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#binary-artifacts"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#branch-protection"}},{"name":"CI-Tests","score":10,"reason":"3 out of 3 merged PRs checked by a CI test -- score normalized to 10","details":null,"documentation":{"short":"Determines if the project runs tests before pull requests are merged.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#ci-tests"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#cii-best-practices"}},{"name":"Code-Review","score":0,"reason":"Found 2/30 approved changesets -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#code-review"}},{"name":"Contributors","score":0,"reason":"project has 0 contributing companies or organizations -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project has a set of contributors from multiple organizations (e.g., companies).","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#contributors"}},{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#dangerous-workflow"}},{"name":"Dependency-Update-Tool","score":10,"reason":"update tool detected","details":["Info: detected update tool: RenovateBot: renovate.json:1"],"documentation":{"short":"Determines if the project uses a dependency update tool.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#dependency-update-tool"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#license"}},{"name":"Maintained","score":0,"reason":"project was created in last 90 days. please review its contents carefully","details":["Warn: Repository was created in last 90 days."],"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#maintained"}},{"name":"Packaging","score":10,"reason":"packaging workflow detected","details":["Info: Project packages its releases by way of GitHub Actions.: .github/workflows/e2e.yml:107"],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#packaging"}},{"name":"Pinned-Dependencies","score":8,"reason":"dependency not pinned by hash detected -- score normalized to 8","details":["Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:138","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:139","Warn: pipCommand not pinned by hash: .github/workflows/ci.yml:250","Warn: goCommand not pinned by hash: .github/workflows/ci.yml:429","Warn: downloadThenRun not pinned by hash: .github/workflows/e2e.yml:177","Warn: downloadThenRun not pinned by hash: .github/workflows/release.yml:200","Warn: pipCommand not pinned by hash: .github/workflows/release.yml:221","Warn: goCommand not pinned by hash: .github/workflows/scheduled.yml:74","Info:  54 out of  54 GitHub-owned GitHubAction dependencies pinned","Info:  39 out of  39 third-party GitHubAction dependencies pinned","Info:   2 out of   2 containerImage dependencies pinned","Info:   0 out of   4 pipCommand dependencies pinned","Info:   2 out of   4 goCommand dependencies pinned","Info:   0 out of   2 downloadThenRun dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#pinned-dependencies"}},{"name":"SAST","score":7,"reason":"SAST tool detected but not run on all commits","details":["Info: SAST configuration detected: CodeQL","Info: SAST configuration detected: CodeQL","Warn: 0 commits out of 3 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#sast"}},{"name":"Security-Policy","score":10,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Info: Found linked content: SECURITY.md:1","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#security-policy"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v0.11.1 not signed: https://api.github.com/repos/PalenaAI/litellm-operator/releases/310412768","Warn: release artifact v0.11.0 not signed: https://api.github.com/repos/PalenaAI/litellm-operator/releases/310387154","Warn: release artifact v0.10.0 not signed: https://api.github.com/repos/PalenaAI/litellm-operator/releases/308563286","Warn: release artifact v0.9.0 not signed: https://api.github.com/repos/PalenaAI/litellm-operator/releases/307905568","Warn: release artifact v0.7.0 not signed: https://api.github.com/repos/PalenaAI/litellm-operator/releases/305758220","Warn: release artifact v0.11.1 does not have provenance: https://api.github.com/repos/PalenaAI/litellm-operator/releases/310412768","Warn: release artifact v0.11.0 does not have provenance: https://api.github.com/repos/PalenaAI/litellm-operator/releases/310387154","Warn: release artifact v0.10.0 does not have provenance: https://api.github.com/repos/PalenaAI/litellm-operator/releases/308563286","Warn: release artifact v0.9.0 does not have provenance: https://api.github.com/repos/PalenaAI/litellm-operator/releases/307905568","Warn: release artifact v0.7.0 does not have provenance: https://api.github.com/repos/PalenaAI/litellm-operator/releases/305758220"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#signed-releases"}},{"name":"Token-Permissions","score":10,"reason":"GitHub workflow tokens follow principle of least privilege","details":["Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:230","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:292","Info: jobLevel 'pull-requests' permission set to 'read': .github/workflows/ci.yml:293","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:408","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:65","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:199","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:268","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:308","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:329","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:370","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:21","Info: jobLevel 'contents' permission set to 'read': .github/workflows/ci.yml:113","Info: jobLevel 'contents' permission set to 'read': .github/workflows/e2e.yml:21","Info: jobLevel 'contents' permission set to 'read': .github/workflows/e2e.yml:62","Info: jobLevel 'contents' permission set to 'read': .github/workflows/e2e.yml:112","Info: jobLevel 'contents' permission set to 'read': .github/workflows/release.yml:19","Warn: jobLevel 'contents' permission set to 'write': .github/workflows/release.yml:45","Info: jobLevel 'contents' permission set to 'read': .github/workflows/scheduled.yml:212","Info: jobLevel 'contents' permission set to 'read': .github/workflows/scheduled.yml:16","Info: jobLevel 'contents' permission set to 'read': .github/workflows/scheduled.yml:53","Info: jobLevel 'contents' permission set to 'read': .github/workflows/scheduled.yml:114","Info: jobLevel 'contents' permission set to 'read': .github/workflows/scorecard.yml:20","Info: jobLevel 'actions' permission set to 'read': .github/workflows/scorecard.yml:21","Warn: jobLevel 'contents' permission set to 'write': .github/workflows/sync-docs.yml:18","Info: topLevel 'contents' permission set to 'read': .github/workflows/ci.yml:10","Info: topLevel 'contents' permission set to 'read': .github/workflows/e2e.yml:10","Info: topLevel 'contents' permission set to 'read': .github/workflows/release.yml:9","Info: topLevel 'contents' permission set to 'read': .github/workflows/scheduled.yml:9","Info: topLevel permissions set to 'read-all': .github/workflows/scorecard.yml:11","Info: topLevel 'contents' permission set to 'read': .github/workflows/sync-docs.yml:11"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#token-permissions"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/ea7e27ed41b76ab879c862fa0ca4cc9c61764ee4/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2026-05-05T10:30:25.270Z","repository_id":348665199,"created_at":"2026-05-05T10:30:25.270Z","updated_at":"2026-05-05T10:30:25.270Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35330738,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-10T02:00:06.465Z","response_time":60,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-gateway","cloud-native","gitops","kubernetes","kubernetes-operator","litellm","llm","openai","openshift","operator"],"created_at":"2026-04-04T23:08:04.159Z","updated_at":"2026-07-10T12:00:39.106Z","avatar_url":"https://github.com/PalenaAI.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"docs/public/logo.png\" width=\"120\" alt=\"LiteLLM Operator\" /\u003e\n\u003c/p\u003e\n\n\u003ch1 align=\"center\"\u003eLiteLLM Operator\u003c/h1\u003e\n\n\u003cp align=\"center\"\u003e\n  Production-grade Kubernetes operator for \u003ca href=\"https://github.com/BerriAI/litellm\"\u003eLiteLLM\u003c/a\u003e — declarative AI gateway deployments, bidirectional config sync, and first-class OpenShift support.\n\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://github.com/PalenaAI/litellm-operator/releases\"\u003e\u003cimg src=\"https://img.shields.io/github/v/release/PalenaAI/litellm-operator?include_prereleases\u0026sort=semver\u0026label=release\" alt=\"Release\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/PalenaAI/litellm-operator/actions/workflows/ci.yml\"\u003e\u003cimg src=\"https://img.shields.io/github/actions/workflow/status/PalenaAI/litellm-operator/ci.yml?branch=main\u0026label=ci\" alt=\"CI\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/PalenaAI/litellm-operator/blob/main/LICENSE\"\u003e\u003cimg src=\"https://img.shields.io/github/license/PalenaAI/litellm-operator\" alt=\"License\" /\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/PalenaAI/litellm-operator\"\u003e\u003cimg src=\"https://img.shields.io/github/go-mod/go-version/PalenaAI/litellm-operator\" alt=\"Go version\" /\u003e\u003c/a\u003e\n  \u003cimg src=\"https://img.shields.io/badge/kubernetes-%E2%89%A5%201.28-blue\" alt=\"Kubernetes\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/openshift-supported-ee0000\" alt=\"OpenShift\" /\u003e\n  \u003cimg src=\"https://img.shields.io/badge/operator--sdk-v1.38+-5f87af\" alt=\"Operator SDK\" /\u003e\n\u003c/p\u003e\n\n---\n\n## Why this operator?\n\nThe community LiteLLM Helm chart deploys the proxy, but leaves you with a hard trade-off: manage models and keys through the **Admin UI** (convenient, but not GitOps-friendly) or through `proxy_server_config.yaml` (reproducible, but no UI). Pick one and you lose the other.\n\nThis operator dissolves that trade-off. Every resource — instances, organizations, models, teams, users, keys, customers, credentials, guardrails, budget tiers — is a first-class Kubernetes CRD, reconciled continuously against the LiteLLM REST API. Git is the source of truth; Admin-UI drift is detected on every sync interval and resolved per your policy (`crd-wins`, `api-wins`, or `manual`). You get GitOps **and** the Admin UI, backed by the same state.\n\nIt also handles the parts a Helm chart can't: finalizer-based cleanup that deletes upstream API objects, generated virtual keys stored as garbage-collected Kubernetes Secrets, enterprise license activation, rollback-on-failure, OpenShift-native routing, six-backend response caching, and external secret-manager integration so provider API keys never touch etcd.\n\n## Architecture at a glance\n\n```text\n                        kubectl apply\n                             │\n                             ▼\n ┌─────────────────────────────────────────────────────────────┐\n │                        Kubernetes API                       │\n │                                                             │\n │   LiteLLMInstance     LiteLLMOrganization   LiteLLMModel    │\n │   LiteLLMTeam         LiteLLMUser           LiteLLMCustomer │\n │   LiteLLMVirtualKey   LiteLLMCredential     LiteLLMGuardrail│\n │   LiteLLMBudget                                             │\n └──────────────────────────────┬──────────────────────────────┘\n                                │   watches / reconciles\n                                ▼\n                     ┌────────────────────┐\n                     │  LiteLLM Operator  │\n                     └─────────┬──────────┘\n         ┌───────────────────┬─┴─┬───────────────────────┐\n         │ Deployment        │   │  LiteLLM REST API     │\n         │ ConfigMap         │   │  (bidirectional sync) │\n         │ Secrets / HPA     │   │  crd-wins · api-wins  │\n         │ Ingress / Route / │   │  preserve · prune     │\n         │ HTTPRoute         │   │  adopt                │\n         │ ServiceMonitor    │   │                       │\n         │ PrometheusRule    │   │                       │\n         │ Grafana dashboard │   │                       │\n         └─────────┬─────────┘   └───────────┬───────────┘\n                   ▼                         ▼\n         ┌──────────────────────────────────────────┐\n         │  LiteLLM Proxy  +  Postgres  +  Redis    │\n         └──────────────────────────────────────────┘\n```\n\n## Features\n\n| Area | What you get |\n| --- | --- |\n| **Infrastructure** | Declarative Deployment, ConfigMap, Service, Secrets, HPA v2, PDB, NetworkPolicy; migration Jobs per image tag; auto-rollback on `ProgressDeadlineExceeded`; topology spread constraints; `runAsNonRoot` mode using the official non-root image |\n| **Networking** | Kubernetes Ingress, OpenShift Route, Gateway API HTTPRoute — pick one declaratively per instance |\n| **Multi-tenancy** | Full Organization → Team → User → Key hierarchy with budgets, member management (`crd` / `sso` / `mixed` modes), and org-scoped model access |\n| **API-managed CRDs** | `LiteLLMOrganization`, `LiteLLMModel`, `LiteLLMTeam`, `LiteLLMUser`, `LiteLLMCustomer`, `LiteLLMVirtualKey`, `LiteLLMBudget` — reconciled via the LiteLLM REST API with finalizer-based cleanup and spec-hash change detection |\n| **Config-managed CRDs** | `LiteLLMCredential` (reusable provider API keys via `credentialRef`) and `LiteLLMGuardrail` (Aporia, Lakera, Presidio, Bedrock, LLM Guard, Guardrails AI, Azure, Google Text Moderation, custom) — materialized into `proxy_server_config.yaml`, keys injected via `secretKeyRef` (never read into operator memory) |\n| **Bidirectional sync** | Periodic drift detection with `crd-wins` / `api-wins` / `manual` resolution and `preserve` / `prune` / `adopt` policies for unmanaged resources |\n| **VirtualKey lifecycle** | Generated API keys stored in owner-referenced Kubernetes Secrets; rotation and revocation follow CRD deletion |\n| **Authentication** | SSO for Azure Entra, Okta, Google, generic OIDC; SCIM v2 provisioning; JWT and OAuth2 auth for M2M flows; custom SSO handlers via ConfigMap or image |\n| **Security** | IP allowlisting with `X-Forwarded-For` support, RBAC via `spec.rbac`, external secret managers (AWS Secrets Manager / KMS, Azure Key Vault, Google Secret Manager / KMS, HashiCorp Vault) with IRSA and workload-identity support |\n| **Reliability** | 6-backend response caching (Redis / S3 / GCS / Qdrant / Redis-semantic / local), fallback chains (default, per-model, content-policy, context-window), per-error-type retry policies, tag-based routing, per-provider budget caps |\n| **Observability** | ServiceMonitor + PrometheusRule with 6 built-in alerts and runbook annotations; auto-provisioned Grafana dashboard ConfigMap |\n| **Data** | Optional CloudNativePG integration with `ScheduledBackup` (snapshot or `barmanObjectStore`) |\n| **Admin UI** | Disable, admin-only mode, DB-backed model management, personal-key gating, custom docs URL, logo, email branding, color themes via ConfigMap |\n| **Distribution** | OLM bundle for OperatorHub / OpenShift Catalog **and** a Helm chart for clusters without OLM |\n| **Enterprise** | Convention-based license Secret detection (`{instance}-license` or `litellm-license`) with `EnterpriseLicenseRequired` status conditions when unlicensed enterprise features are requested |\n\n\u003e **Full documentation:** see the [`docs/`](docs/) folder — guides for [SSO](docs/guide/sso.md), [config sync](docs/guide/config-sync.md), [caching](docs/guide/caching.md), [RBAC](docs/guide/rbac.md), [observability](docs/guide/observability.md), [secret managers](docs/guide/secret-managers.md), and per-CRD reference pages under [`docs/reference/`](docs/reference/).\n\n## Custom Resource Definitions\n\n| CRD | Short Name | Description |\n| --- | ---------- | ----------- |\n| `LiteLLMInstance` | `li` | Deploys a LiteLLM proxy with database, Redis, networking, and SSO |\n| `LiteLLMOrganization` | `lo` | Creates an organization for multi-tenant isolation with budget and model access |\n| `LiteLLMModel` | `lm` | Registers a model (e.g., `openai/gpt-4o`) with the proxy |\n| `LiteLLMTeam` | `lt` | Creates a team with budget limits and member management |\n| `LiteLLMUser` | `lu` | Creates a user (service accounts, bot users, non-SSO environments) |\n| `LiteLLMCustomer` | `lcust` | Manages an external end-user (SaaS customer) with budgets and rate limits |\n| `LiteLLMCredential` | `lc` | Defines a reusable provider credential (API key + optional base URL) shared across models |\n| `LiteLLMGuardrail` | `lg` | Defines a content moderation / safety integration (Aporia, Lakera, Presidio, Bedrock, etc.) |\n| `LiteLLMVirtualKey` | `lk` | Generates an API key scoped to a team/user with budget and rate limits |\n| `LiteLLMBudget` | `lb` | Declares a reusable budget / rate-limit tier (via `/budget/*`) referenced by `budgetId` from keys, customers, and the instance default |\n\nAll secondary resources reference a `LiteLLMInstance` via `spec.instanceRef`. Teams can optionally reference a `LiteLLMOrganization` via `spec.organizationRef`.\n\n## Prerequisites\n\n- Go 1.22+\n- Docker 17.03+\n- kubectl v1.28+\n- Access to a Kubernetes v1.28+ cluster\n- A PostgreSQL database for LiteLLM state storage\n\n## Quick Start\n\n### 1. Install CRDs\n\n```sh\nmake install\n```\n\n### 2. Deploy the operator\n\n```sh\nmake deploy IMG=ghcr.io/palenaai/litellm-operator:latest\n```\n\n### 3. Create a database secret\n\n```sh\nkubectl create secret generic litellm-db-credentials \\\n  --from-literal=DATABASE_URL='postgresql://user:pass@host:5432/litellm'\n```\n\n### 4. Deploy a LiteLLM instance\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  replicas: 2\n  masterKey:\n    autoGenerate: true\n  database:\n    external:\n      connectionSecretRef:\n        name: litellm-db-credentials\n        key: DATABASE_URL\n  service:\n    type: ClusterIP\n    port: 4000\n```\n\n### 5. Register a model\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMModel\nmetadata:\n  name: gpt4o\nspec:\n  instanceRef:\n    name: my-gateway\n  modelName: gpt-4o\n  litellmParams:\n    model: openai/gpt-4o\n    apiKeySecretRef:\n      name: openai-credentials\n      key: OPENAI_API_KEY\n```\n\n### Reusable Credentials\n\nWhen many models share the same provider API key (e.g., several OpenAI deployments), define the credential once and reference it from each `LiteLLMModel` via `credentialRef`:\n\n```yaml\napiVersion: v1\nkind: Secret\nmetadata:\n  name: openai-credentials\ntype: Opaque\nstringData:\n  api-key: sk-...\n---\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMCredential\nmetadata:\n  name: openai-prod\nspec:\n  instanceRef:\n    name: my-gateway\n  # The name used under `credential_list` in the generated proxy config.\n  # Models reference this via `litellm_params.litellm_credential_name`.\n  credentialName: openai-prod\n  apiKeySecretRef:\n    name: openai-credentials\n    key: api-key\n  # Optional extras merged into credential_info (api_base / api_version /\n  # free-form params are supported — params cannot override reserved keys).\n  apiBase: https://api.openai.com/v1\n---\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMModel\nmetadata:\n  name: gpt4o\nspec:\n  instanceRef:\n    name: my-gateway\n  modelName: gpt-4o\n  litellmParams:\n    model: openai/gpt-4o\n    credentialRef:\n      name: openai-prod   # takes precedence over inline apiKeySecretRef/apiBase\n```\n\nThe operator injects the API key into the proxy pod via a `secretKeyRef`-backed env var (`CREDENTIAL_OPENAI_PROD_API_KEY`) and writes a matching `os.environ/…` reference to the generated `proxy_server_config.yaml` — the key value itself is never read into the operator's memory. Rotating the key is a Secret update: the operator rolls out a new Deployment pod to pick up the new value.\n\n### 6. Create a team and API key\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMTeam\nmetadata:\n  name: engineering\nspec:\n  instanceRef:\n    name: my-gateway\n  teamAlias: engineering\n  models: [gpt-4o]\n  maxBudgetMonthly: 1000\n  budgetDuration: \"30d\"\n  members:\n    - email: dev@example.com\n      role: user\n---\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMVirtualKey\nmetadata:\n  name: eng-ci-key\nspec:\n  instanceRef:\n    name: my-gateway\n  keyAlias: eng-ci-key\n  teamRef:\n    name: engineering\n  models: [gpt-4o]\n  maxBudget: \"100\"\n```\n\n### Multi-Tenant Organizations\n\nCreate an organization to group teams under a shared budget and model access policy:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMOrganization\nmetadata:\n  name: acme-corp\nspec:\n  instanceRef:\n    name: my-gateway\n  organizationAlias: acme-corp\n  models: [gpt-4o, claude-4-sonnet]\n  maxBudget: 5000\n  budgetDuration: \"30d\"\n  members:\n    - email: admin@acme.com\n      role: org_admin\n    - email: user@acme.com\n      role: internal_user\n---\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMTeam\nmetadata:\n  name: acme-engineering\nspec:\n  instanceRef:\n    name: my-gateway\n  organizationRef:\n    name: acme-corp\n  teamAlias: acme-engineering\n  models: [gpt-4o]\n```\n\n### OpenShift / Non-Root Environments\n\nFor OpenShift or clusters enforcing Pod Security Standards, enable non-root mode:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  security:\n    runAsNonRoot: true\n  # ... rest of spec\n```\n\nThis automatically switches to the official `litellm-non_root` image (runs as `nobody`, UID 65534) and applies a restricted pod security context compatible with OpenShift's restricted SCC.\n\n### IP Allowlisting (Enterprise)\n\nRestrict API access to specific IP addresses or CIDR ranges:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  security:\n    ipAllowlist:\n      enabled: true\n      allowedIPs:\n        - \"10.0.0.0/8\"\n        - \"192.168.1.0/24\"\n        - \"203.0.113.50\"\n      useXForwardedFor: true  # required behind load balancers\n  # ... rest of spec\n```\n\n### RBAC (Role-Based Access Control)\n\nEnforce route restrictions and key generation controls:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  rbac:\n    enabled: true\n    adminOnlyRoutes:\n      - /model/new\n      - /model/delete\n    allowedRoutes:\n      - /chat/completions\n      - /embeddings\n      - /key/info\n    defaultTeamDisabled: true   # force team-based keys\n    keyGeneration:              # enterprise\n      teamKeyGeneration:\n        allowedTeamMemberRoles: [\"admin\"]\n    rolePermissions:            # enterprise\n      internal_user:\n        routes: [\"/key/generate\", \"/key/info\"]\n        models: [\"gpt-4\", \"claude-3-haiku\"]\n  # ... rest of spec\n```\n\n### OpenShift Route\n\nFor OpenShift clusters, create a Route instead of an Ingress:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  route:\n    enabled: true\n    host: litellm.apps.example.com\n    tlsTermination: edge   # edge | passthrough | reencrypt\n  # ... rest of spec\n```\n\n### Gateway API HTTPRoute\n\nFor clusters using the [Gateway API](https://gateway-api.sigs.k8s.io/) (Istio, Envoy Gateway, Cilium, etc.):\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  gatewayHTTPRoute:\n    enabled: true\n    host: litellm.example.com\n    parentRefs:\n      - name: my-gateway       # Name of the Gateway resource\n        namespace: istio-system # Optional: namespace of the Gateway\n        sectionName: https     # Optional: specific listener on the Gateway\n  # ... rest of spec\n```\n\n### Observability (Prometheus + Grafana)\n\nEnable ServiceMonitor, alerting rules, and a Grafana dashboard:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  observability:\n    serviceMonitor:\n      enabled: true\n      interval: \"30s\"\n    prometheusRule:\n      enabled: true\n      # disabledAlerts: [\"LiteLLMHighCPUUsage\"]  # optionally disable specific alerts\n    grafanaDashboard:\n      enabled: true\n      folder: \"LiteLLM\"\n  # ... rest of spec\n```\n\nBuilt-in alerts: `LiteLLMInstanceDown` (critical), `LiteLLMInstanceDegraded`, `LiteLLMPodRestarting`, `LiteLLMPodNotReady`, `LiteLLMHighMemoryUsage`, `LiteLLMHighCPUUsage`. Each alert includes a runbook annotation with troubleshooting commands.\n\n### CloudNativePG Backups\n\nWhen using CloudNativePG for the database, enable scheduled backups:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  database:\n    cloudnativepg:\n      clusterName: litellm-db\n      backup:\n        enabled: true\n        schedule: \"0 2 * * *\"   # daily at 2am\n        retention: 7\n        method: snapshot        # snapshot or barmanObjectStore\n  # ... rest of spec\n```\n\n### Tag-Based Routing\n\nRoute requests to different model deployments based on tags. Useful for free/paid tiers or team-specific model access:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  routerSettings:\n    enableTagFiltering: true\n  # ... rest of spec\n---\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMModel\nmetadata:\n  name: gpt4-paid\nspec:\n  instanceRef:\n    name: my-gateway\n  modelName: gpt-4\n  litellmParams:\n    model: openai/gpt-4\n    apiKeySecretRef:\n      name: openai-credentials\n      key: OPENAI_API_KEY\n  tags: [\"paid\"]\n---\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMTeam\nmetadata:\n  name: paid-tier\nspec:\n  instanceRef:\n    name: my-gateway\n  teamAlias: paid-tier\n  tags: [\"paid\"]\n```\n\nRequests from the `paid-tier` team are routed to model deployments tagged `paid`. Use `tagFilteringMatchAny: true` in `routerSettings` to match requests having ANY of the specified tags (default is ALL must match).\n\n### Fallback Chains\n\nConfigure model fallback routing so requests automatically try alternative models on failure:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  fallbacks:\n    # Global fallbacks applied on any error\n    defaultFallbacks: [\"gpt-4-mini\", \"claude-3-haiku\"]\n\n    # Per-model fallbacks for general errors\n    modelFallbacks:\n      - model: gpt-4\n        fallbacks: [\"gpt-4-mini\", \"claude-3-haiku\"]\n\n    # Fallbacks for content policy violations\n    contentPolicyFallbacks:\n      - model: gpt-4\n        fallbacks: [\"claude-3-sonnet\"]\n\n    # Fallbacks for context window exceeded\n    contextWindowFallbacks:\n      - model: gpt-4\n        fallbacks: [\"gpt-4-32k\", \"claude-3-sonnet\"]\n\n    maxFallbacks: 3\n\n  routerSettings:\n    # Retry policy by error type (retries on same model before fallback)\n    retryPolicy:\n      TimeoutError: 2\n      RateLimitError: 3\n      ContentPolicyViolationError: 0\n    # Per-model-group retry overrides\n    modelGroupRetryPolicy:\n      gpt-4:\n        TimeoutError: 1\n        RateLimitError: 0\n  # ... rest of spec\n```\n\n### Response Caching\n\nConfigure response caching to reduce latency and costs:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  caching:\n    enabled: true\n    type: redis             # redis, redis-semantic, s3, gcs, qdrant, local\n    ttl: 600                # cache TTL in seconds\n    namespace: \"my-app\"     # key isolation namespace\n    mode: default_on        # default_on or default_off\n    supportedCallTypes:     # restrict to specific call types\n      - acompletion\n      - aembedding\n    redis:\n      host: redis.example.com\n      port: 6379\n      passwordSecretRef:\n        name: redis-secret\n        key: password\n      ssl: true\n  # ... rest of spec\n```\n\nWhen `type: redis` and no `caching.redis` block is provided, the operator reuses the instance's existing `spec.redis` connection — no need to duplicate Redis details.\n\nOther backends: `s3` (with bucket, region, AWS credentials), `gcs` (with bucket, GCS service account), `qdrant` (semantic caching with embeddings), `local` (in-memory, no external dependencies).\n\n### Auto-Rollback\n\nAutomatically rollback failed deployments:\n\n```yaml\napiVersion: litellm.palena.ai/v1alpha1\nkind: LiteLLMInstance\nmetadata:\n  name: my-gateway\nspec:\n  upgrade:\n    strategy: rolling\n    autoRollback: true\n    healthCheckTimeout: \"300s\"\n  # ... rest of spec\n```\n\nWhen enabled, the operator tracks the last successful deployment revision. If a new deployment exceeds the progress deadline, the operator triggers a rollback and sets a status condition.\n\n### Enterprise License\n\nTo activate LiteLLM Enterprise features, create a Secret with your license key. The operator detects it automatically and injects the `LITELLM_LICENSE` environment variable into the proxy Deployment.\n\n**Per-instance license** (takes precedence):\n\n```sh\nkubectl create secret generic my-gateway-license \\\n  --from-literal=license-key='your-litellm-enterprise-license-key'\n```\n\n**Namespace-wide license** (fallback for all instances in the namespace):\n\n```sh\nkubectl create secret generic litellm-license \\\n  --from-literal=license-key='your-litellm-enterprise-license-key'\n```\n\nThe operator checks for `{instance-name}-license` first, then falls back to `litellm-license`. License status is reported in `.status.license`:\n\n```sh\nkubectl get litellminstance my-gateway -o jsonpath='{.status.license}'\n# {\"active\":true,\"secretName\":\"my-gateway-license\"}\n```\n\nIf a downstream resource (Model, Team, User, VirtualKey) requires an enterprise feature and no license is present, the operator sets `Reason: EnterpriseLicenseRequired` on the resource's status condition without retrying.\n\n### Namespace-Scoped Watching\n\nBy default, the operator watches all namespaces. To restrict it to specific namespaces:\n\n**Helm:**\n\n```bash\nhelm install litellm-operator deploy/charts/litellm-operator/ \\\n  --set watchNamespaces=\"team-a,team-b\"\n```\n\n**Flag:**\n\n```bash\n/manager --watch-namespaces=team-a,team-b\n```\n\n**Environment variable (set automatically by OLM for OwnNamespace/SingleNamespace install modes):**\n\n```bash\nWATCH_NAMESPACE=team-a,team-b\n```\n\n### 7. Retrieve a generated API key\n\nThe generated API key is stored in a Secret (default name: `{name}-key`):\n\n```sh\nkubectl get secret eng-ci-key-key -o jsonpath='{.data.api_key}' | base64 -d\n```\n\n## Installation Methods\n\n### Direct (Makefile)\n\n```sh\nmake install       # Install CRDs\nmake deploy        # Deploy operator\n```\n\n### OLM (OpenShift / clusters with OLM)\n\n```sh\noperator-sdk run bundle ghcr.io/palenaai/litellm-operator-bundle:latest\n```\n\n### Helm\n\n```sh\nhelm install litellm-operator deploy/charts/litellm-operator/\n```\n\n## Development\n\n### Build\n\n```sh\nmake build                    # Build operator binary\nmake docker-build IMG=...     # Build container image\n```\n\n### Test\n\n```sh\nmake test          # Unit + integration tests (envtest)\nmake test-e2e      # End-to-end tests (requires cluster)\n```\n\n### Generate\n\n```sh\nmake generate      # DeepCopy functions\nmake manifests     # CRD YAMLs, RBAC, webhooks\n```\n\n### Run locally (against current kubeconfig cluster)\n\n```sh\nmake install       # Install CRDs first\nmake run           # Run operator outside the cluster\n```\n\n## Architecture\n\nKey design points:\n\n- **LiteLLMInstance** controller manages Deployment, ConfigMap, Service, Secrets, Ingress, HPA, PDB, NetworkPolicy, migration Jobs, ServiceMonitor, PrometheusRule, Grafana dashboard ConfigMaps, and CNPG ScheduledBackups\n- **Secondary controllers** (Organization, Model, Team, User, VirtualKey) resolve their `instanceRef` to discover the LiteLLM API endpoint and master key, then sync state via the REST API\n- **Finalizers** ensure cleanup: deleting a CRD calls the corresponding LiteLLM API delete endpoint before removing the Kubernetes resource\n- **Spec hash annotations** (`litellm.palena.ai/sync-hash`) enable change detection to avoid unnecessary API calls\n\n## Project Structure\n\n```text\napi/v1alpha1/          CRD type definitions\ninternal/controller/   Reconciliation controllers\ninternal/litellm/      LiteLLM REST API client\ninternal/resources/    Kubernetes resource generators\nconfig/crd/bases/      Generated CRD manifests\nconfig/samples/        Example custom resources\nbundle/                OLM bundle manifests\ndeploy/charts/         Helm chart\n```\n\n## License\n\nCopyright 2026. Licensed under the Apache License, Version 2.0.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpalenaai%2Flitellm-operator","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpalenaai%2Flitellm-operator","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpalenaai%2Flitellm-operator/lists"}