{"id":34524866,"url":"https://github.com/dpguthrie/litellm-gateway","last_synced_at":"2026-05-28T22:31:37.669Z","repository":{"id":328196036,"uuid":"1114550480","full_name":"dpguthrie/litellm-gateway","owner":"dpguthrie","description":null,"archived":false,"fork":false,"pushed_at":"2025-12-11T15:51:38.000Z","size":7,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-25T16:28:46.764Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dpguthrie.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-11T14:34:01.000Z","updated_at":"2025-12-11T15:51:41.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/dpguthrie/litellm-gateway","commit_stats":null,"previous_names":["dpguthrie/litellm-gateway"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/dpguthrie/litellm-gateway","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpguthrie%2Flitellm-gateway","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpguthrie%2Flitellm-gateway/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpguthrie%2Flitellm-gateway/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpguthrie%2Flitellm-gateway/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dpguthrie","download_url":"https://codeload.github.com/dpguthrie/litellm-gateway/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dpguthrie%2Flitellm-gateway/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33629560,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-05-28T02:00:06.440Z","response_time":99,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-12-24T05:07:00.042Z","updated_at":"2026-05-28T22:31:37.651Z","avatar_url":"https://github.com/dpguthrie.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# LiteLLM with Custom Email-Based Auth\n\nLiteLLM with custom authentication that validates the `x-user-email` header using LiteLLM's built-in `custom_auth` feature.\n\n## Architecture\n\n```\nClient Request → LiteLLM (port 4000) → LLM Providers\n                    ↓\n              Custom Auth Hook (custom_auth.py)\n              Checks x-user-email header\n```\n\nThis approach extends LiteLLM directly without needing a separate gateway/proxy.\n\n## How It Works\n\n1. **custom_auth.py** - Python module with auth logic\n2. **litellm_config.yaml** - References custom auth function\n3. **Docker mount** - Custom module mounted into container\n4. **Environment variables** - API keys loaded from .env file\n\nLiteLLM automatically calls the custom auth function before processing each request.\n\n## Setup\n\n### 1. Configure API Keys\n\nCopy the example env file and add your API keys:\n\n```bash\ncp .env.example .env\n```\n\nThen edit `.env` with your actual API keys:\n\n```bash\nOPENAI_API_KEY=sk-your-openai-key-here\nANTHROPIC_API_KEY=sk-ant-your-anthropic-key-here\nGROQ_API_KEY=gsk-your-groq-key-here\n```\n\n**Important:** The `.env` file is gitignored and will never be committed to version control.\n\n### 2. Run LiteLLM\n\n```bash\n# Start LiteLLM with custom auth\nmake run\n```\n\nThat's it! The docker container will:\n- Mount `litellm_config.yaml` (includes custom_auth config)\n- Mount `custom_auth.py` (your auth logic)\n- Start on port 4000 with auth enabled\n\n## Usage\n\n### ✅ Successful Request (with x-user-email header)\n```bash\nmake curl\n```\n\nOr manually:\n```bash\ncurl -X POST 'http://localhost:4000/chat/completions' \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer sk-1234567890' \\\n  -H 'x-user-email: user@example.com' \\\n  -d '{\"model\": \"gpt-4o\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n```\n\n### ❌ Failed Request (missing x-user-email header)\n```bash\nmake curl-noauth\n```\n\nReturns 401: `Missing x-user-email header`\n\n## Exposing Locally with ngrok\n\nTo expose your local LiteLLM server publicly (for testing from external services, sharing with teammates, etc.), use ngrok:\n\n### 1. Install ngrok\n\nIf not already installed:\n```bash\n# macOS\nbrew install ngrok\n\n# Or download from https://ngrok.com/download\n```\n\n### 2. Authenticate ngrok\n\n```bash\nngrok config add-authtoken YOUR_NGROK_TOKEN\n```\n\nGet your auth token from: https://dashboard.ngrok.com/get-started/your-authtoken\n\n### 3. Start LiteLLM (in one terminal)\n\n```bash\nmake run\n```\n\n### 4. Start ngrok tunnel (in another terminal)\n\n```bash\nmake ngrok\n```\n\nOr directly:\n```bash\nngrok http 4000\n```\n\nYou'll see output like:\n```\nForwarding   https://abc123.ngrok.io -\u003e http://localhost:4000\n```\n\n### 5. Use the ngrok URL\n\nNow you can make requests to your public ngrok URL:\n\n```bash\ncurl -X POST 'https://abc123.ngrok.io/chat/completions' \\\n  -H 'Content-Type: application/json' \\\n  -H 'Authorization: Bearer sk-1234567890' \\\n  -H 'x-user-email: user@example.com' \\\n  -d '{\"model\": \"gpt-4o\", \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n```\n\n**Bonus:** Add a custom subdomain (requires ngrok paid plan):\n```bash\nngrok http 4000 --domain=your-custom-domain.ngrok-free.app\n```\n\n## Using as a Custom Provider in Braintrust\n\nOnce your LiteLLM gateway is running (via ngrok), you can add it as a custom provider in Braintrust for evaluations, experiments, and observability.\n\n### Why Use This with Braintrust?\n\n- **Unified observability** - Track requests across multiple LLM providers through a single gateway\n- **Custom auth** - Enforce your x-user-email authentication in Braintrust experiments\n- **Cost tracking** - Monitor usage and costs across OpenAI, Anthropic, Groq, etc. in one place\n- **A/B testing** - Compare different models and providers in Braintrust experiments\n\n### Setup in Braintrust\n\n1. **Go to Braintrust Settings** → AI Providers → Add Custom Provider\n\n2. **Configure the provider:**\n\n   **For local testing:**\n   - **Name:** LiteLLM Gateway (Local)\n   - **Base URL:** `http://ngrok-url:4000`\n   - **API Key:** `sk-1234567890` (or any value - matches Authorization header)\n\n   **For remote/production (with ngrok):**\n   - **Name:** LiteLLM Gateway\n   - **Base URL:** `https://YOUR-NGROK-URL.ngrok.io`\n   - **API Key:** `sk-1234567890`\n\n3. **Add custom headers:**\n   - Header: `x-user-email`\n   - Value: `your-email@example.com` (required by the auth hook)\n\n4. **Test the connection** in Braintrust's Playground\n\n\n### Available Models\n\nThe models configured in this gateway:\n- `gpt-4o` - OpenAI GPT-4o\n- `gpt-4o-mini` - OpenAI GPT-4o Mini\n- `claude-4` - Anthropic Claude Sonnet 4\n- `llama-3.3-70b` - Groq Llama 3.3 70B\n\nLearn more: [Braintrust Custom Providers Documentation](https://www.braintrust.dev/docs/integrations/ai-providers/custom)\n\n## Customizing Auth Logic\n\nEdit `custom_auth.py` and modify the `is_email_allowed()` function:\n\n```python\ndef is_email_allowed(email: str) -\u003e bool:\n    # Example: only allow specific domains\n    allowed_domains = [\"company.com\", \"partner.com\"]\n    domain = email.split(\"@\")[1]\n    return domain in allowed_domains\n\n    # Or check against a database, API, etc.\n```\n\n## What Gets Logged\n\nThe auth hook logs to stdout:\n- All incoming request headers\n- The `x-user-email` value\n- Auth success/failure for each request\n\nView logs in the terminal running `make run`.\n\n## Key Files\n\n- **custom_auth.py** - Your custom auth logic (mounted into container)\n- **litellm_config.yaml** - LiteLLM config with `general_settings.custom_auth` and env var references\n- **.env** - Your API keys (gitignored, never committed)\n- **.env.example** - Template for required environment variables\n- **Makefile** - Docker run command with volume mounts and env file\n- **.gitignore** - Ensures .env is never committed to git\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdpguthrie%2Flitellm-gateway","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdpguthrie%2Flitellm-gateway","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdpguthrie%2Flitellm-gateway/lists"}