{"id":35203898,"url":"https://github.com/ruslanmv/ollabridge","last_synced_at":"2026-01-13T21:40:48.660Z","repository":{"id":330960797,"uuid":"1124373349","full_name":"ruslanmv/ollabridge","owner":"ruslanmv","description":"OllaBridge transforms your laptop or workstation into a production-grade, OpenAI-compatible LLM provider. It features self-healing setup, built-in security, and automatic tunneling—making your local hardware ready for real applications in 60 seconds.","archived":false,"fork":false,"pushed_at":"2026-01-08T16:41:44.000Z","size":526,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-01-09T01:46:09.806Z","etag":null,"topics":["fastapi","llm-gateway","local-ai","mcp-server","ollama","openai-api","self-hosted"],"latest_commit_sha":null,"homepage":"https://ollabridge.com/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ruslanmv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-28T22:52:05.000Z","updated_at":"2026-01-08T13:30:33.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ruslanmv/ollabridge","commit_stats":null,"previous_names":["ruslanmv/ollabridge"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/ruslanmv/ollabridge","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruslanmv%2Follabridge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruslanmv%2Follabridge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruslanmv%2Follabridge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruslanmv%2Follabridge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ruslanmv","download_url":"https://codeload.github.com/ruslanmv/ollabridge/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ruslanmv%2Follabridge/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28401044,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-13T14:36:09.778Z","status":"ssl_error","status_checked_at":"2026-01-13T14:35:19.697Z","response_time":56,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fastapi","llm-gateway","local-ai","mcp-server","ollama","openai-api","self-hosted"],"created_at":"2025-12-29T13:04:12.200Z","updated_at":"2026-01-13T21:40:48.654Z","avatar_url":"https://github.com/ruslanmv.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n\u003cimg src=\"assets/logo.svg\" alt=\"OllaBridge Logo\" width=\"500\"/\u003e\n\n# OllaBridge ⚡️\n\n**Your single gateway to ALL your LLMs — local, remote, anywhere.**\n\n[![PyPI version](https://badge.fury.io/py/ollabridge.svg)](https://badge.fury.io/py/ollabridge)\n[![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n[Quick Start](#-60-second-start) • [Why OllaBridge](#-why-ollabridge) • [Distributed Compute](#-add-any-gpu-in-60-seconds) • [Examples](#-use-it-anywhere) • [Demo Client](#-try-the-interactive-demo-client) • [MCP Mode](#-ai-agents-love-ollabridge)\n\n\u003c/div\u003e\n\n---\n\n## 🎯 What is OllaBridge?\n\n\u003e **One gateway. All your LLMs. Everywhere.**\n\nOllaBridge is your **single, OpenAI-compatible API** for every LLM you run — on your laptop, workstation, free GPU servers, cloud instances, anywhere.\n\n**The Problem:** You have models running everywhere (laptop, cloud GPU, friend's gaming PC), and every app needs different configs.\n\n**OllaBridge Solution:** Apps connect to ONE place. OllaBridge routes to the right compute automatically.\n\n```mermaid\ngraph TB\n    A[Your Apps] --\u003e|OpenAI SDK| B[OllaBridge\u003cbr/\u003eControl Plane]\n\n    B --\u003e|Auto Routes| C[Local Laptop\u003cbr/\u003ellama3.1]\n    B --\u003e|Auto Routes| D[Free GPU Cloud\u003cbr/\u003edeepseek-r1]\n    B --\u003e|Auto Routes| E[Remote Workstation\u003cbr/\u003emixtral]\n\n    C -.-\u003e|Dials Out| B\n    D -.-\u003e|Dials Out| B\n    E -.-\u003e|Dials Out| B\n\n    style B fill:#6366f1,stroke:#4f46e5,stroke-width:3px,color:#fff\n    style A fill:#10b981,stroke:#059669,stroke-width:2px,color:#fff\n    style C fill:#8b5cf6,stroke:#7c3aed,stroke-width:2px,color:#fff\n    style D fill:#ec4899,stroke:#db2777,stroke-width:2px,color:#fff\n    style E fill:#f59e0b,stroke:#d97706,stroke-width:2px,color:#fff\n```\n\n**Key Innovation:** Compute nodes **dial out** to your gateway. No port forwarding, no VPN, no config hell.\n\n---\n\n## 🚀 Why OllaBridge?\n\n### 🎯 **Single Source of Truth**\n- ✅ **One URL for everything** — Your apps never change code\n- ✅ **Zero config** — Add new GPUs without touching your app\n- ✅ **Smart routing** — OllaBridge picks the best node automatically\n- ✅ **OpenAI compatible** — Works with any SDK, framework, or tool\n\n### 🛡️ **Enterprise-Grade Security**\n- ✅ **API key authentication** — Protect your LLMs\n- ✅ **Rate limiting** — Control usage per key\n- ✅ **Request logging** — Full audit trail\n- ✅ **Encrypted connections** — TLS for remote nodes\n\n### 🌍 **Works Everywhere**\n- ✅ **Free GPU clouds** — Colab, Kaggle, Lightning AI (no port forwarding needed!)\n- ✅ **Ephemeral instances** — Nodes dial out, IPs don't matter\n- ✅ **Behind firewalls** — Your laptop can join from coffee shop WiFi\n- ✅ **Mixed environments** — Combine local + cloud seamlessly\n\n### 🤖 **AI Agent Ready**\n- ✅ **MCP server** — Agents can control your infrastructure\n- ✅ **Tool exposure** — Manage nodes, routes, health via tools\n- ✅ **Self-healing** — Auto-install, auto-configure, auto-recover\n\n---\n\n## ⚡ 60-Second Start\n\n### Step 1: Install\n\n```bash\npip install ollabridge\n```\n\n### Step 2: Start Your Gateway\n\n```bash\nollabridge start\n```\n\n**That's it!** You'll see:\n\n```\n✅ Ollama installed (if needed)\n✅ Model downloaded (if needed)\n✅ Gateway online at http://localhost:11435\n\n╭─────────────────── 🚀 Gateway Ready ────────────────────╮\n│                                                          │\n│ ✅ OllaBridge is Online                                  │\n│                                                          │\n│ Model:        deepseek-r1                                │\n│ Local API:    http://localhost:11435/v1                 │\n│ Key:          sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA        │\n│                                                          │\n│ Node join token:  eyJ0eXAi...                           │\n│ Example node command:                                    │\n│   ollabridge-node join --control http://localhost:11435 │\n│                        --token eyJ0eXAi...              │\n│                                                          │\n╰──────────────────────────────────────────────────────────╯\n```\n\n### Step 3: Use It!\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:11435/v1\",\n    api_key=\"sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA\"\n)\n\nresponse = client.chat.completions.create(\n    model=\"deepseek-r1\",\n    messages=[{\"role\": \"user\", \"content\": \"Hello!\"}]\n)\n\nprint(response.choices[0].message.content)\n```\n\n**Done!** You're running private LLMs with the OpenAI API.\n\n---\n\n## 🌍 Add Any GPU in 60 Seconds\n\nHave a free GPU on Colab? A remote workstation? Add it instantly:\n\n### On Your Remote GPU/Machine:\n\n```bash\n# Install\npip install ollabridge\n\n# Join your gateway (copy the command from gateway startup)\nollabridge-node join \\\n  --control http://YOUR_GATEWAY_IP:11435 \\\n  --token eyJ0eXAi...\n```\n\n**That's it!** The remote GPU:\n- ✅ Auto-installs Ollama if needed\n- ✅ Auto-downloads models if needed\n- ✅ **Dials out** to your gateway (no port forwarding!)\n- ✅ Shows up as available compute\n\n### Your Apps See It Automatically\n\n```python\n# Same code, now uses both local + remote GPU!\nclient = OpenAI(base_url=\"http://localhost:11435/v1\", ...)\nresponse = client.chat.completions.create(...)  # Auto-routed\n```\n\n**OllaBridge routes requests** across all your nodes automatically.\n\n---\n\n## 🎯 Real-World Scenarios\n\n### Scenario 1: \"I have a gaming PC at home\"\n\n```bash\n# On your gaming PC:\nollabridge-node join --control https://your-gateway.com --token ...\n\n# Now your laptop can use your gaming PC's GPU\n# Even if you're at a coffee shop!\n```\n\n### Scenario 2: \"I want to use free Colab GPUs\"\n\n```python\n# In Colab notebook:\n!pip install ollabridge\n!ollabridge-node join --control https://your-gateway.com --token ...\n\n# Now your production app can use free Colab compute\n# Colab session ends? Start a new one. Zero config changes.\n```\n\n### Scenario 3: \"I have multiple cloud GPUs\"\n\n```bash\n# Each GPU instance:\nollabridge-node join --control https://gateway.company.com --token ...\n\n# Your team shares one API URL\n# OllaBridge load-balances across all GPUs\n```\n\n---\n\n## 💻 Use It Anywhere\n\n### Python (OpenAI SDK)\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:11435/v1\",\n    api_key=\"your-key-here\"\n)\n\n# Chat\nresponse = client.chat.completions.create(\n    model=\"deepseek-r1\",\n    messages=[{\"role\": \"user\", \"content\": \"Explain quantum computing\"}]\n)\n\n# Embeddings\nembeddings = client.embeddings.create(\n    model=\"nomic-embed-text\",\n    input=\"Hello, world!\"\n)\n```\n\n### Node.js / TypeScript\n\n```typescript\nimport OpenAI from \"openai\";\n\nconst client = new OpenAI({\n  baseURL: \"http://localhost:11435/v1\",\n  apiKey: process.env.OLLABRIDGE_KEY\n});\n\nconst completion = await client.chat.completions.create({\n  model: \"deepseek-r1\",\n  messages: [{ role: \"user\", content: \"Hello!\" }]\n});\n```\n\n### LangChain\n\n```python\nfrom langchain_openai import ChatOpenAI\n\nllm = ChatOpenAI(\n    base_url=\"http://localhost:11435/v1\",\n    api_key=\"your-key-here\",\n    model=\"deepseek-r1\"\n)\n\nresponse = llm.invoke(\"What is the meaning of life?\")\n```\n\n### cURL\n\n```bash\ncurl -X POST http://localhost:11435/v1/chat/completions \\\n  -H \"Authorization: Bearer your-key-here\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"deepseek-r1\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n  }'\n```\n\n**Works with ANY OpenAI-compatible tool or library.**\n\n---\n\n## 🎨 Try the Interactive Demo Client\n\nWant to see OllaBridge in action? Check out our **2-click demo client** in the `example/` folder!\n\n### ⚡ Quick Start\n\n```bash\n# 1. Install and start OllaBridge\ncd example\n./install-ollabridge.sh  # Mac/Linux\n# or\n.\\install-ollabridge.ps1  # Windows\n\n# 2. Run the demo client\nmake run\n\n# 3. Open http://localhost:3000 in your browser\n```\n\n### ✨ Features\n\n- 🎯 **Beautiful UI** — Modern, responsive web interface\n- 🔌 **Real Integration** — Actual API calls to OllaBridge endpoints\n- 📊 **Live Metrics** — Request stats, latency, uptime tracking\n- 🔑 **Auth Demo** — See how API key authentication works\n- 📝 **Best Practices** — Production-ready code examples\n\n### 📚 Perfect for Learning\n\nThe example client shows you:\n- ✅ How to connect to OllaBridge from a browser\n- ✅ How to handle CORS properly\n- ✅ How to implement authentication with API keys\n- ✅ How to load models dynamically\n- ✅ How to send chat requests and handle responses\n\n**[View Full Documentation →](example/README.md)**\n\n---\n\n## 🤖 AI Agents Love OllaBridge\n\nOllaBridge has a **Model Context Protocol (MCP) server** built-in.\n\nAgents can:\n- ✅ Create enrollment tokens\n- ✅ List connected compute nodes\n- ✅ Check gateway health\n- ✅ Manage your LLM infrastructure via tools\n\n### Start MCP Server\n\n```bash\nollabridge-mcp\n```\n\n### Example: Agent Workflow\n\n```python\n# Agent can call these tools:\nawait session.call_tool(\"ollabridge.enroll.create\", {})\n# → Returns enrollment token\n\nawait session.call_tool(\"ollabridge.runtimes.list\", {})\n# → Shows all connected nodes\n\nawait session.call_tool(\"ollabridge.gateway.health\", {})\n# → Checks gateway status\n```\n\n**Use Case:** \"Hey Claude, add my workstation's GPU to our LLM gateway\"\n\n→ Agent creates token, gives you the command, you run it. Done.\n\n---\n\n## 🔐 Security \u0026 Configuration\n\n### Authentication\n\nOllaBridge auto-generates a secure API key on first run (saved in `.env`):\n\n```env\nAPI_KEYS=sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA\n```\n\nUse it in your apps:\n\n```python\n# Option 1: Bearer token\nheaders = {\"Authorization\": \"Bearer sk-ollabridge-...\"}\n\n# Option 2: Custom header\nheaders = {\"X-API-Key\": \"sk-ollabridge-...\"}\n```\n\n### Configuration (`.env`)\n\n```env\n# API Keys (comma-separated for multiple)\nAPI_KEYS=sk-ollabridge-abc123,sk-ollabridge-def456\n\n# Server\nHOST=0.0.0.0\nPORT=11435\n\n# Default models\nDEFAULT_MODEL=deepseek-r1\nDEFAULT_EMBED_MODEL=nomic-embed-text\n\n# Rate limiting\nRATE_LIMIT=60/minute\n\n# Security\nENROLLMENT_SECRET=your-secret-here\nENROLLMENT_TTL_SECONDS=3600\n\n# Database (optional)\nDATABASE_URL=postgresql://user:pass@localhost/ollabridge\n```\n\n### Enrollment Tokens\n\nCreate short-lived tokens for nodes to join:\n\n```bash\nollabridge enroll-create --ttl 3600\n```\n\nTokens expire automatically for security.\n\n---\n\n## 📡 API Reference\n\n### Core Endpoints\n\n| Endpoint | Method | Description |\n|----------|--------|-------------|\n| `/health` | GET | Gateway health + node count |\n| `/v1/chat/completions` | POST | OpenAI-compatible chat |\n| `/v1/embeddings` | POST | Generate embeddings |\n| `/v1/models` | GET | List available models (aggregated from nodes) |\n\n### Admin Endpoints (require API key)\n\n| Endpoint | Method | Description |\n|----------|--------|-------------|\n| `/admin/recent` | GET | Recent request logs |\n| `/admin/runtimes` | GET | List connected nodes |\n| `/admin/enroll` | POST | Create enrollment token |\n\n### Example: Check Connected Nodes\n\n```bash\ncurl -H \"X-API-Key: your-key\" http://localhost:11435/admin/runtimes\n```\n\n**Response:**\n```json\n{\n  \"runtimes\": [\n    {\n      \"node_id\": \"local\",\n      \"connector\": \"local_ollama\",\n      \"healthy\": true,\n      \"tags\": [\"local\"],\n      \"models\": [\"deepseek-r1\", \"llama3.1\"]\n    },\n    {\n      \"node_id\": \"colab-gpu-1\",\n      \"connector\": \"relay_link\",\n      \"healthy\": true,\n      \"tags\": [\"gpu\", \"free\"],\n      \"models\": [\"mixtral\", \"codellama\"]\n    }\n  ]\n}\n```\n\n---\n\n## 🏗️ Architecture Deep Dive\n\n### How It Works\n\n1. **Control Plane (Gateway)**: Your apps connect here\n2. **Nodes**: Any machine with GPUs/CPUs running models\n3. **Relay Link**: Nodes dial OUT to gateway (WebSocket)\n4. **Router**: Picks the best node for each request\n\n### Why \"Dial Out\" Matters\n\n**Traditional (broken):**\n```\nApp → Gateway → Try to reach GPU\n                ❌ Blocked by firewall\n                ❌ NAT issues\n                ❌ No public IP\n```\n\n**OllaBridge (works everywhere):**\n```\nApp → Gateway ← GPU dials in\n               ✅ Works from anywhere\n               ✅ No port forwarding\n               ✅ Ephemeral IPs OK\n```\n\n### Connector Types\n\n- **RelayLink**: Node dials out via WebSocket (default, works everywhere)\n- **DirectEndpoint**: HTTP to stable node (best performance)\n- **LocalOllama**: Built-in local runtime (zero config)\n\nOllaBridge picks the right one automatically.\n\n---\n\n## 📈 Scaling\n\n### Add More Workers\n\n```bash\nollabridge start --workers 4\n```\n\n### Use PostgreSQL\n\n```bash\npip install psycopg2-binary\nexport DATABASE_URL=postgresql://user:pass@localhost/ollabridge\nollabridge start --workers 8\n```\n\n### Add More Nodes\n\n```bash\n# Just keep adding nodes!\nollabridge-node join --control ... --token ...\n```\n\nOllaBridge automatically load-balances across all healthy nodes.\n\n---\n\n## 🌍 Public Access (Optional)\n\n### Quick Demo (Ngrok)\n\n```bash\nollabridge start --share\n```\n\n### Production (Cloudflare Tunnel)\n\n```bash\n# Terminal 1: Start gateway\nollabridge start\n\n# Terminal 2: Expose it\ncloudflared tunnel --url http://localhost:11435\n```\n\nNow your gateway has a public `https://` URL!\n\n**Security:** Always use API keys for public gateways.\n\n---\n\n## 🎓 Beginner's Guide\n\n### \"I've never used LLMs before\"\n\n1. Install: `pip install ollabridge`\n2. Start: `ollabridge start`\n3. Copy the API key from the output\n4. Use this code:\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:11435/v1\",\n    api_key=\"PASTE_KEY_HERE\"\n)\n\nresponse = client.chat.completions.create(\n    model=\"deepseek-r1\",\n    messages=[{\"role\": \"user\", \"content\": \"Explain Python in simple terms\"}]\n)\n\nprint(response.choices[0].message.content)\n```\n\n**That's it!** You're running AI models on your computer.\n\n### \"I want to add my gaming PC's GPU\"\n\n1. On your main computer (gateway):\n   ```bash\n   ollabridge start\n   # Copy the \"Node join token\" and gateway URL\n   ```\n\n2. On your gaming PC:\n   ```bash\n   pip install ollabridge\n   ollabridge-node join --control http://GATEWAY_IP:11435 --token TOKEN_HERE\n   ```\n\n3. Done! Your apps can now use your gaming PC's power.\n\n### \"I want to use free Colab GPUs\"\n\n1. Start your gateway at home:\n   ```bash\n   ollabridge start --share\n   # Note the public URL (https://xxx.ngrok.io)\n   ```\n\n2. In Colab notebook:\n   ```python\n   !pip install ollabridge\n   !ollabridge-node join --control https://xxx.ngrok.io --token YOUR_TOKEN\n   ```\n\n3. Now your apps use FREE Colab GPUs!\n\n**Pro tip:** When Colab disconnects, just restart and run step 2 again. Zero config changes needed.\n\n---\n\n## 🛠️ CLI Commands Reference\n\nOllaBridge includes powerful CLI commands for diagnostics, testing, and management.\n\n### Diagnostic Commands\n\n#### `ollabridge doctor`\nDiagnose your OllaBridge setup (Ollama, gateway, auth, CORS):\n\n```bash\nollabridge doctor\n```\n\n**Output:**\n```\n                     OllaBridge Doctor\n┏━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n┃ Check               ┃ Result                          ┃\n┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n│ Ollama /api/tags    │ ✅ OK                           │\n│ OllaBridge /health  │ ✅ OK                           │\n│ API_KEYS configured │ ✅ yes                          │\n│ CORS_ORIGINS        │ http://localhost:5173,...       │\n│ Auth usage          │ Use Authorization: Bearer \u003ckey\u003e │\n└─────────────────────┴─────────────────────────────────┘\n```\n\n**Use case:** Troubleshooting connection issues, verifying setup before deployment.\n\n#### `ollabridge models`\nList available models (requires API key):\n\n```bash\nollabridge models --api-key sk-ollabridge-xY9kL2mN8pQ4rT6vW1zA\n```\n\n**Output:**\n```\ndeepseek-r1\nllama3.1\nmixtral\n```\n\n**Use case:** Verify which models are available across all nodes.\n\n#### `ollabridge test-chat`\nSend a test chat completion (requires API key):\n\n```bash\n# Simple test\nollabridge test-chat \"Hello, how are you?\" --api-key sk-ollabridge-...\n\n# Specify model\nollabridge test-chat \"Explain quantum computing\" \\\n  --model deepseek-r1 \\\n  --api-key sk-ollabridge-...\n```\n\n**Output:**\n```\n╭─────────── Assistant ───────────╮\n│ Hello! I'm doing well, thank    │\n│ you for asking. How can I help  │\n│ you today?                       │\n╰──────────────────────────────────╯\n```\n\n**Use case:** Verify end-to-end connectivity, test API keys, validate model responses.\n\n### Gateway Management\n\n#### `ollabridge start`\nStart the gateway (standard mode):\n\n```bash\nollabridge start\n```\n\n#### `ollabridge start --lan`\nStart with LAN URLs displayed (for classroom/shared networks):\n\n```bash\nollabridge start --lan\n```\n\n**Output includes:**\n```\n🌐 LAN Access\nLAN API base:    http://192.168.1.50:11435/v1\nLAN Health:      http://192.168.1.50:11435/health\n\nExample (with API key):\ncurl -H 'Authorization: Bearer \u003cAPI_KEY\u003e' http://192.168.1.50:11435/v1/models\n```\n\n**Use case:** Sharing your gateway with other devices on your network (Quest headsets, phones, other laptops).\n\n\n#### `ollabridge start --share`\nExpose a public URL (via ngrok):\n\n```bash\nollabridge start --share\n```\n\n**Use case:** Remote access, connecting nodes from anywhere.\n\n#### `ollabridge enroll-create`\nCreate enrollment tokens for nodes:\n\n```bash\nollabridge enroll-create --ttl 3600\n```\n\n### Quick Tasks\n\n| Task | Command |\n|------|---------|\n| **List models (API)** | `ollabridge models --api-key \u003ckey\u003e` |\n| **Test connectivity** | `ollabridge test-chat \"test\" --api-key \u003ckey\u003e` |\n| **Check health** | `curl http://localhost:11435/health` |\n| **Diagnose setup** | `ollabridge doctor` |\n| **See nodes** | `curl -H \"X-API-Key: \u003ckey\u003e\" http://localhost:11435/admin/runtimes` |\n| **View logs** | `curl -H \"X-API-Key: \u003ckey\u003e\" http://localhost:11435/admin/recent` |\n| **Create token** | `ollabridge enroll-create` |\n\n\n### For Developers\n\nOllaBridge requires an API key to authenticate requests. If no `.env` file is provided, or the `.env` file does not contain `API_KEYS`, OllaBridge will automatically generate a **temporary, per-run secret API key** (`sk-ollabridge-...`), print it to the screen, and use it only for the current run so you can start developing immediately. **This key is not written to disk by default**, which prevents accidental persistence of credentials and improves security. If you explicitly want OllaBridge to persist the generated API key, you must opt in by starting the gateway with:\n\n```bash\nollabridge start --write-env\n```\n\nIn this case, OllaBridge will write the generated key to `.env`. For production deployments, it is strongly recommended to set `API_KEYS` using **environment variables or a secure secret manager**, rather than relying on a `.env` file. This design provides safe defaults while avoiding unintentionally storing sensitive information.\n\n\n---\n\n## ☁️ Optional: OllaBridge Cloud\n\nOllaBridge Local can **optionally** connect to **OllaBridge Cloud** for multi-user, multi-device deployments.\n\n### Cloud Features\n\n- 🔐 **Secure device pairing** with user approval\n- 👥 **Multi-user support** with device ownership\n- 🌍 **No port forwarding needed** (devices dial out to Cloud)\n- 📱 **Multi-device per user** (PC + Quest + phone, etc.)\n- 🔄 **Streaming support** for real-time responses\n\n### Pairing Your Device with Cloud\n\n```bash\n# 1. Pair this device with OllaBridge Cloud\nollabridge-node cloud-pair --cloud https://your-cloud-url.com\n\n# Shows pairing code - approve via web UI\n\n# 2. Connect to Cloud (uses saved credentials)\nollabridge-node cloud-connect\n```\n\n**How it works:**\n1. `cloud-pair` gets a pairing code from Cloud\n2. You approve the code via Cloud's web UI\n3. Device credentials saved to `~/.ollabridge/cloud_device.json`\n4. `cloud-connect` connects your device to Cloud relay\n5. Cloud routes requests to your device securely\n\n### Local Mode (Default) vs Cloud Mode\n\n| Feature | Local Mode | Cloud Mode |\n|---------|------------|------------|\n| **Setup** | `ollabridge-node join --control \u003cgateway\u003e --token \u003ctoken\u003e` | `ollabridge-node cloud-pair --cloud \u003curl\u003e` |\n| **Authentication** | Enrollment token | Device pairing + approval |\n| **Users** | Single self-hosted | Multi-user cloud accounts |\n| **Devices** | Manual node management | Per-user device ownership |\n| **Streaming** | Not yet | ✅ Supported |\n| **Port forwarding** | Not needed (outbound) | Not needed (outbound) |\n\n**Both modes work together!** Run local gateway + nodes for self-hosting, and optionally pair devices with Cloud for multi-user scenarios.\n\n---\n\n## 🗺️ Roadmap\n\n- [x] ✅ Control Plane + Node architecture\n- [x] ✅ Outbound-only node enrollment (no port forwarding)\n- [x] ✅ MCP server for AI agent control\n- [x] ✅ Multi-node load balancing\n- [x] ✅ Diagnostic CLI commands (doctor, models, test-chat)\n- [x] ✅ Enhanced CORS handling for browser clients\n- [x] ✅ LAN mode for classroom/shared network deployments\n- [x] ✅ Cloud compatibility (optional device pairing)\n- [x] ✅ Streaming support for chat completions (Cloud mode)\n- [ ] 🚧 Tag-based routing (send \"coding\" requests to GPU nodes)\n- [ ] 🚧 Model-specific routing rules\n- [ ] 🚧 Web UI for node management\n- [ ] 🚧 Prometheus metrics\n- [ ] 🚧 Support for more runtimes (vLLM, llama.cpp, LM Studio)\n\n---\n\n## 🤝 Contributing\n\nWe welcome contributions! Areas we'd love help:\n\n- 🔌 More runtime adapters (vLLM, llama.cpp, etc.)\n- 🎨 Web UI for management\n- 📊 Better monitoring/metrics\n- 🔒 Security enhancements\n- 📖 Documentation improvements\n\n**How to contribute:**\n\n1. Fork the repo\n2. Create a branch (`git checkout -b feature/amazing`)\n3. Make your changes\n4. Add tests\n5. Submit a PR\n\n---\n\n## 📄 License\n\nApache License 2.0 - see [LICENSE](LICENSE)\n\n---\n\n## 🙏 Built With\n\n- [FastAPI](https://fastapi.tiangolo.com/) — Modern async web framework\n- [Ollama](https://ollama.ai/) — Run LLMs locally\n- [WebSockets](https://websockets.readthedocs.io/) — Real-time node connections\n- [SQLModel](https://sqlmodel.tiangolo.com/) — Database with Python types\n\n---\n\n## 💬 Support\n\n- 📖 [Documentation](docs/)\n- 🐛 [Report Bug](https://github.com/ruslanmv/ollabridge/issues)\n- 💡 [Request Feature](https://github.com/ruslanmv/ollabridge/issues)\n- 💬 [Discussions](https://github.com/ruslanmv/ollabridge/discussions)\n\n---\n\n## 🌟 Star History\n\nIf OllaBridge helped you, give it a star! ⭐\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**Made with ❤️ for the local-first AI community**\n\n**Stop paying cloud tokens. Use your own compute.**\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruslanmv%2Follabridge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fruslanmv%2Follabridge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fruslanmv%2Follabridge/lists"}