{"id":50897841,"url":"https://github.com/shouvik12/trooper","last_synced_at":"2026-07-22T07:01:15.187Z","repository":{"id":352760833,"uuid":"125002340","full_name":"shouvik12/trooper","owner":"shouvik12","description":"LLM reliability layer -keeps agents alive with smart routing, context compaction, and local fallback","archived":false,"fork":false,"pushed_at":"2026-06-18T17:07:57.000Z","size":904,"stargazers_count":33,"open_issues_count":1,"forks_count":5,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-18T19:09:49.288Z","etag":null,"topics":["ai-gateway","context-compression","fallback","go","golang","llm","llm-proxy","local-llm","ollama","proxy"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shouvik12.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2018-03-13T06:21:12.000Z","updated_at":"2026-06-18T17:08:05.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/shouvik12/trooper","commit_stats":null,"previous_names":["shouvik12/trooper"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/shouvik12/trooper","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shouvik12%2Ftrooper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shouvik12%2Ftrooper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shouvik12%2Ftrooper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shouvik12%2Ftrooper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shouvik12","download_url":"https://codeload.github.com/shouvik12/trooper/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shouvik12%2Ftrooper/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35751644,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-07-20T02:08:10.276Z","status":"online","status_checked_at":"2026-07-22T02:00:06.236Z","response_time":124,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-gateway","context-compression","fallback","go","golang","llm","llm-proxy","local-llm","ollama","proxy"],"created_at":"2026-06-16T01:31:30.082Z","updated_at":"2026-07-22T07:01:15.171Z","avatar_url":"https://github.com/shouvik12.png","language":"Go","funding_links":[],"categories":["go"],"sub_categories":[],"readme":"**NEW:** [Chat UI demo →](http://127.0.0.1:3000/chat) | [Subagent recovery demo →](https://youtu.be/NN2uwQZDCck) | [Subagent recovery article →](https://dev.to/shouvik12/i-added-a-recovery-endpoint-to-my-llm-proxy-so-agents-never-lose-progress-mid-task-524b) | [4-agent privacy routing demo →](https://dev.to/shouvik12/i-tested-privacy-aware-routing-with-4-ai-agents-what-actually-stayed-local-39oa)\n\n# 🪖 Trooper\n\n\u003e **Trooper 4.0 — Chat UI with escalate. Local first. Cloud when needed. One click to switch. Context never lost.**\n\nTrooper started as a fallback proxy. It's now a local-first AI workspace.\n\nYour agent runs. Trooper watches. You chat. You escalate. Context flows. Nothing breaks.\n\n```\n→ Any agent          → point at Trooper, open dashboard, see everything\n→ Chat with Llama    → fast, free, private by default\n→ Need more power    → one click escalates to Claude with full context\n→ Back to local      → Llama knows everything Claude said\n→ Claude fails       → continues on Ollama, context preserved\n→ Simple prompts     → never hit the cloud\n```\n\n**Trooper is a zero-instrumentation agent observability platform with local fallback — and now a local-first chat interface.**\n\n![Trooper Home Screen](assets/home.png)\n\n---\n\n## What's new in 4.0\n\n**Chat UI** — open `http://localhost:3000/chat`:\n\n- **Local first** — all messages route to your local Ollama model by default\n- **One-click escalate** — stuck on something? Escalate to Claude with full session context injected automatically\n- **Context never lost** — back to local, Llama knows exactly what Claude said\n- **SITREP panel** — live intent, confidence, entities, open loops extracted from your conversation\n- **Markdown rendering** — code blocks, tables, headers render cleanly\n- **Session memory** — full conversation history flows between providers\n\n---\n\n## What you see\n\n**In the chat UI** — open `http://localhost:3000/chat`:\n\n```\nLocal first. Escalate when stuck. Context never lost.\n\nDev: explain why my postgres query is slow despite an index on user_id\n\nLlama: Even with an index on user_id, your query might be slow if filtering\n       on other columns, or using functions like LIKE which prevent index usage.\n\n[↑ Escalate]\n\n— escalated to Claude —\n\nClaude: Common reasons your Postgres query is slow despite a user_id index:\n        1. Index not being used — functions, type casts\n        2. Filtering on other columns — unindexed columns cause row scans\n        3. Returning large result sets — add LIMIT or filter more\n        Run EXPLAIN ANALYZE to see which step is slow.\n\n[↩ Back to local]\n```\n\n![Trooper Escalate to Claude](assets/escalate.png)\n\n![Claude Response](assets/clauderesponse.png)\n\n— returned to local —\n\nDev: summarize what Claude just recommended\n\nLlama: Your query might be slow due to functions preventing index usage,\n       or filtering on unindexed columns. Run EXPLAIN ANALYZE to identify\n       which step is slow and whether the index is being used.\n```\n\n![Trooper Memory Proof](assets/memory.png)\n\n**In the dashboard** — open `http://localhost:3000/dashboard` while your agent runs:\n\n- **Intent** — what your agent is trying to do, extracted automatically\n- **Open Loops** — what it's stuck on, highlighted in real time\n- **Completed Steps** — what it finished, tracked as it happens\n- **Session Transcript** — every message, colour coded by role\n\n**In every response header** — no dashboards required:\n\n```bash\n# Simple question → Ollama handled it, cloud never contacted\nX-Trooper-Provider: ollama\nX-Trooper-Decision: ollama (simple turn) | cloud skipped\nX-Trooper-Session-Saved: 42 tokens\n\n# Complex question → Claude handled it\nX-Trooper-Provider: claude\nX-Trooper-Summary: claude (direct) ✓\n\n# Claude quota hit → fell back to Ollama, context preserved\nX-Trooper-Provider: ollama\nX-Trooper-Decision: ollama (fallback: credit_balance)\nX-Trooper-Session-Saved: 42 tokens\n```\n\n---\n\n## What Trooper is\n\nTrooper is a drop-in proxy that sits between your agent (or you) and any LLM provider. It observes every request, extracts intent and signals, and builds a live picture of what's happening — all without touching your code.\n\nWhen cloud models fail — quota, rate limits, outages — it automatically falls back to your local Ollama instance while preserving full conversation context.\n\n**Trooper is no longer passive.** It started as a fallback proxy. Now it watches every session, makes that data visible, and gives you a local-first chat interface to work with.\n\nNo retries. No crashes. No lost sessions. No SDK. No instrumentation. ⏱ Runs in under 60 seconds.\n\n---\n\n## Who uses Trooper\n\n**Local LLM developers** — run Ollama for privacy and cost, escalate to Claude when you need more. Context flows automatically. No copy-pasting, no re-explaining.\n\n**Agent builders** — see exactly what your agent is doing, what it's stuck on, and what it completed. Zero instrumentation — just point your agent at Trooper.\n\n**App developers** — your users never see quota errors. Trooper falls over to local Ollama transparently while your app keeps running.\n\n**Claude Code / Cursor users** — coding sessions survive quota hits. No lost context, no starting over.\n\n**Privacy-conscious developers** — sensitive requests stay local. Cloud only when you choose.\n\n---\n\n## Why not LiteLLM, Bifrost, or Helicone\n\n| | LiteLLM / Bifrost | Helicone | Trooper |\n|---|---|---|---|\n| Chat UI | ❌ | ❌ | ✅ Local-first with escalate |\n| Context handover | ❌ | ❌ | ✅ Full context on provider switch |\n| Observability | ❌ | Request-level only | ✅ Intent, open loops, completed steps |\n| Instrumentation needed | SDK required | None | None |\n| Fallback target | Another cloud | Another cloud | Your local machine |\n| Local / private | ❌ | ❌ Cloud only | ✅ Data never leaves machine |\n| Setup | `pip install`, YAML | API key, cloud account | One Go binary, env vars |\n| Status | Active | Maintenance mode | Active |\n\n---\n\n## Quickstart\n\n⏱ Runs in under 60 seconds.\n\n### Option 1 — Docker (no Go required)\n\n```bash\ngit clone https://github.com/shouvik12/trooper\ncd trooper\ncp .env.example .env\n# edit .env — set CLAUDE_API_KEY\ndocker compose up\n\n# First run: pull the model\ndocker compose exec ollama ollama pull llama3.1:8b\n```\n\n### Option 2 — Run from source (Go 1.22+)\n\n```bash\ngit clone https://github.com/shouvik12/trooper\ncd trooper\nexport CLAUDE_API_KEY=sk-ant-...   # optional — works without it\nexport OLLAMA_MODEL=llama3.1:8b\ngo run .\n```\n\nOpen the chat UI: `http://localhost:3000/chat`\nOpen the dashboard: `http://localhost:3000/dashboard`\n\n---\n\n## Chat UI\n\n```bash\nopen http://localhost:3000/chat\n```\n\n**How it works:**\n\n1. Messages default to your local Ollama model — free, private, fast\n2. Not satisfied? Click **↑ Escalate** — Claude receives full session context automatically\n3. Claude answers. Click **↩ Back to local** — Llama picks up with full context of what Claude said\n4. SITREP panel updates live — intent, confidence, entities, open loops\n\n**No copy-pasting. No re-explaining. Context never lost.**\n\n---\n\n## Agent usage\n\nPoint your existing client at Trooper — nothing else changes:\n\n**Python + Anthropic SDK:**\n```python\nimport anthropic\nclient = anthropic.Anthropic(\n    api_key=\"your-key\",\n    base_url=\"http://localhost:3000\",  # only change\n)\n```\n\n**Python + OpenAI SDK:**\n```python\nfrom openai import OpenAI\nclient = OpenAI(\n    api_key=\"your-key\",\n    base_url=\"http://localhost:3000\",  # only change\n)\n```\n\n**curl:**\n```bash\ncurl http://localhost:3000/v1/messages \\\n  -H \"Content-Type: application/json\" \\\n  -H \"X-Session-ID: my-session\" \\\n  -d '{\"model\": \"claude-haiku-4-5\", \"max_tokens\": 1024, \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]}'\n```\n\n---\n\n## Smart routing\n\nTrooper decides when the cloud is overkill.\n\n\u003e **The classifier is rule-based and deterministic — no LLM call, no latency, no cost to classify.**\n\n```\n\"how many days in a week\"  →  Ollama directly 🪖  (cloud never contacted)\n\"explain why goroutines…\"  →  Claude ✅           (needs reasoning)\n```\n\n---\n\n## How Trooper handles context\n\nThe hard part of fallback isn't switching models — it's keeping context.\n\nTrooper solves that with a 3-layer compaction system:\n\n```\nANCHOR  (~10%)  — First 2 turns verbatim, never dropped\nSITREP  (~20%)  — Rule-based summary of middle turns\nTAIL    (~70%)  — Last N turns verbatim\n                  Total \u003c= 6144 tokens (configurable)\n```\n\n---\n\n## Provider chain\n\n```bash\nCLAUDE_API_KEY=sk-ant-...                          # Chain: Claude → Ollama\nCLAUDE_API_KEY=sk-ant-...  GEMINI_API_KEY=AIza...  # Chain: Claude → Gemini → Ollama\nCLAUDE_API_KEY=sk-ant-...  OPENAI_API_KEY=sk-...   # Chain: Claude → OpenAI → Ollama\n```\n\n---\n\n## Fallback behaviour\n\n| Status | Trooper action |\n|---|---|\n| `200 OK` | Pass through |\n| `429 Rate Limited` | Retry with 2s backoff, then try next |\n| `402 Payment Required` | Fall back immediately |\n| `400 Credit Balance / Invalid Key` | Fall back immediately |\n| `401 Unauthorized` | Surface error — bad keys are never masked |\n| `529 Overloaded` | Fall back immediately |\n| Network error | Fall back immediately — 30s timeout per provider |\n\n---\n\n## Configuration\n\n| Variable | Default | Description |\n|---|---|---|\n| `CLAUDE_API_KEY` | — | Anthropic API key |\n| `CLAUDE_MODEL` | `claude-haiku-4-5-20251001` | Default Claude model |\n| `GEMINI_API_KEY` | — | Google Gemini API key |\n| `OPENAI_API_KEY` | — | OpenAI API key |\n| `OLLAMA_MODEL` | `llama3.1:8b` | Local model |\n| `FALLBACK_URL` | `http://localhost:11434/api/chat` | Ollama endpoint |\n| `CONTEXT_WINDOW` | `6144` | Token budget for context compaction |\n| `QUOTA_STATUS_CODES` | `429,402,529,400,404` | HTTP codes that trigger fallback |\n| `TROOPER_PORT` | `3000` | Port Trooper listens on |\n| `TROOPER_BIND` | `127.0.0.1` | Bind address |\n| `AUTO_RECOVERY` | `false` | Enable automatic recovery to primary provider |\n\n---\n\n## Recommended local models\n\n| Model | Size | Notes |\n|---|---|---|\n| `llama3.1:8b` | 4.9GB | Default — strong all-rounder |\n| `qwen2.5:3b` | 1.9GB | Fast, lightweight |\n| `qwen2.5:7b` | 4.7GB | Better quality, still fast |\n| `mistral:7b` | 4.1GB | Good reasoning |\n\n---\n\n## Running tests\n\n```bash\ngo test ./... -v\n./sanity.sh\n```\n\n---\n\n## Roadmap\n\n**V4.0 — Released**\n- ✅ Chat UI — `localhost:3000/chat`\n- ✅ Local-first routing — Ollama by default, Claude on demand\n- ✅ One-click escalate with full context injection\n- ✅ Back to local — Llama knows what Claude said\n- ✅ SITREP panel — live intent, confidence, entities, open loops\n- ✅ Markdown rendering — code blocks, tables, headers\n- ✅ Session memory across provider switches\n- ✅ Provider-aware context store\n\n**V3.3 — Released**\n- ✅ Live dashboard — `localhost:3000/dashboard`\n- ✅ Sessions endpoint — `localhost:3000/sessions`\n- ✅ Zero instrumentation agent observability\n\n**V3.2 — Released**\n- ✅ Subagent recovery — `/recovery/{session_id}`\n- ✅ Response normalization\n\n**V3.1 — Released**\n- ✅ Smart routing — simple turns skip the cloud\n- ✅ Deterministic classifier — zero latency to route\n\n**V3.0 — Released**\n- ✅ Circuit breaker\n- ✅ X-Trooper headers\n\n---\n\n## Recognition\n\n- Featured in [Agent Brief](https://news.agentcommunity.org/issues/2026-04-22-the-agentic-stack) by agentcommunity.org — curated alongside Anthropic, Shopify MCP, and LangGraph updates (April 2026)\n- Featured on [@github_unpacked](https://www.instagram.com/reel/DXfDrCOCNHE/) — Instagram reel with 76 saves\n- Featured on [PatentLLM](https://media.patentllm.org/news/local-ai/qwen3-6-27b-local-inference-on-rtx-3090-with-native-vllm-oll-20260502) — covered alongside Qwen3.6-27B RTX 3090 local inference story (May 2026)\n- Featured on [dev.to](https://dev.to/soytuber/qwen36-27b-local-inference-on-rtx-3090-with-native-vllm-ollama-fallama-fallback-2jgg) — local AI tooling roundup (May 2026)\n- Cited by [kylebrodeur](https://github.com/kylebrodeur) as inspiration for *\"robust, transparent HTTP rate-limit fallback triggers\"*\n- Listed on [UND-RDR](https://undrdr.com) — underrated GitHub repo discovery index\n\n---\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshouvik12%2Ftrooper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshouvik12%2Ftrooper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshouvik12%2Ftrooper/lists"}