{"id":47140649,"url":"https://github.com/ducks/llm-mux","last_synced_at":"2026-03-12T22:02:33.714Z","repository":{"id":337353217,"uuid":"1153204731","full_name":"ducks/llm-mux","owner":"ducks","description":"Multiplexer for LLMs - route prompts to multiple backends and orchestrate multi-step workflows","archived":false,"fork":false,"pushed_at":"2026-02-22T02:22:40.000Z","size":415,"stargazers_count":0,"open_issues_count":16,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-22T04:22:28.770Z","etag":null,"topics":["cli","llm","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ducks.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-09T03:03:22.000Z","updated_at":"2026-02-22T02:22:43.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/ducks/llm-mux","commit_stats":null,"previous_names":["ducks/llmux","ducks/llm-mux"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/ducks/llm-mux","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducks%2Fllm-mux","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducks%2Fllm-mux/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducks%2Fllm-mux/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducks%2Fllm-mux/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ducks","download_url":"https://codeload.github.com/ducks/llm-mux/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ducks%2Fllm-mux/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":30446425,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-03-12T21:31:01.033Z","status":"ssl_error","status_checked_at":"2026-03-12T21:30:43.161Z","response_time":114,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","llm","rust"],"created_at":"2026-03-12T22:02:29.886Z","updated_at":"2026-03-12T22:02:33.679Z","avatar_url":"https://github.com/ducks.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# llm-mux\n\nMultiplexer for LLMs. Route prompts to multiple backends, run them in parallel,\nand orchestrate multi-step workflows.\n\n## Quick Start\n\n### 1. Install\n\n```bash\n# From crates.io\ncargo install llm-mux\n\n# Or from source\ncargo install --path .\n```\n\n### 2. Configure Backends\n\nCreate `~/.config/llm-mux/config.toml`:\n\n```toml\n# CLI backends (any command that accepts a prompt)\n[backends.claude]\ncommand = \"claude\"\nargs = [\"-p\"]\n\n[backends.codex]\ncommand = \"codex\"\nargs = [\"exec\", \"-q\"]\n\n[backends.ollama]\ncommand = \"ollama\"\nargs = [\"run\", \"llama3\"]\n\n# HTTP backends (OpenAI-compatible APIs)\n[backends.openai]\ncommand = \"https://api.openai.com/v1\"\nmodel = \"gpt-4\"\napi_key = \"${OPENAI_API_KEY}\"  # from environment\n\n[backends.local]\ncommand = \"http://localhost:11434/v1\"\nmodel = \"llama3\"\n```\n\n### 3. Define Roles\n\nRoles map task types to backends:\n\n```toml\n[roles.analyzer]\ndescription = \"Code analysis tasks\"\nbackends = [\"claude\", \"codex\"]\nexecution = \"parallel\"  # first | parallel | fallback\n\n[roles.quick]\ndescription = \"Fast responses\"\nbackends = [\"ollama\"]\nexecution = \"first\"\n```\n\n### 4. Create a Workflow\n\nCreate `.llm-mux/workflows/review.toml` in your project:\n\n```toml\nname = \"review\"\ndescription = \"Review code changes\"\n\n[[steps]]\nname = \"diff\"\ntype = \"shell\"\nrun = \"git diff HEAD~1\"\n\n[[steps]]\nname = \"analyze\"\ntype = \"query\"\nrole = \"analyzer\"\nprompt = \"\"\"\nReview these changes for bugs and improvements:\n\n{{ steps.diff.output }}\n\"\"\"\ndepends_on = [\"diff\"]\n```\n\n### 5. Run It\n\n```bash\nllm-mux run review\n```\n\n## Configuration\n\n### Config Locations\n\n1. `~/.config/llm-mux/config.toml` (user defaults)\n2. `.llm-mux/config.toml` (project overrides)\n\nLater files override earlier ones.\n\n### Backend Options\n\n```toml\n[backends.example]\ncommand = \"claude\"           # CLI command or HTTP URL\nargs = [\"-p\"]                # arguments for CLI backends\nmodel = \"gpt-4\"              # model name for HTTP backends\napi_key = \"${ENV_VAR}\"       # API key (supports env vars)\nenabled = true               # enable/disable\ntimeout = 300                # seconds\nmax_retries = 3              # retry attempts\nretry_delay = 1000           # base delay in ms (exponential backoff)\nretry_rate_limit = true      # auto-retry on rate limits\nretry_timeout = false        # auto-retry on timeouts\n```\n\n### Role Execution Modes\n\n- `first`: Use first available backend (default)\n- `parallel`: Run all backends, collect results\n- `fallback`: Try each backend until one succeeds\n\n### Teams\n\nAuto-detect project type and apply team-specific settings:\n\n```toml\n[teams.rust]\ndescription = \"Rust projects\"\ndetect = [\"Cargo.toml\"]\nverify = \"cargo clippy \u0026\u0026 cargo test\"\n\n[teams.rust.roles.analyzer]\nbackends = [\"claude\", \"codex\"]  # override for Rust projects\n```\n\n### Ecosystems\n\nTrack relationships between projects and store ecosystem knowledge:\n\n```toml\n[ecosystems.myapp]\ndescription = \"MyApp web application and services\"\nknowledge = [\n    \"API uses JWT tokens with 1 hour expiration\",\n    \"Database migrations run automatically on deploy\",\n    \"Redis cache invalidation happens via pub/sub\"\n]\n\n[ecosystems.myapp.projects.frontend]\npath = \"~/projects/myapp-frontend\"\ntype = \"javascript\"\ndepends_on = [\"api\"]\ntags = [\"production\", \"web\"]\n\n[ecosystems.myapp.projects.api]\ndescription = \"REST API backend\"\npath = \"~/projects/myapp-api\"\ntype = \"rust\"\ndepends_on = [\"database\"]\ntags = [\"production\", \"backend\"]\n\n[ecosystems.myapp.projects.database]\ndescription = \"PostgreSQL database with migrations\"\npath = \"~/projects/myapp-db\"\ntype = \"sql\"\ntags = [\"infrastructure\", \"database\"]\n\n[ecosystems.myapp.projects.worker]\ndescription = \"Background job processor\"\npath = \"~/projects/myapp-worker\"\ntype = \"rust\"\ndepends_on = [\"database\", \"api\"]\ntags = [\"production\", \"background\"]\n```\n\nWorkflows automatically detect which ecosystem you're in and can access:\n\n```jinja2\n{{ ecosystem.name }}                    - Ecosystem name\n{{ ecosystem.description }}             - Description\n{{ ecosystem.knowledge }}               - Array of facts\n{{ ecosystem.projects }}                - All projects\n{{ ecosystem.current_project.name }}    - Current project name\n{{ ecosystem.current_project.type }}    - Project type\n{{ ecosystem.current_project.depends_on }} - Dependencies\n{{ ecosystem.current_project.tags }}    - Project tags\n```\n\n## Workflows\n\n### Step Types\n\n```toml\n# Shell: run a command\n[[steps]]\nname = \"fetch\"\ntype = \"shell\"\nrun = \"gh issue view {{ args.issue }}\"\n\n# Query: call LLM backend(s)\n[[steps]]\nname = \"analyze\"\ntype = \"query\"\nrole = \"analyzer\"\nprompt = \"Analyze: {{ steps.fetch.output }}\"\ndepends_on = [\"fetch\"]\n\n# Apply: apply edits from LLM output\n[[steps]]\nname = \"fix\"\ntype = \"apply\"\nsource = \"steps.analyze\"\nverify = \"cargo test\"\nverify_retries = 2\nrollback_on_failure = true\ndepends_on = [\"analyze\"]\n\n# Store: persist data to SQLite memory database\n[[steps]]\nname = \"save_facts\"\ntype = \"store\"\nprompt = \"{{ steps.analyze.output }}\"\ndepends_on = [\"analyze\"]\n```\n\n### Template Variables\n\n- `{{ args.name }}`: workflow arguments\n- `{{ steps.name.output }}`: previous step output\n- `{{ env.VAR }}`: environment variables\n- `{{ team }}`: detected team name\n- `{{ ecosystem.name }}`: detected ecosystem\n- `{{ ecosystem.knowledge }}`: ecosystem facts\n- `{{ ecosystem.current_project }}`: current project info\n\n### Filters\n\n```\n{{ value | shell_escape }}   # escape for shell\n{{ value | json }}           # JSON encode\n{{ list | join(\", \") }}      # join array\n{{ text | lines }}           # split into lines\n{{ text | trim }}            # trim whitespace\n{{ value | default(\"x\") }}   # default if empty\n```\n\n### Conditionals\n\n```toml\n[[steps]]\nname = \"rust-only\"\ntype = \"shell\"\nrun = \"cargo clippy\"\nif = \"team == 'rust'\"\n```\n\n### Iteration\n\n```toml\n[[steps]]\nname = \"check-files\"\ntype = \"shell\"\nrun = \"wc -l {{ item }}\"\nfor_each = \"steps.list.output | lines\"\n```\n\n### Store Steps and Ecosystem Memory\n\nStore steps persist LLM analysis results to a SQLite database for later querying.\n\n**Database Location:**\n`~/.config/llm-mux/memory/\u003cecosystem-name\u003e.db`\n\n**Data Format:**\nStore steps parse JSON from previous steps and save to the memory database.\n\nFacts require:\n```json\n{\n  \"facts\": [\n    {\n      \"project\": \"project-name\",\n      \"fact\": \"description\",\n      \"source\": \"where found (e.g., Cargo.toml)\",\n      \"confidence\": 1.0\n    }\n  ]\n}\n```\n\nRelationships require:\n```json\n{\n  \"relationships\": [\n    {\n      \"from\": \"source-project\",\n      \"to\": \"target-project\",\n      \"type\": \"depends_on|calls_api|shares_db|deploys_with\",\n      \"evidence\": \"brief explanation\"\n    }\n  ]\n}\n```\n\n**Usage:**\n```toml\n[[steps]]\nname = \"analyze\"\ntype = \"query\"\nrole = \"analyzer\"\nprompt = \"\"\"\nAnalyze the codebase and return facts.\n\nIMPORTANT: Return JSON with this exact structure:\n{\n  \"facts\": [\n    {\"project\": \"myapp\", \"fact\": \"Uses PostgreSQL\", \"source\": \"config.yml\", \"confidence\": 1.0}\n  ]\n}\n\"\"\"\n\n[[steps]]\nname = \"store\"\ntype = \"store\"\nprompt = \"{{ steps.analyze.output }}\"\ndepends_on = [\"analyze\"]\n```\n\nSee `examples/workflows/discover-ecosystem.toml` for a complete example.\n\n## CLI Reference\n\n```\nllm-mux run \u003cworkflow\u003e [args...]   Run a workflow\nllm-mux validate \u003cworkflow\u003e        Validate workflow syntax\nllm-mux doctor                     Check backend availability\nllm-mux backends                   List configured backends\nllm-mux teams                      List configured teams\nllm-mux roles                      List configured roles\nllm-mux ecosystems                 List configured ecosystems\n\nOptions:\n  --team \u003cname\u003e      Override team detection\n  --output \u003cmode\u003e    Output format: console, json, quiet\n  --debug            Enable debug output\n  --quiet            Suppress progress output\n```\n\n## Examples\n\n### Simple Review\n\n```toml\n# .llm-mux/workflows/review.toml\nname = \"review\"\n\n[[steps]]\nname = \"diff\"\ntype = \"shell\"\nrun = \"git diff\"\n\n[[steps]]\nname = \"review\"\ntype = \"query\"\nrole = \"analyzer\"\nprompt = \"Review this diff:\\n{{ steps.diff.output }}\"\ndepends_on = [\"diff\"]\n```\n\n### Parallel Analysis\n\n```toml\nname = \"analyze\"\n\n[roles.multi]\nbackends = [\"claude\", \"codex\", \"gemini\"]\nexecution = \"parallel\"\n\n[[steps]]\nname = \"analyze\"\ntype = \"query\"\nrole = \"multi\"\nprompt = \"Find bugs in: {{ args.file }}\"\n```\n\n### Fix with Verification\n\n```toml\nname = \"fix\"\n\n[[steps]]\nname = \"identify\"\ntype = \"query\"\nrole = \"analyzer\"\nprompt = \"Find the bug in {{ args.file }}\"\n\n[[steps]]\nname = \"fix\"\ntype = \"query\"\nrole = \"coder\"\nprompt = \"\"\"\nFix this bug: {{ steps.identify.output }}\n\nReturn edits as JSON: {\"path\": \"...\", \"old\": \"...\", \"new\": \"...\"}\n\"\"\"\ndepends_on = [\"identify\"]\n\n[[steps]]\nname = \"apply\"\ntype = \"apply\"\nsource = \"steps.fix\"\nverify = \"cargo test\"\nverify_retries = 2\ndepends_on = [\"fix\"]\n```\n\n### Ecosystem-Aware Bug Hunt\n\n```toml\nname = \"bug-hunt\"\ndescription = \"Search for bugs across ecosystem\"\n\n[[steps]]\nname = \"analyze\"\ntype = \"query\"\nrole = \"analyzer\"\nprompt = \"\"\"\nSearch for potential bugs in {{ ecosystem.current_project.name }}.\n\nProject type: {{ ecosystem.current_project.type }}\n\nKnown ecosystem facts:\n{% for fact in ecosystem.knowledge %}\n- {{ fact }}\n{% endfor %}\n\nDependencies to consider:\n{% for dep in ecosystem.current_project.depends_on %}\n- {{ dep }}\n{% endfor %}\n\nFocus on common issues for {{ ecosystem.current_project.type }} projects.\n\"\"\"\n```\n\n### Ecosystem Discovery\n\nAnalyze projects and store findings in the memory database:\n\n```toml\nname = \"discover\"\ndescription = \"Discover and store ecosystem facts\"\n\n[[steps]]\nname = \"analyze\"\ntype = \"query\"\nrole = \"analyzer\"\ntimeout = 180\nprompt = \"\"\"\nAnalyze the {{ ecosystem.current_project.name }} project structure.\n\nIMPORTANT: Return JSON with this exact structure:\n{\n  \"facts\": [\n    {\n      \"project\": \"project-name\",\n      \"fact\": \"brief fact description\",\n      \"source\": \"where you found this (e.g., Cargo.toml)\",\n      \"confidence\": 1.0\n    }\n  ]\n}\n\nEach fact MUST have all four fields: project, fact, source, confidence.\n\"\"\"\n\n[steps.output_schema]\ntype = \"object\"\nrequired = [\"facts\"]\n\n[steps.output_schema.properties.facts]\ntype = \"array\"\n\n[[steps]]\nname = \"store\"\ntype = \"store\"\nprompt = \"{{ steps.analyze.output }}\"\ndepends_on = [\"analyze\"]\n```\n\nSee `examples/workflows/discover-ecosystem.toml` for a complete implementation with multiple analysis steps and relationship discovery.\n\n## Contributing\n\nllm-mux is built with Rust. To contribute:\n\n```bash\ngit clone https://github.com/ducks/llm-mux\ncd llm-mux\ncargo build\ncargo test\n```\n\n## Publishing\n\nPublished to [crates.io](https://crates.io/crates/llm-mux) and [GitHub](https://github.com/ducks/llm-mux).\n\n## License\n\nMIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fducks%2Fllm-mux","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fducks%2Fllm-mux","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fducks%2Fllm-mux/lists"}