{"id":51543919,"url":"https://github.com/eulogik/fugusashi","last_synced_at":"2026-07-09T16:01:15.780Z","repository":{"id":366808425,"uuid":"1277825694","full_name":"eulogik/fugusashi","owner":"eulogik","description":"Fugusashi — Federated Learning of LLM Routing with Human-Interpretable Decisions. Open-source alternative to Sakana AI Fugu.","archived":false,"fork":false,"pushed_at":"2026-07-07T13:34:06.000Z","size":157,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-07-07T14:02:35.485Z","etag":null,"topics":["ai-orchestrator","cma-es","cost-optimization","federated-learning","llm","llm-router","machine-learning","model-router","multi-agent","open-source","openrouter","python","sakana-fugu-alternative"],"latest_commit_sha":null,"homepage":"https://eulogik.github.io/fugusashi/","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eulogik.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-06-23T08:13:50.000Z","updated_at":"2026-07-07T13:34:51.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/eulogik/fugusashi","commit_stats":null,"previous_names":["eulogik/fugusashi"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/eulogik/fugusashi","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eulogik%2Ffugusashi","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eulogik%2Ffugusashi/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eulogik%2Ffugusashi/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eulogik%2Ffugusashi/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eulogik","download_url":"https://codeload.github.com/eulogik/fugusashi/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eulogik%2Ffugusashi/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35304875,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-09T02:00:07.329Z","response_time":57,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-orchestrator","cma-es","cost-optimization","federated-learning","llm","llm-router","machine-learning","model-router","multi-agent","open-source","openrouter","python","sakana-fugu-alternative"],"created_at":"2026-07-09T16:01:14.659Z","updated_at":"2026-07-09T16:01:15.774Z","avatar_url":"https://github.com/eulogik.png","language":"Python","funding_links":[],"categories":["3. Inference Engines \u0026 Serving"],"sub_categories":[],"readme":"\u003cdiv align=\"center\"\u003e\n\n[![PyPI version](https://img.shields.io/pypi/v/fugusashi?color=ef4444\u0026label=pypi\u0026logo=pypi\u0026logoColor=white)](https://pypi.org/project/fugusashi/)\n[![Python](https://img.shields.io/badge/python-3.12+-blue.svg?logo=python\u0026logoColor=white)](https://python.org)\n[![License](https://img.shields.io/badge/license-MIT-green.svg)](LICENSE)\n[![GitHub stars](https://img.shields.io/github/stars/eulogik/fugusashi?style=social)](https://github.com/eulogik/fugusashi/stargazers)\n[![HuggingFace](https://img.shields.io/badge/%F0%9F%A4%20HuggingFace-ef4444.svg)](https://huggingface.co/eulogik/fugusashi-router)\n[![Docs](https://img.shields.io/badge/docs-eulogik.github.io-red.svg)](https://eulogik.github.io/fugusashi/)\n[![Paper](https://img.shields.io/badge/arXiv-paper-B31B1B.svg)](https://github.com/eulogik/fugusashi/blob/main/paper/main.tex)\n[![Website](https://img.shields.io/badge/website-eulogik.com-ef4444.svg)](https://eulogik.com)\n\n**By [eulogik](https://eulogik.com) — building AI infrastructure for everyone.**\n\n---\n\n# Fugusashi\n\n### Fugu Sashi. Served Free.\n\n*Fugusashi* (Japanese: 不縛 — \"unbound, unrestrained\") is an intelligent model router and multi-agent orchestrator. Named after *Fugu Sashi* — the famous Japanese pufferfish delicacy — because this router serves up the world's best AI models without the poison of vendor lock-in or the pricing of Sakana Fugu. It automatically picks the best model for each prompt, learns from every request via a CMA-ES coordinator inspired by Sakana's TRINITY paper, and runs entirely on your infrastructure.\n\n**Research contributions:** (1) **Federated routing learning** — multiple organizations collaboratively improve routing without sharing data; (2) **Human-interpretable routing** — every decision comes with a natural language explanation; (3) **Continuous CMA-ES adaptation** — routing weights evolve from outcomes.\n\n**Like Sakana Fugu. But Free. [Live Demo](https://huggingface.co/spaces/eulogik/fugusashi) · [Docs](https://eulogik.github.io/fugusashi/) · [PyPI](https://pypi.org/project/fugusashi/) · [GitHub](https://github.com/eulogik/fugusashi)**\n\n\u003c/div\u003e\n\n---\n\n## Why Fugusashi?\n\n[Sakana AI's Fugu](https://sakana.ai/fugu) is a trained orchestration model — a 7B LLM that coordinates frontier models behind a single API. It's powerful, but it's a black box: you can't see why it routes where it does, you can't self-host it, you can't train it on your data, and you pay $5-30 per million tokens plus a $20-200/month subscription.\n\n**Fugusashi is the transparent alternative.** Same CMA-ES evolution concept (both inspired by Sakana's TRINITY paper), but open, self-hostable, and designed to learn from your specific traffic. Where Fugu hides its routing logic, Fugusashi exposes every decision. Where Fugu runs only in Sakana's cloud, Fugusashi runs on your infrastructure. Where Fugu is static, Fugusashi learns from every request via a feedback loop.\n\n### Fugusashi vs Sakana AI Fugu\n\n| Feature | Sakana Fugu | Fugusashi |\n|---|---|---|\n| **Architecture** | Trained 7B coordinator LLM | CMA-ES evolved weights + rule-based ensemble |\n| **Orchestration** | ✅ TRINITY/Conductor (ICLR 2026) | ✅ Rule-based + GRPO learning |\n| **Self-Hosting** | ❌ Cloud-only | ✅ Local-first, air-gapped |\n| **Cost** | $5-30/M tokens + $20-200/mo | ✅ Free (pay only for model APIs) |\n| **Transparency** | ❌ Black box routing | ✅ Every decision visible + explained |\n| **Feedback Loop** | ❌ Static | ✅ Learns from your traffic |\n| **Federated Learning** | ❌ | ✅ Collaborative routing without data sharing |\n| **Model Pool** | Sakana-controlled frontier models | ✅ You control (100+ providers) |\n| **Training Data** | ❌ Proprietary | ✅ Community preference datasets |\n| **Dashboard** | ❌ | ✅ Live routing visualization |\n| **EU Availability** | ❌ Blocked (GDPR) | ✅ Available everywhere |\n| **License** | Proprietary | ✅ MIT |\n\n**Our edge:** Fugu is a powerful trained orchestrator with frontier models in its pool. Fugusashi is the transparent, self-hosting alternative you can run on your own infrastructure, train on your own data, and audit every decision. Different bets — Fugu bets on a trained coordinator, we bet on transparency and control.\n\n---\n\n## Quickstart\n\n### Install from PyPI\n\n```bash\npip install fugusashi\n```\n\n### Or run with Docker\n\n```bash\ndocker run -p 6060:6060 ghcr.io/eulogik/fugusashi:latest\n```\n\n### Or from source\n\n```bash\ngit clone https://github.com/eulogik/fugusashi.git\ncd fugusashi\npython3 -m venv .venv\nsource .venv/bin/activate\npip install -e .\n```\n\n### Configure\n\nEdit `config.yaml` to add your models:\n\n```yaml\ndefault_model: \"llama3.2-local\"\n\nmodels:\n  - name: \"llama3.2-local\"\n    provider: \"ollama\"\n    model: \"llama3.2:1b\"\n    api_base: \"http://localhost:11434\"\n    cost_per_input_token: 0.0\n    cost_per_output_token: 0.0\n    capabilities: [\"chat\", \"reasoning\"]\n    description: \"Llama 3.2 1B (local, free)\"\n\n  - name: \"gpt-4o-mini\"\n    provider: \"openai\"\n    model: \"gpt-4o-mini\"\n    cost_per_input_token: 0.00000015\n    cost_per_output_token: 0.0000006\n    capabilities: [\"chat\", \"reasoning\", \"code\", \"creative\"]\n    description: \"OpenAI GPT-4o-mini\"\n```\n\n### Run\n\n```bash\nfugusashi serve --config config.yaml\n# → Fugusashi router listening on 0.0.0.0:6060\n```\n\n### Use\n\n```bash\n# Auto-route — the router picks the best model\ncurl http://localhost:6060/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"auto\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hello\"}]}'\n\n# Force a specific model\ncurl http://localhost:6060/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"llama3.2-local\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hello\"}]}'\n```\n\nEvery response includes a `routing_decision` showing which model was picked, why, and with what confidence.\n\n---\n\n## Architecture\n\n![Fugusashi Architecture](https://raw.githubusercontent.com/eulogik/fugusashi/main/assets/arch.svg)\n\n### Tier 1 — Intelligent Model Router\n\nThree routing strategies in priority order:\n\n1. **SimilarityRouter** — Uses sentence-transformers to find similar past prompts and route to the model that worked best. Gets smarter with every request via the feedback loop.\n2. **CostRouter** — Capability-aware routing with cost optimization. Respects `prefer_local` for air-gapped deployments.\n3. **FallbackRouter** — Always returns a result, even with no data.\n\n### Tier 2 — Multi-Agent Orchestrator\n\nA planning model that decomposes hard tasks into subtasks, assigns them to specialist models, and synthesizes results. Uses reinforcement learning (GRPO-style) to learn teamwork patterns.\n\n```bash\n# Orchestrate a complex task\ncurl -X POST http://localhost:6060/v1/orchestrate \\\n  -d '{\"prompt\": \"Write a Python web scraper, test it, and write documentation\"}'\n\n# View orchestration history\ncurl http://localhost:6060/v1/orchestration/history\n\n# Check GRPO learning stats\ncurl http://localhost:6060/v1/orchestration/grpo/stats\n```\n\n### Federated Routing\n\nMultiple Fugusashi instances collaboratively improve a shared routing model **without sharing prompts or data**.\nEach organization trains locally, adds differential privacy noise, and contributes weight updates.\nThe result: a router that's smarter than any single deployment.\n\n```bash\n# Register as a federated client\ncurl -X POST http://localhost:6060/v1/federated/register \\\n  -d '{\"client_id\": \"my-org\", \"metadata\": {\"type\": \"healthcare\"}}'\n\n# Submit local routing updates\ncurl -X POST http://localhost:6060/v1/federated/submit \\\n  -d '{\"client_id\": \"my-org\", \"weights\": [...], \"n_samples\": 1000}'\n\n# Trigger aggregation (requires min 3 clients)\ncurl -X POST http://localhost:6060/v1/federated/aggregate\n```\n\n### Routing Explanations\n\nEvery routing decision comes with a natural language explanation:\n\n```bash\ncurl -X POST http://localhost:6060/v1/explain \\\n  -d '{\"prompt\": \"Write a Python class for a binary tree\"}'\n```\n\nResponse:\n```\nDecision: Route to gpt-oss-120b (confidence: 87%)\nWhy: This prompt involves code generation. gpt-oss-120b is best suited for complex reasoning.\nAlternatives:\n  - lfm-2.5-1.2b (12%): better for fast responses\n  - hermes-3-405b (1%): better for creative writing\nLatency: 5.5ms | Strategy: cma-es\n```\n\nUsers can override decisions with natural language feedback, which becomes training data.\n\n---\n\n## API Reference\n\n### `POST /v1/chat/completions`\n\nOpenAI-compatible. Set `model: \"auto\"` for intelligent routing.\n\nResponse includes `routing_decision`:\n```json\n{\n  \"id\": \"fugu-698f0a66db98\",\n  \"model\": \"llama3.2-local\",\n  \"choices\": [...],\n  \"routing_decision\": {\n    \"model\": \"llama3.2-local\",\n    \"confidence\": 0.9,\n    \"strategy\": \"ensemble(cost)\",\n    \"latency_ms\": 0.05,\n    \"explanation\": \"Routed by capability fit + cost\"\n  }\n}\n```\n\n### `GET /v1/models` — List available models\n\n### `GET /v1/routing/decisions` — Recent routing decisions\n\n### `GET /v1/stats` — Aggregated stats (cost, tokens, per-model)\n\n### `GET /v1/trace/{request_id}` — Full request trace\n\n### `POST /v1/routing/training` — Seed similarity router\n\n```bash\ncurl -X POST http://localhost:6060/v1/routing/training \\\n  -H \"Content-Type: application/json\" \\\n  -d '[{\"prompt\":\"Write Python code\",\"model\":\"gpt-4o-mini\",\"score\":0.95}]'\n```\n\n### `POST /v1/feedback/rate` — Rate a response (1-5)\n\n```bash\ncurl -X POST http://localhost:6060/v1/feedback/rate \\\n  -d '{\"request_id\":\"fugu-698f0a66db98\",\"rating\":5}'\n```\n\n### `POST /v1/feedback/retrain` — Rebuild similarity index from feedback\n\n### `GET /v1/feedback/stats` — Outcome statistics\n\n### `GET /v1/feedback/rankings` — Per-model win rates\n\n### `POST /v1/orchestrate` — Multi-agent orchestration\n\n```bash\ncurl -X POST http://localhost:6060/v1/orchestrate \\\n  -d '{\"prompt\": \"Write code, test it, and document it\"}'\n```\n\n### `GET /v1/orchestration/history` — Orchestration history\n\n### `GET /v1/orchestration/grpo/stats` — GRPO learning stats\n\n---\n\n## Dashboard\n\nOpen `http://localhost:6060/dashboard` for a live view:\n\n- **Overview**: total requests, tokens, cost, avg routing latency\n- **Model Usage**: bar chart of which models are being picked\n- **Strategy Distribution**: cost vs similarity routing breakdown\n- **Recent Decisions**: live table of every routing decision\n\nAuto-refreshes every 3 seconds. Light and dark themes.\n\n---\n\n## Feedback Loop — The Killer Feature\n\nFugu's router is static. Fugusashi's **learns from every request**:\n\n```\nRoute → Execute → Evaluate → Learn → (repeat)\n```\n\n1. **Route**: Router picks a model for the prompt\n2. **Execute**: Model generates a response\n3. **Evaluate**: Track outcome (success/failure, cost, latency)\n4. **Learn**: Feed outcomes back into the similarity router\n5. **Auto-Retrain**: Every 10 requests, the similarity index rebuilds automatically\n\nOutcomes are stored in `.fugusashi_data/outcomes.jsonl` — inspectable, shareable, yours.\n\n---\n\n## Benchmarking\n\n```bash\npip install fugusashi\nfugusashi benchmark                      # Default 20-sample dataset\nfugusashi benchmark --train --verbose    # With training data\nfugusashi benchmark --train --json       # JSON output\nfugusashi benchmark -d my_data.jsonl     # Custom dataset\n```\n\n**Results:**\n\n| Metric | Without Training | With Training |\n|---|---|---|\n| Accuracy | 70% | **85%** |\n| Code accuracy | 60% | **90%** |\n| Strategy | 100% cost | 60% cost / 40% similarity |\n| Routing latency | \u003c1ms | ~18ms |\n\nCustom dataset format (JSONL):\n```jsonl\n{\"prompt\":\"How do I center a div?\",\"expected_model\":\"gpt-4o-mini\",\"category\":\"code\"}\n{\"prompt\":\"What is 2+2?\",\"expected_model\":\"llama3.2-local\",\"category\":\"factual\"}\n```\n\n---\n\n## Project Structure\n\n```\nfugusashi/\n├── config.yaml              # Model pool + routing config\n├── pyproject.toml           # Dependencies + metadata\n├── LIVING.md                # Living development walkthrough\n├── README.md                # This file\n├── LICENSE                  # MIT\n├── src/fugusashi/\n│   ├── __init__.py\n│   ├── __main__.py          # CLI: serve, benchmark, train, expand-data\n│   ├── server.py            # FastAPI app factory\n│   ├── config.py            # Pydantic config from YAML\n│   ├── providers.py         # LiteLLM multi-provider wrapper\n│   ├── tracker.py           # Cost/routing transparency\n│   ├── feedback.py          # Feedback loop + learning\n│   ├── benchmark.py         # Benchmark runner\n│   ├── training.py          # ModernBERT training pipeline\n│   ├── dataset.py           # Preference dataset management\n│   ├── orchestrator.py      # Multi-agent orchestrator\n│   ├── grpo.py              # GRPO-style reward learning\n│   ├── api/\n│   │   └── routes.py        # All API endpoints\n│   ├── router/\n│   │   ├── interface.py     # Abstract router protocol\n│   │   ├── strategies.py    # Cost, Similarity, Fallback routers\n│   │   ├── learned.py       # ModernBERT learned classifier router\n│   │   └── ensemble.py      # Priority-chain ensemble (learned → similarity → cost)\n│   └── static/\n│       └── dashboard.html   # Live web dashboard\n├── tests/\n│   ├── test_integration.py  # Integration tests\n│   ├── test_learned.py      # Learned router tests\n│   └── test_orchestrator.py # Orchestrator + GRPO tests\n└── docs/                    # GitHub Pages documentation\n```\n\n---\n\n## Why Fugusashi Over Fugu?\n\n1. **Transparent**: Every routing decision is visible and explainable. Fugu is a black box.\n2. **Self-hosting**: Runs entirely on-premise with local models via Ollama. Fugu is cloud-only.\n3. **Learning**: Gets smarter from your specific traffic via the feedback loop. Fugu is static.\n4. **Federated**: Multiple organizations can collaboratively improve routing without sharing data. Fugu doesn't offer this.\n5. **Free**: MIT licensed, no usage fees beyond model APIs. Fugu costs $5-30/M tokens + subscription.\n6. **EU-available**: No GDPR restrictions. Fugu is blocked in the EU/EEA.\n7. **Yours**: Community-owned preference datasets, customizable, extensible. Fugu is Sakana's.\n\n---\n\n## Paper\n\nThis project is accompanied by a research paper:\n\n\u003e **Fugusashi: Federated Learning of LLM Routing with Human-Interpretable Decisions**\n\nThe paper introduces three contributions:\n1. **Federated routing learning** — collaborative model routing without data sharing\n2. **Human-interpretable routing** — natural language explanations for every decision\n3. **CMA-ES adaptation** — continuous evolution of routing weights\n\n📄 [Read the paper](https://github.com/eulogik/fugusashi/blob/main/paper/main.tex)\n\n## Links\n\n| Resource | Link |\n|---|---|\n| 🌐 Website | [eulogik.com](https://eulogik.com) |\n| 💻 GitHub | [github.com/eulogik/fugusashi](https://github.com/eulogik/fugusashi) |\n| 📦 PyPI | [pypi.org/project/fugusashi](https://pypi.org/project/fugusashi/) |\n| 🤗 HF Model | [huggingface.co/eulogik/fugusashi-router](https://huggingface.co/eulogik/fugusashi-router) |\n| 📊 HF Dataset | [huggingface.co/datasets/eulogik/fugusashi-preferences](https://huggingface.co/datasets/eulogik/fugusashi-preferences) |\n| 🚀 HF Space | [huggingface.co/spaces/eulogik/fugusashi](https://huggingface.co/spaces/eulogik/fugusashi) |\n| 📖 Docs | [eulogik.github.io/fugusashi](https://eulogik.github.io/fugusashi/) |\n| 📝 Paper | [paper/main.tex](https://github.com/eulogik/fugusashi/blob/main/paper/main.tex) |\n| 🌍 eulogik | [eulogik.com](https://eulogik.com) |\n\n---\n\n## License\n\nMIT — use it however you want.\n\n---\n\n\u003cdiv align=\"center\"\u003e\n\n**Built with ❤️ by [eulogik](https://eulogik.com)**\n\n**[⭐ Star on GitHub](https://github.com/eulogik/fugusashi) · [🤗 HuggingFace](https://huggingface.co/eulogik/fugusashi-router) · [🐦 X/Twitter](https://x.com/eulogik) · [📦 Docker](https://github.com/orgs/eulogik/packages/container/fugusashi)**\n\n\u003c/div\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feulogik%2Ffugusashi","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feulogik%2Ffugusashi","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feulogik%2Ffugusashi/lists"}