{"id":48406528,"url":"https://github.com/omnidotdev/synapse","last_synced_at":"2026-04-06T03:37:17.390Z","repository":{"id":339226339,"uuid":"1150829709","full_name":"omnidotdev/synapse","owner":"omnidotdev","description":"🧠 Orchestrated intelligence","archived":false,"fork":false,"pushed_at":"2026-04-04T07:22:31.000Z","size":292,"stargazers_count":3,"open_issues_count":12,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-04-06T03:37:14.344Z","etag":null,"topics":["ai","gateway","inference","llm","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/omnidotdev.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null},"funding":{"github":["omnidotdev"],"custom":["https://omni.dev"]}},"created_at":"2026-02-05T18:34:13.000Z","updated_at":"2026-04-04T07:19:04.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/omnidotdev/synapse","commit_stats":null,"previous_names":["omnidotdev/synapse-gateway","omnidotdev/synapse"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/omnidotdev/synapse","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnidotdev%2Fsynapse","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnidotdev%2Fsynapse/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnidotdev%2Fsynapse/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnidotdev%2Fsynapse/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/omnidotdev","download_url":"https://codeload.github.com/omnidotdev/synapse/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/omnidotdev%2Fsynapse/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31458838,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T21:22:52.476Z","status":"online","status_checked_at":"2026-04-06T02:00:07.287Z","response_time":112,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","gateway","inference","llm","rust"],"created_at":"2026-04-06T03:37:16.375Z","updated_at":"2026-04-06T03:37:17.382Z","avatar_url":"https://github.com/omnidotdev.png","language":"Rust","funding_links":["https://github.com/sponsors/omnidotdev","https://omni.dev"],"categories":[],"sub_categories":[],"readme":"# Synapse\n\nUnified AI router for LLM, embeddings, image generation, MCP, STT, and TTS provider management. Configure your API keys once, route all AI traffic through a single gateway.\n\n## Features\n\n- **Multi-Provider LLM Routing** -- OpenAI, Anthropic, Google, AWS Bedrock with automatic failover\n- **Smart Model Selection** -- Threshold, cost, cascade, score, and ONNX-based routing strategies\n- **Embeddings \u0026 Image Generation** -- Unified endpoints for embedding and image generation providers\n- **MCP Aggregation** -- Aggregate Model Context Protocol servers through a single endpoint\n- **STT/TTS** -- Speech-to-text (Whisper, Deepgram) and text-to-speech (OpenAI TTS, ElevenLabs)\n- **Rate Limiting** -- In-memory and Redis-backed rate limiting with per-client policies\n- **Authentication** -- API key validation, OAuth2/JWT with JWKS, CSRF protection\n- **Billing Integration** -- Usage metering, managed margins, credit-based billing via Aether\n- **Telemetry** -- OpenTelemetry metrics, tracing, and structured logs\n\n## Quick Start\n\n### Install\n\n```bash\n# Build from source\ncargo build --release -p synapse\n\n# Or use Docker\ndocker pull ghcr.io/omnidotdev/synapse:latest\n```\n\n### Configure\n\nCreate `synapse.toml`:\n\n```toml\n[server]\nlisten_address = \"0.0.0.0:6000\"\n\n[llm.providers.anthropic]\ntype = \"anthropic\"\napi_key = \"{{ env.ANTHROPIC_API_KEY }}\"\n\n[llm.providers.openai]\ntype = \"openai\"\napi_key = \"{{ env.OPENAI_API_KEY }}\"\n```\n\nAPI keys support `{{ env.VAR }}` template syntax for environment variable substitution.\n\n### Run\n\n```bash\n./target/release/synapse --config synapse.toml\n```\n\n### Send a Request\n\n```bash\n# OpenAI-compatible\ncurl http://localhost:6000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"claude-sonnet-4-20250514\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n  }'\n\n# Anthropic-compatible\ncurl http://localhost:6000/v1/messages \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"claude-sonnet-4-20250514\",\n    \"max_tokens\": 1024,\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n  }'\n```\n\nWorks with existing OpenAI and Anthropic SDKs -- just point `base_url` at your Synapse instance.\n\n## Supported Providers\n\n| Modality | Providers |\n|----------|-----------|\n| LLM | Anthropic, OpenAI, Google, AWS Bedrock |\n| Embeddings | OpenAI |\n| Image Generation | OpenAI (DALL-E) |\n| STT | OpenAI Whisper, Deepgram |\n| TTS | OpenAI TTS, ElevenLabs |\n| MCP | Any STDIO, SSE, or StreamableHTTP server |\n\n## Routing Strategies\n\nSynapse can automatically select the best model for each request using virtual model names (`auto`, `fast`, `best`, `cheap`):\n\n| Strategy | Description |\n|----------|-------------|\n| **Threshold** | Route by query complexity -- cheap models for simple queries, strong models for complex ones |\n| **Cost** | Maximize quality within a per-request cost budget |\n| **Cascade** | Try a cheap model first, escalate to a stronger model on low-confidence responses |\n| **Score** | Multi-objective optimization balancing quality, cost, and latency |\n| **ONNX** | ML-based classification using a trained ONNX model |\n\n## Billing Modes\n\n| Mode | Description |\n|------|-------------|\n| **BYOK** | Bring your own provider API keys -- no token metering |\n| **Managed** | Synapse provides API keys and meters usage with configurable margins |\n| **Credits** | Prepaid credit balance deducted per request based on model pricing |\n\n## API Endpoints\n\n| Endpoint | Method | Description |\n|----------|--------|-------------|\n| `/v1/chat/completions` | POST | LLM chat (OpenAI-compatible, streaming) |\n| `/v1/messages` | POST | LLM chat (Anthropic-compatible, streaming) |\n| `/v1/models` | GET | List available models |\n| `/v1/embeddings` | POST | Generate embeddings |\n| `/v1/images/generations` | POST | Generate images |\n| `/v1/audio/transcriptions` | POST | Speech-to-text |\n| `/v1/audio/speech` | POST | Text-to-speech |\n| `/mcp/tools/list` | POST | List MCP tools |\n| `/mcp/tools/call` | POST | Execute an MCP tool |\n| `/health` | GET | Health check |\n\n## Configuration\n\nSee the [full documentation](https://omni.dev/grid/synapse/configuration) for all configuration options including:\n\n- Server settings (TLS, CORS, health endpoints)\n- Provider configuration per modality\n- Smart routing and model profiles\n- Rate limiting (memory and Redis)\n- Failover and circuit breaker\n- Authentication (API keys, JWT/JWKS)\n- Billing and usage metering\n- OpenTelemetry exporters\n\n## Architecture\n\n```\n┌──────────────────────────────┐\n│           Synapse            │ :6000\n│      Unified AI Router       │\n├──────────────────────────────┤\n│  ┌───┬───┬───┬───┬───┬───┐  │\n│  │LLM│EMB│IMG│MCP│STT│TTS│  │\n│  └─┬─┴─┬─┴─┬─┴─┬─┴─┬─┴─┬─┘  │\n└────┼───┼───┼───┼───┼───┼────┘\n     │   │   │   │   │   │\n     ▼   ▼   ▼   ▼   ▼   ▼\n Anthropic OpenAI Google Bedrock\n Deepgram  ElevenLabs  MCP Servers\n```\n\n## Contributing\n\nSee the [contributing guide](/.github/CONTRIBUTING.md).\n\n## License\n\nThe code in this repository is licensed under Apache 2.0, \u0026copy; [Omni LLC](https://omni.dev). See [LICENSE.md](LICENSE.md) for more information.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomnidotdev%2Fsynapse","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fomnidotdev%2Fsynapse","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fomnidotdev%2Fsynapse/lists"}