{"id":51123386,"url":"https://github.com/sunerpy/bedrock-gateway-rust","last_synced_at":"2026-06-25T05:01:30.374Z","repository":{"id":366048258,"uuid":"1273081097","full_name":"sunerpy/bedrock-gateway-rust","owner":"sunerpy","description":"A 100% Rust, OpenAI-compatible API gateway for AWS Bedrock — drop-in, single-binary, blazingly fast. Chat Completions, Responses API, embeddings, prompt caching \u0026 extended thinking.","archived":false,"fork":false,"pushed_at":"2026-06-20T03:47:28.000Z","size":333,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-06-20T04:14:22.173Z","etag":null,"topics":["api-gateway","aws-bedrock","axum","bedrock","claude","llm","openai","openai-api","rust","tokio"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/bedrock-gateway-rust","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit-0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sunerpy.png","metadata":{"files":{"readme":"README.md","changelog":"changelog/v0.1.0.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":"audit.toml","citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":"AGENTS.md","dco":null,"cla":null}},"created_at":"2026-06-18T07:34:21.000Z","updated_at":"2026-06-20T03:47:31.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/sunerpy/bedrock-gateway-rust","commit_stats":null,"previous_names":["sunerpy/bedrock-gateway-rust"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/sunerpy/bedrock-gateway-rust","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunerpy%2Fbedrock-gateway-rust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunerpy%2Fbedrock-gateway-rust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunerpy%2Fbedrock-gateway-rust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunerpy%2Fbedrock-gateway-rust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sunerpy","download_url":"https://codeload.github.com/sunerpy/bedrock-gateway-rust/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sunerpy%2Fbedrock-gateway-rust/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34760219,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-25T02:00:05.521Z","response_time":101,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["api-gateway","aws-bedrock","axum","bedrock","claude","llm","openai","openai-api","rust","tokio"],"created_at":"2026-06-25T05:01:29.673Z","updated_at":"2026-06-25T05:01:30.358Z","avatar_url":"https://github.com/sunerpy.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# bedrock-gateway-rust\n\n[![CI](https://github.com/sunerpy/bedrock-gateway-rust/actions/workflows/ci.yml/badge.svg)](https://github.com/sunerpy/bedrock-gateway-rust/actions/workflows/ci.yml)\n[![Docker Pulls](https://img.shields.io/docker/pulls/sunerpy/bedrock-gateway-rust)](https://hub.docker.com/r/sunerpy/bedrock-gateway-rust)\n[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)\n\n**A 100% Rust, OpenAI-compatible API gateway for AWS Bedrock — drop-in, single-binary, blazingly fast.**\n\n\u003e 📖 [中文文档 / Chinese README](docs/readme/README_CN.md)\n\nPoint any OpenAI SDK, tool, or agent at this gateway and it routes requests to AWS Bedrock without a single line of client-side changes. The runtime is **axum + tokio + aws-sdk-bedrockruntime** — fully async, no GC pauses, no Python dependency. It replaces an earlier Python/FastAPI implementation while preserving wire-exact OpenAI API compatibility.\n\n---\n\n## Highlights\n\n- **100% Rust** — single statically-linked binary, distroless Docker image (~12 MB compressed), no GC, low memory footprint, high concurrency under load.\n- **Wire-exact OpenAI compatibility** — existing OpenAI SDKs, clients, and agents connect with zero code changes. No invented top-level fields; Bedrock-only features go through the standard `extra_body` mechanism.\n- **OpenAI Responses API** — full support for `POST /api/v1/responses`, including streaming. Required by `codex` (`wire_api = \"responses\"`). Stateless; `store` and `previous_response_id` are accepted and silently ignored.\n- **Automatic prompt caching** — cache-point injection is on by default. The gateway places cache points across tools, system prompt, and messages automatically (up to `max_cache_checkpoints` per model). No client changes needed. Thresholds and limits are config-driven per model; a family fallback entry covers new Claude models automatically.\n- **Extended thinking / reasoning** — Claude `budget_tokens`, `adaptive_thinking`, and DeepSeek string-form reasoning all supported. Map OpenAI's `reasoning_effort` levels (`low` / `medium` / `high` / `xhigh` / `max`) through `extra_body`; the gateway picks the right Bedrock wire format per model.\n- **Cross-region inference profiles** — all seven geographic prefixes (`us.` / `eu.` / `apac.` / `jp.` / `au.` / `ca.` / `global.`) work transparently. Capability matching strips the prefix; Bedrock calls always use the original model ID.\n- **Zero-hardcoding** — all model knowledge lives in `config/models.toml`. Adding a new model or tuning cache thresholds never requires a recompile.\n- **Four deployment targets** — standalone binary, Docker, ECS/Fargate + ALB (CloudFormation one-click), and Lambda + Function URL (Lambda Web Adapter, no Lambda-specific Rust code).\n- **Dual auth, dual direction** — client-to-gateway via Bearer token (SSM / Secrets Manager / env); gateway-to-Bedrock via Bedrock API Key bearer or SigV4 credential chain.\n- **Structured observability** — per-request `request_id`, `cached_tokens`, `cache_hit`, `ttfb_ms`, `duration_ms` in structured JSON logs. No prompt text or secrets ever logged.\n\n---\n\n## Supported Endpoints\n\nAll endpoints are prefixed by `API_ROUTE_PREFIX` (default `/api/v1`).\n\n| Method | Path                       | Description                                                   |\n| ------ | -------------------------- | ------------------------------------------------------------- |\n| `POST` | `/api/v1/chat/completions` | Chat completions — streaming (SSE) and non-streaming          |\n| `POST` | `/api/v1/responses`        | OpenAI Responses API — stateless, streaming and non-streaming |\n| `POST` | `/api/v1/embeddings`       | Embeddings — Cohere, Titan, and Nova families                 |\n| `GET`  | `/api/v1/models`           | Live model catalog from Bedrock control plane                 |\n| `GET`  | `/api/v1/models/{id}`      | Single model lookup (supports inference profile IDs)          |\n| `GET`  | `/api/v1/health`           | Liveness probe — returns `200 OK`                             |\n\n---\n\n## Client Compatibility\n\n| Client          | Wire API                | Endpoint                        | Status       |\n| --------------- | ----------------------- | ------------------------------- | ------------ |\n| **opencode**    | OpenAI Chat Completions | `POST /api/v1/chat/completions` | ✅ Supported |\n| **hermes**      | OpenAI Chat Completions | `POST /api/v1/chat/completions` | ✅ Supported |\n| **codex**       | OpenAI Responses API    | `POST /api/v1/responses`        | ✅ Supported |\n| **claude code** | Anthropic Messages      | `POST /v1/messages`             | ⏳ Roadmap   |\n\n---\n\n## Quick Start\n\n### Prerequisites\n\n- AWS credentials available (instance role, `AWS_PROFILE`, or access key pair)\n- Bedrock model access enabled in your target region\n- Rust 1.80+ (to build from source) or Docker\n\n### 30-second start with Docker\n\n```bash\ndocker run \\\n  -e API_KEY=sk-my-secret-key \\\n  -e AWS_REGION=us-east-1 \\\n  -e AWS_BEARER_TOKEN_BEDROCK=bedrock-api-key-... \\\n  -p 8080:8080 \\\n  sunerpy/bedrock-gateway-rust\n```\n\nUsing an EC2 instance or ECS task with an IAM role? Omit `AWS_BEARER_TOKEN_BEDROCK` and the access key pair — the SDK picks up instance credentials automatically.\n\n### Build from source\n\n```bash\ncargo build --release\nAPI_KEY=sk-my-secret-key AWS_REGION=us-east-1 ./target/release/bedrock-gateway\n```\n\n### Local dev\n\n```bash\nAPI_KEY=testkey cargo run\n```\n\n### Verify\n\n```bash\ncurl http://localhost:8080/api/v1/health\n# OK\n\ncurl http://localhost:8080/api/v1/chat/completions \\\n  -H \"Authorization: Bearer sk-my-secret-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"anthropic.claude-3-5-sonnet-20241022-v2:0\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello!\"}]}'\n```\n\n### OpenAI Python SDK — drop-in replacement\n\n```python\nfrom openai import OpenAI\n\nclient = OpenAI(\n    base_url=\"http://localhost:8080/api/v1\",\n    api_key=\"sk-my-secret-key\",\n)\n\nresponse = client.chat.completions.create(\n    model=\"anthropic.claude-3-5-sonnet-20241022-v2:0\",\n    messages=[{\"role\": \"user\", \"content\": \"Explain serverless in one sentence.\"}],\n)\nprint(response.choices[0].message.content)\n```\n\nStreaming, tool calling, vision, and embeddings work the same way — no special configuration.\n\n---\n\n## Deployment\n\nFour targets share the same binary and the same environment-variable interface.\n\n| Target                | Guide                                                                          |\n| --------------------- | ------------------------------------------------------------------------------ |\n| Standalone binary     | Run `cargo build --release`, ship `target/release/bedrock-gateway` + `config/` |\n| Docker (distroless)   | Pull `sunerpy/bedrock-gateway-rust` — [Deploy → Docker](docs/deploy/docker.md) |\n| ECS / Fargate + ALB   | CloudFormation one-click — [Deploy → ECS / Fargate](docs/deploy/ecs.md)        |\n| Lambda + Function URL | Lambda Web Adapter, no Lambda code — [Deploy → Lambda](docs/deploy/lambda.md)  |\n\n### ECS / Fargate quick deploy\n\n```bash\naws cloudformation deploy \\\n  --template-file deployment/BedrockGatewayFargate.template \\\n  --stack-name bedrock-gateway \\\n  --capabilities CAPABILITY_IAM \\\n  --parameter-overrides \\\n    ApiKey=sk-my-secret-key \\\n    VpcId=vpc-... \\\n    SubnetIds=subnet-...,subnet-...\n```\n\n### Lambda quick deploy\n\n```bash\n# Build the Lambda container image\ndocker build -f deployment/lambda/Dockerfile -t bedrock-gateway:lambda .\n# Push to ECR, then:\naws cloudformation deploy \\\n  --template-file deployment/BedrockGatewayLambda.template \\\n  --stack-name bedrock-gateway-lambda \\\n  --capabilities CAPABILITY_IAM \\\n  --parameter-overrides \\\n    ImageUri=\u003cECR_URI\u003e:latest \\\n    ApiKeySecretArn=arn:aws:secretsmanager:...\n```\n\n\u003e Lambda has a 10-minute maximum timeout. For long-running streaming sessions, prefer ECS/Fargate.\n\n---\n\n## Configuration\n\nAll settings are environment variables. The gateway accepts both an `APP_` prefix and bare names (e.g., `API_KEY` or `APP_API_KEY`); bare names take precedence.\n\n### The three required variables\n\n| Variable                   | Purpose                                                                           |\n| -------------------------- | --------------------------------------------------------------------------------- |\n| `API_KEY`                  | Bearer token your clients send to the gateway. Pick any string.                   |\n| `AWS_BEARER_TOKEN_BEDROCK` | Bedrock API Key the gateway presents to AWS (recommended). Or omit and use SigV4. |\n| `AWS_REGION`               | AWS region for Bedrock calls. Defaults to `us-west-2`.                            |\n\n**These two keys go in opposite directions — don't mix them up:**\n\n- `API_KEY` is what your clients send _to_ the gateway.\n- `AWS_BEARER_TOKEN_BEDROCK` is what the gateway sends _to_ AWS Bedrock.\n\n### AWS credential options\n\nThe gateway auto-selects — no switch needed:\n\n1. `AWS_BEARER_TOKEN_BEDROCK` (alias `BEDROCK_API_KEY`) — Bedrock API Key, sent as `Authorization: Bearer`. Bypasses SigV4 entirely. Recommended for new deployments.\n2. SigV4 credential chain — resolves `AWS_ACCESS_KEY_ID` + `AWS_SECRET_ACCESS_KEY`, then `AWS_PROFILE`, then IMDS / ECS task role.\n\n### Production secret management\n\nPrefer fetching `API_KEY` from a secrets store. Priority order:\n\n1. `API_KEY_PARAM_NAME` — SSM Parameter Store\n2. `API_KEY_SECRET_ARN` — Secrets Manager (secret must contain an `api_key` field)\n3. `API_KEY` — plaintext env var\n\n### All optional variables\n\n\u003cdetails\u003e\n\u003csummary\u003eShow all optional variables\u003c/summary\u003e\n\n**Model defaults**\n\n| Variable                  | Default                                     | Description                                  |\n| ------------------------- | ------------------------------------------- | -------------------------------------------- |\n| `DEFAULT_MODEL`           | `anthropic.claude-3-5-sonnet-20241022-v2:0` | Model used when the client omits `model`     |\n| `DEFAULT_EMBEDDING_MODEL` | `cohere.embed-multilingual-v3`              | Embedding model used when `model` is omitted |\n\n**Feature flags**\n\n| Variable                                | Default  | Description                                                  |\n| --------------------------------------- | -------- | ------------------------------------------------------------ |\n| `ENABLE_CROSS_REGION_INFERENCE`         | `true`   | Transparently route cross-region inference profiles          |\n| `ENABLE_APPLICATION_INFERENCE_PROFILES` | `true`   | Surface inference profile metadata in `GET /models`          |\n| `ENABLE_PROMPT_CACHING`                 | `true`   | Auto-inject cache points for Claude and Nova models          |\n| `CONFIG_DIR`                            | `config` | External config directory path (overrides embedded defaults) |\n\n**Server**\n\n| Variable           | Default   | Description                   |\n| ------------------ | --------- | ----------------------------- |\n| `PORT`             | `8080`    | HTTP listen port              |\n| `BIND_ADDR`        | `0.0.0.0` | Network interface to bind     |\n| `API_ROUTE_PREFIX` | `/api/v1` | Path prefix for all endpoints |\n\n**Logging**\n\n| Variable    | Default | Description                                              |\n| ----------- | ------- | -------------------------------------------------------- |\n| `LOG_LEVEL` | `info`  | Verbosity: `trace` / `debug` / `info` / `warn` / `error` |\n| `DEBUG`     | `false` | Enable verbose error details in responses                |\n\n**Timeouts and retries**\n\n| Variable                   | Default | Description                                               |\n| -------------------------- | ------- | --------------------------------------------------------- |\n| `AWS_CONNECT_TIMEOUT_SECS` | `60`    | TCP connection timeout to AWS                             |\n| `AWS_READ_TIMEOUT_SECS`    | `900`   | Response read timeout (15 min, accommodates long streams) |\n| `AWS_MAX_RETRY_ATTEMPTS`   | `8`     | Retries on transient throttling or 5xx failures           |\n\n\u003c/details\u003e\n\n### Config files (no recompile needed)\n\nConfig files live in `config/` alongside the binary and are read at startup. They are also embedded in the binary as a fallback — the gateway always starts, even if the files are absent.\n\n| File                     | Purpose                                                                        |\n| ------------------------ | ------------------------------------------------------------------------------ |\n| `config/models.toml`     | Model capability registry: capability flags, reasoning paths, cache thresholds |\n| `config/regions.toml`    | Cross-region routing rules                                                     |\n| `config/embeddings.toml` | Embedding model registry (Cohere / Titan / Nova families)                      |\n| `config/app.toml`        | Application defaults — lowest priority                                         |\n\n**Adding a new model** requires only a `config/models.toml` entry:\n\n```toml\n[[model]]\nmatch = \"your-provider.your-model-id\"\ncapabilities = []\n[model.params]\nmax_tokens = 8192\ncontext_window = 200000\n# reasoning_path = \"budget_tokens\"  # uncomment if the model supports extended thinking\n```\n\n---\n\n## API Usage\n\nAll endpoints require `Authorization: Bearer \u003cAPI_KEY\u003e`.\n\nBase URL: `http://localhost:8080/api/v1`\n\n### Streaming chat\n\n```bash\ncurl http://localhost:8080/api/v1/chat/completions \\\n  -H \"Authorization: Bearer sk-my-secret-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"anthropic.claude-3-5-sonnet-20241022-v2:0\",\"messages\":[{\"role\":\"user\",\"content\":\"Count to 5 slowly.\"}],\"stream\":true}'\n```\n\n### Embeddings\n\n```bash\ncurl http://localhost:8080/api/v1/embeddings \\\n  -H \"Authorization: Bearer sk-my-secret-key\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"cohere.embed-multilingual-v3\",\"input\":[\"Hello world\",\"Bedrock gateway\"]}'\n```\n\n### Extended thinking\n\n```python\nresponse = client.chat.completions.create(\n    model=\"anthropic.claude-3-5-sonnet-20241022-v2:0\",\n    messages=[{\"role\": \"user\", \"content\": \"Solve step by step: ...\"}],\n    extra_body={\"reasoning_effort\": \"high\"},  # none|minimal|low|medium|high|xhigh|max\n)\n# Reasoning appears inline as \u003cthink\u003e...\u003c/think\u003e inside content\n```\n\n### codex configuration\n\nAdd to `~/.codex/config.toml`:\n\n```toml\n[model_providers.bgw]\nname = \"Bedrock Gateway\"\nbase_url = \"http://localhost:8080/api/v1\"\nwire_api = \"responses\"\nrequires_openai_auth = false\n\n[model_providers.bgw.env_key]\nname = \"BGW_API_KEY\"\ndescription = \"Bearer token for the Bedrock gateway\"\n```\n\n```bash\nexport BGW_API_KEY=sk-my-gateway-key\ncodex --provider bgw --model anthropic.claude-3-5-sonnet-20241022-v2:0\n```\n\n---\n\n## Architecture\n\n```\nsrc/\n├── main.rs / lib.rs         # Entry point, crate root\n├── error.rs                 # AppError, OpenAI error envelope\n├── telemetry.rs             # tracing subscriber, dynamic log level\n├── openai/                  # Wire types: ChatRequest, ChatResponse, ResponsesRequest, ...\n├── domain/                  # Provider-agnostic traits: ChatProvider, ResponsesProvider, ...\n├── config/                  # Settings, ModelCapabilityConfig, RegionRoutingConfig, ...\n├── bedrock/                 # AWS Bedrock backend: translate, cache, reasoning, stream, ...\n└── server/                  # axum router, auth middleware, AppState\n```\n\nThe codebase is strictly layered — dependencies flow downward only. The Bedrock backend implements the domain traits; adding a non-Bedrock backend means implementing those traits without touching any existing code.\n\nFor the full architecture reference, contract rules, and contributor guidelines, see the [Architecture \u0026 Contracts guide](AGENTS.md).\n\nFor caching behavior, reasoning budget paths, and cross-region inference profile details, see the [Caching \u0026 Reasoning deep dive](docs/caching-and-reasoning.md).\n\n---\n\n## Supported Models\n\nThe authoritative list is `config/models.toml` and the live `GET /api/v1/models` endpoint. The registry currently covers:\n\n- **Claude** — Sonnet 4.x, Haiku 4.x, Opus 4.x (via Bedrock model IDs and cross-region inference profiles)\n- **Amazon Nova** — multimodal and text models\n- **DeepSeek** — v3 (string-form reasoning path)\n- Any Bedrock foundation model or inference profile accessible in your account — the catalog refreshes from the control plane at startup\n\nAdding a model requires only a `config/models.toml` entry and no recompile.\n\n---\n\n## Build, Test, and Contributing\n\n```bash\ncargo build --release          # release binary → target/release/bedrock-gateway\ncargo test                     # unit + golden replay tests (no AWS credentials needed)\ncargo clippy --all-targets --all-features -- -D warnings\ncargo fmt\n```\n\n**Pre-commit gate (mandatory before every commit):**\n\n```bash\ncargo fmt \u0026\u0026 cargo clippy --all-targets --all-features -- -D warnings \u0026\u0026 cargo test\n```\n\nGolden fixtures in `tests/golden/` are pinned offline and run in CI without AWS credentials. Live integration tests require real credentials:\n\n```bash\nBEDROCK_INTEGRATION=1 AWS_PROFILE=us cargo test -- --ignored\n```\n\nSee the [Contributing Guide](docs/readme/CONTRIBUTING.md) for contributor guidelines.\n\n---\n\n## Links\n\n| Resource                   | Link                                                           |\n| -------------------------- | -------------------------------------------------------------- |\n| 中文文档 / Chinese README  | [Read in Chinese](docs/readme/README_CN.md)                    |\n| Contributing               | [Contributing Guide](docs/readme/CONTRIBUTING.md)              |\n| Code of Conduct            | [Code of Conduct](docs/readme/CODE_OF_CONDUCT.md)              |\n| Caching and reasoning      | [Caching \u0026 Reasoning deep dive](docs/caching-and-reasoning.md) |\n| Docker deployment          | [Deploy → Docker](docs/deploy/docker.md)                       |\n| ECS/Fargate deployment     | [Deploy → ECS / Fargate](docs/deploy/ecs.md)                   |\n| Lambda deployment          | [Deploy → Lambda](docs/deploy/lambda.md)                       |\n| Architecture and contracts | [Architecture \u0026 Contracts guide](AGENTS.md)                    |\n\n---\n\n## License\n\n[MIT](LICENSE)\n\n---\n\n## Acknowledgements\n\nThis project is inspired by the Python gateway concept in\n[aws-samples/bedrock-access-gateway](https://github.com/aws-samples/bedrock-access-gateway).\nThe Rust implementation is a full rewrite with wire-exact OpenAI compatibility, automatic\nprompt caching, Responses API support, and production-grade observability.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunerpy%2Fbedrock-gateway-rust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsunerpy%2Fbedrock-gateway-rust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsunerpy%2Fbedrock-gateway-rust/lists"}