{"id":28533494,"url":"https://github.com/epam/ai-dial-adapter-bedrock","last_synced_at":"2026-01-17T07:38:42.231Z","repository":{"id":199527132,"uuid":"693244830","full_name":"epam/ai-dial-adapter-bedrock","owner":"epam","description":"The project implements AI DIAL API for language models from AWS Bedrock","archived":false,"fork":false,"pushed_at":"2026-01-16T11:10:50.000Z","size":2663,"stargazers_count":12,"open_issues_count":13,"forks_count":5,"subscribers_count":17,"default_branch":"development","last_synced_at":"2026-01-17T00:59:34.491Z","etag":null,"topics":["ai-dial","llm"],"latest_commit_sha":null,"homepage":"https://dialx.ai","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-09-18T16:28:49.000Z","updated_at":"2026-01-15T17:12:53.000Z","dependencies_parsed_at":"2023-12-20T12:33:17.726Z","dependency_job_id":"b1dd82e5-8bed-417c-b64b-c0dad811647b","html_url":"https://github.com/epam/ai-dial-adapter-bedrock","commit_stats":null,"previous_names":["epam/ai-dial-adapter-bedrock"],"tags_count":48,"template":false,"template_full_name":null,"purl":"pkg:github/epam/ai-dial-adapter-bedrock","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-bedrock","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-bedrock/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-bedrock/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-bedrock/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epam","download_url":"https://codeload.github.com/epam/ai-dial-adapter-bedrock/tar.gz/refs/heads/development","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-bedrock/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28504356,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-17T06:57:29.758Z","status":"ssl_error","status_checked_at":"2026-01-17T06:56:03.931Z","response_time":85,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-dial","llm"],"created_at":"2025-06-09T17:07:16.676Z","updated_at":"2026-01-17T07:38:42.190Z","avatar_url":"https://github.com/epam.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  DIAL Bedrock Adapter\n\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n  \u003cp align=\"center\"\u003e\n  \u003ca href=\"https://dialx.ai/\"\u003e\n    \u003cimg src=\"https://dialx.ai/dialx_logo.svg\" alt=\"About DIALX\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003ch4 align=\"center\"\u003e\n  \u003ca href=\"https://discord.gg/ukzj9U9tEe\"\u003e\n    \u003cimg src=\"https://img.shields.io/static/v1?label=DIALX%20Community%20on\u0026message=Discord\u0026color=blue\u0026logo=Discord\u0026style=flat-square\" alt=\"Discord\"\u003e\n  \u003c/a\u003e\n\u003c/h4\u003e\n\n- [Overview](#overview)\n  - [Supported models](#supported-models)\n    - [Chat completion models](#chat-completion-models)\n      - [Implementation basis](#implementation-basis)\n      - [Configurable models](#configurable-models)\n        - [Converse API models](#converse-api-models)\n          - [Performance configuration](#performance-configuration)\n          - [Guardrail configuration](#guardrail-configuration)\n          - [Claude 3/4 models](#claude-34-models)\n        - [Claude 3.7 Sonnet](#claude-37-sonnet)\n        - [Claude models](#claude-models)\n        - [Stability AI models](#stability-ai-models)\n      - [Prompt caching](#prompt-caching)\n      - [Cross-region inference](#cross-region-inference)\n    - [Embedding models](#embedding-models)\n  - [Environment Variables](#environment-variables)\n    - [Resource limits](#resource-limits)\n    - [Default `max_tokens` for Claude models](#default-max_tokens-for-claude-models)\n  - [Compatibility mode](#compatibility-mode)\n    - [Compatibility configuration in DIAL Core config](#compatibility-configuration-in-dial-core-config)\n    - [Compatibility configuration in Adapter](#compatibility-configuration-in-adapter)\n  - [Load balancing](#load-balancing)\n  - [Authentication](#authentication)\n    - [AWS Bedrock](#aws-bedrock)\n    - [Anthropic API](#anthropic-api)\n  - [Development](#development)\n    - [Development environment](#development-environment)\n    - [IDE configuration](#ide-configuration)\n    - [Make on Windows](#make-on-windows)\n    - [Run](#run)\n    - [Lint](#lint)\n    - [Test](#test)\n    - [Clean](#clean)\n\n---\n\n# Overview\n\nLLM Adapters unify the APIs of respective LLMs to align with the Unified Protocol of DIAL Core. Each Adapter operates within a dedicated container. Multi-modality allows supporting non-textual communications such as image-to-text, text-to-image, file transfers and more.\n\nThe project implements [AI DIAL API](https://dialx.ai/dial_api) for language models and embedding models from [AWS Bedrock](https://aws.amazon.com/bedrock/).\n\n![ai-dial-core](https://docs.dialx.ai/assets/images/adapters-62587fb74cfb1c4225c20c08273ec5bc.svg)\n\n---\n\n## Supported models\n\n### Chat completion models\n\nThe following models support `POST SERVER_URL/openai/deployments/MODEL_ID/chat/completions` endpoint along with an optional support of `POST /tokenize` and `POST /truncate_prompt` endpoints:\n\nNote that a model supports `/truncate_prompt` endpoint if and only if it supports `max_prompt_tokens` request parameter.\n\n|Vendor|Model|Deployment name|Modality|`/tokenize`|`/truncate_prompt`, `max_prompt_tokens`|tools/functions|`/configuration`|Implementation|\n|---|---|---|---|---|---|---|---|---|\n|Anthropic|Claude 4.5 Sonnet|anthropic.claude-sonnet-4-5-20250929-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 4.5 Haiku|anthropic.claude-haiku-4-5-20251001-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 4.1 Opus|anthropic.claude-opus-4-1-20250805-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 4 Opus|anthropic.claude-opus-4-20250514-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 4 Sonnet|anthropic.claude-sonnet-4-20250514-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 3.7 Sonnet|anthropic.claude-3-7-sonnet-20250219-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 3.5 Sonnet|anthropic.claude-3-5-sonnet-20240620-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 3.5 Sonnet 2.0|anthropic.claude-3-5-sonnet-20241022-v2:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 3.5 Haiku|anthropic.claude-3-5-haiku-20241022-v1:0|(text/document)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 3 Sonnet|anthropic.claude-3-sonnet-20240229-v1:0|(text/image)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 3 Haiku|anthropic.claude-3-haiku-20240307-v1:0|(text/image)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|Anthropic|Claude 3 Opus|anthropic.claude-3-opus-20240229-v1:0|(text/image)-to-text|🟡|🟡|✅|✅|Anthropic SDK/Converse API|\n|DeepSeek|DeepSeek R1|deepseek.r1-v1:0|text-to-text|🟡|🟡|❌|✅|Converse API|\n|Meta|Llama 4 Chat Scout 17B Instruct|meta.llama4-scout-17b-instruct-v1:0|(text/image)-to-text|🟡|🟡|✅|✅|Converse API|\n|Meta|Llama 4 Chat Maverick 17B Instruct|meta.llama4-maverick-17b-instruct-v1:0|(text/image)-to-text|🟡|🟡|✅|✅|Converse API|\n|Meta|Llama 3.3 70B Instruct|meta.llama3-3-70b-instruct-v1:0|text-to-text|🟡|🟡|✅|✅|Converse API|\n|Meta|Llama 3.2 90B Instruct|meta.llama3-2-90b-instruct-v1:0|(text/image)-to-text|🟡|🟡|✅|✅|Converse API|\n|Meta|Llama 3.2 11B Instruct|meta.llama3-2-11b-instruct-v1:0|(text/image)-to-text|🟡|🟡|❌|✅|Converse API|\n|Meta|Llama 3.2 3B Instruct|meta.llama3-2-3b-instruct-v1:0|text-to-text|🟡|🟡|❌|✅|Converse API|\n|Meta|Llama 3.2 1B Instruct|meta.llama3-2-1b-instruct-v1:0|text-to-text|🟡|🟡|❌|✅|Converse API|\n|Meta|Llama 3.1 405B Instruct|meta.llama3-1-405b-instruct-v1:0|text-to-text|🟡|🟡|✅|✅|Converse API|\n|Meta|Llama 3.1 70B Instruct|meta.llama3-1-70b-instruct-v1:0|text-to-text|🟡|🟡|✅|✅|Converse API|\n|Meta|Llama 3.1 8B Instruct|meta.llama3-1-8b-instruct-v1:0|text-to-text|🟡|🟡|❌|✅|Converse API|\n|Meta|Llama 3 Chat 70B Instruct|meta.llama3-70b-instruct-v1:0|text-to-text|🟡|🟡|❌|✅|Converse API|\n|Meta|Llama 3 Chat 8B Instruct|meta.llama3-8b-instruct-v1:0|text-to-text|🟡|🟡|❌|✅|Converse API|\n|Stability AI|Stable Diffusion 3.5 Large|stability.sd3-5-large-v1:0|(text/image)-to-image|❌|🟡|❌|✅|Bedrock API|\n|Stability AI|Stable Image Ultra 1.0|stability.stable-image-ultra-v1:1|text-to-image|❌|🟡|❌|✅|Bedrock API|\n|Stability AI|Stable Image Core 1.0|stability.stable-image-core-v1:1|text-to-image|❌|🟡|❌|✅|Bedrock API|\n|Amazon|Nova Pro|amazon.nova-pro-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Converse API|\n|Amazon|Nova Lite|amazon.nova-lite-v1:0|(text/image/document)-to-text|🟡|🟡|✅|✅|Converse API|\n|Amazon|Nova Micro|amazon.nova-micro-v1:0|text-to-text|🟡|🟡|❌|✅|Converse API|\n|AI21 Labs|Jamba 1.5 Large|ai21.jamba-1-5-large-v1:0|text-to-text|🟡|🟡|✅|✅|Converse API|\n|AI21 Labs|Jamba 1.5 Mini|ai21.jamba-1-5-mini-v1:0|text-to-text|🟡|🟡|✅|✅|Converse API|\n|Cohere|Command R|cohere.command-r-v1:0|(text/document)-to-text|🟡|🟡|✅|✅|Converse API|\n|Cohere|Command R+|cohere.command-r-plus-v1:0|(text/document)-to-text|🟡|🟡|✅|✅|Converse API|\n\n✅, 🟡, and ❌ denote degrees of support of the given feature:\n\n||`/tokenize`, `/truncate_prompt`, `max_prompt_token`|tools/functions|`/configuration`|\n|---|---|---|---|\n|✅|Fully supported via an official tokenization algorithm|Fully supported via native tools API or official prompts to enable tools|Configurable via the `/configuration` endpoint|\n|🟡|Partially supported, because tokenization algorithm wasn't made public by the model vendor.\u003cbr\u003eAn approximate tokenization algorithm is used instead.\u003cbr\u003eIt conservatively counts **every byte in UTF-8 encoding of a string as a single token**.|Partially supported, because the model doesn't support tools natively.\u003cbr\u003ePrompt engineering is used instead to emulate tools, which may not be very reliable.|Not applicable|\n|❌|Not supported|Not supported|Not configurable|\n\n#### Implementation basis\n\nThe model adapters differ in what SDKs/APIs they are based on:\n\n1. [Converse API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_Converse.html) - the single API unifying different chat completion models\n2. [Bedrock API](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock-runtime/client/invoke_model.html) - the original Bedrock API for calling chat completion models\n3. [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-python) - the SDK for Anthropic Claude models that provides finer control over the model than the Converse API.\n\n---\n\n#### Configurable models\n\nCertain models support configuration via the `/configuration` endpoint.\nGET request to this endpoint returns the schema of the model configuration in [JSON Schema](https://json-schema.org/) format.\nSuch models expect that `custom_fields.configuration` field of the `chat/completions` request will contain a JSON value that conforms to the schema.\nThe `custom_fields.configuration` field is optional iff each field in the schema is optional too.\n\n##### Converse API models\n\n###### Performance configuration\n\nModels accept a configuration parameter that enables the [optimized latency mode](https://docs.aws.amazon.com/bedrock/latest/userguide/latency-optimized-inference.html):\n\n|Configuration|Comment|\n|---|---|\n|`{\"performanceConfig\": {\"latency\":\"standard\"}}`|Default latency|\n|`{\"performanceConfig\": {\"latency\":\"optimized\"}}`|Optimized latency|\n\n\u003e [!NOTE]\n\u003e Not all Bedrock models actually support the optimized latency mode. Check the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/latency-optimized-inference.html) before use.\n\n###### Guardrail configuration\n\nModels accept a configuration parameter that enables guardrails for the given request:\n\n```json\n{\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"hello\"\n    }\n  ],\n  \"custom_fields\": {\n    \"configuration\": {\n      \"guardrailConfig\": {\n        \"guardrailIdentifier\": \"(identifier)\",\n        \"guardrailVersion\": \"(version)\",\n        \"streamProcessingMode\": \"sync | async (opt)\",\n        \"trace\": \"enabled | disabled | enabled_full (opt)\"\n      }\n    }\n  }\n}\n```\n\nThe configuration is identical to the [GuardrailStreamConfiguration](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_GuardrailStreamConfiguration.html) object in the Converse API.\n\nLimitations:\n\n1. [Evaluation](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-converse-api.html#guardrails-use-converse-api-call) of a specific part of the chat completion request isn't supported.\n2. The [trace](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-use-converse-api.html#guardrails-use-converse-api-response) provided by the Bedrock Guardrail isn't attached to the response. When guardrail intervenes, the adapter returns an error with `code=content_filter`.\n\n###### Claude 3/4 models\n\nThe default adapter for **Claude 3/4** models is based on the [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-python) that [doesn't support](https://github.com/anthropics/anthropic-sdk-python/issues/971) optimized latency mode.\nwhen Converse API specific configuration is enabled, the adapter automatically switches the models to Converse API.\nWhen it happens, you are forfeiting all the features exclusive to the Anthropic SDK. Namely:\n\n1. Support of `tool_choice=none`\n2. Support of the Claude configurations [1](#claude-37-sonnet) [2](#claude-models)\n\n##### Claude 3.7 Sonnet\n\nThe model accepts optional configuration that enables [thinking feature](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking):\n\n|Configuration|Comment|\n|---|---|\n|`{\"thinking\": {\"type\": \"enabled\", \"budget_tokens\": 1024}}`|Thinking enabled with the given limit on reasoning tokens|\n|`{\"thinking\": {\"type\": \"disabled\"}}`|Thinking disabled|\n\n##### Claude models\n\nThe Claude models accept an optional list of beta feature flags.\nThe whole list of flags could be found in the [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-python/blob/main/src/anthropic/types/anthropic_beta_param.py).\n\n|Beta flag|Comment|Scope|\n|---|---|---|\n|`{\"betas\": [\"token-efficient-tools-2025-02-19\"]}`|[Token-efficient tool use](https://docs.anthropic.com/en/docs/build-with-claude/tool-use/token-efficient-tool-use)|Claude 3.7 Sonnet|\n|`{\"betas\": [\"output-128k-2025-02-19\"]}`|[Extended output length](https://docs.anthropic.com/en/docs/build-with-claude/extended-thinking#extended-output-capabilities-beta)|Claude 3.7 Sonnet|\n\nNot every model supports all flags. Refer to the official documentation before utilizing any flags.\n\n##### Stability AI models\n\nThe models accept optional configuration with the following fields:\n\n- `aspect_ratio: str` - one of \"16:9\", \"1:1\", \"21:9\", \"2:3\", \"3:2\", \"4:5\", \"5:4\", \"9:16\", \"9:21\"\n- `negative_prompt: str` - a prompt to be used for negative examples\n\n#### Prompt caching\n\nCertain chat completion models support prompt caching via cache breakpoint inserted in tool definitions or request messages.\n\nThe adapter supports cache breakpoint for the models based on Converse API and Claude 3 models.\n\n\u003cdetails\u003e\u003csummary\u003eSystem cache breakpoint\u003c/summary\u003e\n\n```json\n{\n  \"messages\": [\n    {\n      \"role\": \"system\",\n      \"content\": \"Long system prompt\",\n      \"custom_fields\": {\n        \"cache_breakpoint\": {}\n      }\n    },\n    {\n      \"role\": \"user\",\n      \"content\": \"user query\"\n    }\n  ]\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eMessage cache breakpoint\u003c/summary\u003e\n\n```json\n{\n  \"messages\": [\n    {\n      \"role\": \"system\",\n      \"content\": \"System prompt\"\n    },\n    {\n      \"role\": \"user\",\n      \"content\": \"user query\"\n      \"custom_fields\": {\n        \"cache_breakpoint\": {}\n      }\n    }\n  ]\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eTools cache breakpoint\u003c/summary\u003e\n\n```json\n{\n  \"tools\": [\n    {\n      \"type\": \"function\",\n      \"name\": \"get_weather\",\n      \"description\": \"Get current temperature for a given location.\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\n          \"location\": {\n            \"type\": \"string\",\n            \"description\": \"City and country e.g. Bogotá, Colombia\"\n          }\n        },\n        \"required\": [\n          \"location\"\n        ]\n      },\n      \"custom_fields\": {\n        \"cache_breakpoint\": {}\n      }\n    }\n  ],\n  \"messages\": [\n    {\n      \"role\": \"system\",\n      \"content\": \"System prompt\"\n    },\n    {\n      \"role\": \"user\",\n      \"content\": \"user query\"\n    }\n  ]\n}\n```\n\n\u003c/details\u003e\n\n\u003e [!NOTE]\n\u003e Not every model supports prompt caching. Refer to the [official documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/prompt-caching.html#prompt-caching-models) before utilizing any cache breakpoints.\n\n#### Cross-region inference\n\nThe adapter supports cross-region inference for US, EU and APAC regions for the listed models.\n\nE.g. `Claude 3.5 Sonnet 2.0` model can be accessed via the following deployment names:\n\n1. `anthropic.claude-3-5-sonnet-20241022-v2:0`\n2. `us.anthropic.claude-3-5-sonnet-20241022-v2:0`\n3. `eu.anthropic.claude-3-5-sonnet-20241022-v2:0`\n4. `apac.anthropic.claude-3-5-sonnet-20241022-v2:0`\n\n[Check](https://docs.aws.amazon.com/bedrock/latest/userguide/inference-profiles-support.html#inference-profiles-support-system) that your AWS Bedrock account supports cross-region inference for a particular model before using it.\n\n---\n\n### Embedding models\n\nThe following models support `SERVER_URL/openai/deployments/MODEL_ID/embeddings` endpoint:\n\n|Model|Deployment name|Modality|\n|---|---|---|\n|Titan Multimodal Embeddings Generation 1 (G1)|amazon.titan-embed-image-v1|image/text-to-embedding|\n|Amazon Titan Text Embeddings V2|amazon.titan-embed-text-v2:0|text-to-embedding|\n|Titan Embeddings G1 – Text v1.2|amazon.titan-embed-text-v1|text-to-embedding|\n|Cohere Embed English|cohere.embed-english-v3|text-to-embedding|\n|Cohere Multilingual|cohere.embed-multilingual-v3|text-to-embedding|\n\n---\n\n## Environment Variables\n\nCopy `.env.example` to `.env` and customize it for your environment:\n\n|Variable|Default|Description|\n|---|---|---|\n|AWS_ACCESS_KEY_ID|NA|AWS credentials with an access to the Bedrock service|\n|AWS_SECRET_ACCESS_KEY|NA|AWS credentials with an access to the Bedrock service|\n|AWS_SESSION_TOKEN|NA|AWS session token with an access the Bedrock service|\n|AWS_DEFAULT_REGION||AWS region e.g. `us-east-1`|\n|AWS_ASSUME_ROLE_ARN|| AWS assume role ARN e.g. `arn:aws:iam::123456789012:role/RoleName`|\n|LOG_LEVEL|INFO|Log level. Use DEBUG for dev purposes and INFO in prod|\n|AIDIAL_LOG_LEVEL|WARNING|AI DIAL SDK log level|\n|DIAL_URL||URL of the core DIAL server. If defined, images generated by Stability are uploaded to the DIAL file storage and attachments are returned with URLs pointing to the images. Otherwise, the images are returned as base64 encoded strings.|\n|WEB_CONCURRENCY|1|Number of workers for the server|\n|COMPATIBILITY_MAPPING|{}|**Deprecated** in favour of [compatibility configuration in DIAL Core config](#compatibility-configuration-in-dial-core-config). A JSON dictionary that maps Bedrock deployments that **aren't supported** by the Adapter to the Bedrock deployments that **are supported** by the Adapter _(see the [Supported models](#supported-models)_ section). Find more details in the [compatibility mode](#compatibility-configuration-in-adapter) section.|\n|CLAUDE_DEFAULT_MAX_TOKENS|1536|The default value of `max_tokens` chat completion parameter if it is not provided in the request.\u003cbr\u003e**:warning: Using the variable is discouraged**.\u003cbr\u003eConsider configuring the default in the DIAL Core Config instead as demonstrated in the [example below](#default-max_tokens-for-claude-models).|\n|BOTOCORE_MAX_RETRY_ATTEMPTS|0|How many times to retry chat model requests made via the Bedrock API or Converse API when the provider returns a retriable error|\n|ANTHROPIC_MAX_RETRY_ATTEMPTS|0|How many times to retry Anthropic chat model requests when the provider returns a retriable error|\n\n### Resource limits\n\nThe following environment variables reveal adapter's implementation details and therefore are more susceptible to changes in future than the variables discussed so far.\n\n:warning: Don't use the variables unless you are absolutely sure you know what you are doing.\n\n|Variable|Applicable to models implemented via|Default|Description|\n|---|---|---|---|\n|ANTHROPIC_MAX_CONNECTIONS|[Anthropic SDK](#implementation-basis)|1000|The maximum number of concurrent requests. Corresponds to `max_connections` [parameter](https://www.python-httpx.org/advanced/resource-limits/) of the HTTPX client.|\n|ANTHROPIC_MAX_KEEPALIVE_CONNECTIONS|[Anthropic SDK](#implementation-basis)|100|The maximum number of idle connections kept in a connection pool. Corresponds to the `max_keepalive_connections` [parameter](https://www.python-httpx.org/advanced/resource-limits/) of the HTTPX client.|\n|BOTOCORE_CLIENT_MAX_POOL_CONNECTIONS|[Bedrock API \u0026 Conserve API](#implementation-basis)|1000|The maximum number of connections kept in a connection pool.|\n\n### Default `max_tokens` for Claude models\n\nUnlike OpenAI models, Claude models require the `max_tokens` parameter in the chat completion request.\n\nWe recommend configuring `max_tokens` default value on a per-model basis in the DIAL Core Config, for example:\n\n```json\n{\n    \"models\": {\n        \"dial-claude-deployment-id\": {\n            \"type\": \"chat\",\n            \"description\": \"...\",\n            \"endpoint\": \"...\",\n            \"defaults\": {\n                \"max_tokens\": 2048\n            }\n        }\n    }\n}\n```\n\nIf the default is missing in the DIAL Core Config, it will be taken from the `CLAUDE_DEFAULT_MAX_TOKENS` environment variable.\nHowever, we strongly recommend not to rely on this variable and instead configure the defaults in the DIAL Core Config.\nSuch a **per-model** configuration is operationally cleaner since all the information relevant to tokens _(like pricing and token limits)_ is kept in the same place.\n\nThe default value set in the DIAL Core Config takes precedence over the one configured in the adapter.\n\nMake sure the default doesn't exceed Claude's [max output tokens](https://docs.anthropic.com/en/docs/about-claude/models/all-models#model-comparison-table), otherwise, you will receive an error like this one: `The maximum tokens you requested exceeds the model limit of 131072`.\n\n## Compatibility mode\n\nThe Adapter supports a predefined list of AWS Bedrock deployments. The [Supported models](#supported-models) section lists the models. These models could be accessed via `/openai/deployments/MODEL_ID/(chat_completions|embeddings)` endpoints. The Adapter won't recognize any other deployment name and will result in 404 error.\n\nNow, suppose AWS Bedrock released a new version of a model, e.g. `anthropic.claude-3-5-sonnet-20250210-v3:0` that is a better version of an older `anthropic.claude-3-5-sonnet-20241022-v2:0` model.\n\nImmediately after the release, the former model is **unsupported** by the Adapter, but the latter is **supported**.\nTherefore, the request to `openai/deployments/anthropic.claude-3-5-sonnet-20250210-v3:0/chat/completions` will result in 404 error.\n\nIt will take some time for the Adapter to catch up with AWS Bedrock - support the v3 model and publish the release with the fix.\n\nWhat to do in the meantime? Presumably, the v3 model is backward compatible with v2, so we may try to run v3 in **the compatibility mode** - that is to convince the Adapter to process v3 request as if it's v2 request with the only difference that the final upstream request to AWS Bedrock will be to v3 and not v2.\n\nThere are two way to enable compatibility mode in the adapter.\n\n### Compatibility configuration in DIAL Core config\n\nIt's possible to define compatible model on per-upstream basis in the DIAL Core configuration.\n\nE.g. the following configuration enables `anthropic.claude-3-5-sonnet-20250210-v3:0` model _(that isn't supported by the Adapter natively)_ via `anthropic.claude-3-5-sonnet-20241022-v2:0` model _(that is supported by the Adapter natively)_:\n\n```json\n{\n  \"models\": {\n    \"dial-deployment-id-for-claude-3-5\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/deployments/anthropic.claude-3-5-sonnet-20250210-v3:0/chat/completions\",\n      \"upstreams\": [\n        {\n          \"extraData\": {\n            \"compatible_model_id\": \"anthropic.claude-3-5-sonnet-20241022-v2:0\"\n          }\n        }\n      ]\n    }\n  }\n}\n```\n\nThe given configuration enables the adapter to handle requests to `anthropic.claude-3-5-sonnet-20250210-v3:0` deployment.\nThe requests will be processed by the same pipeline as `anthropic.claude-3-5-sonnet-20241022-v2:0`, but the call to AWS Bedrock will be done to `anthropic.claude-3-5-sonnet-20250210-v3:0` deployment name.\n\nNaturally, this will only work if the APIs of v2 and v3 deployments are compatible:\n\n1. The requests utilizing the modalities supported by both v2 and v3 will work just fine.\n2. However, the requests with modalities that are supported by v3 _(e.g. audio)_ and aren't supported by v2, won't be processed correctly. You will have to wait until the adapter supports the v3 deployment natively.\n\nWhen a version of the adapter supporting the v3 model is released, you may migrate to it and safely remove the `compatible_model_id` from the DIAL Core config.\n\nNote that setting `compatible_model_id=stability.stable-image-ultra-v1:0` will be ineffectual, since the APIs of the two model and their capabilities are drastically different.\n\n\u003e [!IMPORTANT]\n\u003e If the DIAL deployment has many upstreams, the `compatible_model_id` field should be set in all of the upstreams.\n\n### Compatibility configuration in Adapter\n\n\u003e [!IMPORTANT]\n\u003e Model compatibility configuration via `COMPATIBILITY_MAPPING` env variable has been deprecated in favour of [configuration on the DIAL Core side](#compatibility-configuration-in-dial-core-config) in DIAL Core config.\n\n`COMPATIBILITY_MAPPING` env variable enables compatibility mode on the adapter level.\nIt hold a mapping from unsupported deployment ids to supported deployment ids.\n\nE.g. the following mapping enables `anthropic.claude-3-5-sonnet-20250210-v3:0` via `anthropic.claude-3-5-sonnet-20241022-v2:0`:\n\n```ini\nCOMPATIBILITY_MAPPING={\"anthropic.claude-3-5-sonnet-20250210-v3:0\": \"anthropic.claude-3-5-sonnet-20241022-v2:0\"}\n```\n\n## Load balancing\n\nIf you use DIAL Core load balancing mechanism, you can provide `extraData` upstream setting with different AWS account credentials/regions to use different model deployments:\n\n```json\n{\n  \"upstreams\": [\n    {\n      \"extraData\": {\n        \"region\": \"eu-west-1\",\n        \"aws_access_key_id\": \"key_id_1\",\n        \"aws_secret_access_key\": \"access_key_1\"\n      }\n    },\n    {\n      \"extraData\": {\n        \"region\": \"eu-west-1\",\n        \"aws_access_key_id\": \"key_id_2\",\n        \"aws_secret_access_key\": \"access_key_2\",\n        \"aws_session_token\": \"optional session token\"\n      }\n    },\n    {\n      \"extraData\": {\n        \"region\": \"eu-west-1\",\n        \"aws_assume_role_arn\": \"arn:aws:iam::123456789012:role/BedrockAccessAdapterRoleName\"\n      }\n    },\n    {\n      \"key\": \"anthropic-api-key\"\n    }\n  ]\n}\n```\n\nThe fields in the extra data override the corresponding environment variables:\n\n|`extraData` field|Env variable|\n|---|---|\n|`region`|`AWS_DEFAULT_REGION`|\n|`aws_access_key_id`|`AWS_ACCESS_KEY_ID`|\n|`aws_secret_access_key`|`AWS_SECRET_ACCESS_KEY`|\n|`aws_session_token`|`AWS_SESSION_TOKEN`|\n|`aws_assume_role_arn`|`AWS_ASSUME_ROLE_ARN`|\n\n## Authentication\n\n### AWS Bedrock\n\nAuthentication with AWS Bedrock is configured either:\n\n1. globally via `AWS_*` environment vars, or\n2. on a [per upstream basis](#load-balancing) via `upstreams.extraData` fields in DIAL Core Config.\n\n### Anthropic API\n\nThe adapter supports authentication with Anthropic API for Claude deployments.\n\n1. Choose one of the **Claude API** model names from [the official documentation](https://docs.claude.com/en/docs/about-claude/models/overview#model-names). Let's call it `CLAUDE_API_MODEL_NAME`.\n2. Find which **AWS Bedrock** model name corresponds to the chosen **Claude API** model name on the same documentation page. Let's call it `AWS_BEDROCK_MODEL_NAME`.\n3. Add the Claude deployment to the DIAL Core configuration with API key configured on a per upstream basis:\n\n    ```json\n    {\n      \"models\": {\n        \"dial-claude-deployment-name\": {\n          \"endpoint\": \"${ADAPTER_ORIGIN}/deployments/${CLAUDE_API_MODEL_NAME}/chat/completions\",\n          \"upstreams\": [\n            {\n              \"key\": \"${ANTHROPIC_API_KEY}\",\n              \"extraData\": {\n                \"compatible_model_id\": \"${AWS_BEDROCK_MODEL_NAME}\"\n              }\n            }\n          ]\n        }\n      }\n    }\n    ```\n\n    Note that there is no need to configure the upstream endpoint, since there is only one endpoint for the model inference in the Anthropic API and it will be used by default: `https://api.anthropic.com/v1/messages`.\n\n    The same Anthropic models have [different model names](https://docs.anthropic.com/en/docs/about-claude/models/overview#model-names) in **Claude API** and **AWS Bedrock**. For example:\n\n    1. `claude-sonnet-4-5-20250929` _(`${CLAUDE_API_MODEL_NAME}`)_ in **Claude API** corresponds to\n    2. `anthropic.claude-sonnet-4-5-20250929-v1:0` _(`${AWS_BEDROCK_MODEL_NAME}`)_ in **AWS Bedrock**.\n\n    The Bedrock adapter uses model names from **AWS Bedrock**. Therefore, in order to use **Claude API** model name you need to specify the corresponding name from **AWS Bedrock** in the [compatible_model_id](#compatibility-configuration-in-dial-core-config) field. Otherwise, the adapter returns 404.\n\n---\n\n## Development\n\n### Development environment\n\nThis project uses [Python\u003e=3.11](https://www.python.org/downloads/) and [Poetry\u003e=2.1.1](https://python-poetry.org/) as a dependency manager.\n\nCheck out Poetry's [documentation on how to install it](https://python-poetry.org/docs/#installation) on your system before proceeding.\n\nTo install requirements:\n\n```sh\npoetry install\n```\n\nThis will install all requirements for running the package, linting, formatting and tests.\n\n### IDE configuration\n\nThe recommended IDE is [VS Code](https://code.visualstudio.com/).\nOpen the project in VS Code and install the recommended extensions.\nVS Code is configured to use PEP-8 compatible formatter [Black](https://black.readthedocs.io/en/stable/index.html).\n\nAlternatively you can use [PyCharm](https://www.jetbrains.com/pycharm/).\nSet up the Black in PyCharm [manually](https://black.readthedocs.io/en/stable/integrations/editors.html#pycharm-intellij-idea) or\ninstall PyCharm\u003e=2023.2 with [built-in Black support](https://blog.jetbrains.com/pycharm/2023/07/2023-2/#black).\n\n### Make on Windows\n\nAs of now, Windows distributions do not include the make tool. To run make commands, the tool can be installed using\nthe following command (since [Windows 10](https://learn.microsoft.com/en-us/windows/package-manager/winget/)):\n\n```sh\nwinget install GnuWin32.Make\n```\n\nFor convenience, the tool folder can be added to the PATH environment variable as `C:\\Program Files (x86)\\GnuWin32\\bin`.\nThe command definitions inside Makefile should be cross-platform to keep the development environment setup simple.\n\n### Run\n\nRun the development server locally:\n\n```sh\nmake serve\n```\n\nRun the server from a Docker container:\n\n```sh\nmake docker_serve\n```\n\n### Lint\n\nDon't forget to run the linting before committing:\n\n```sh\nmake lint\n```\n\nTo auto-fix formatting issues run:\n\n```sh\nmake format\n```\n\n### Test\n\nTo run the unit tests locally:\n\n```sh\nmake test\n```\n\nTo run the unit tests from the Docker container:\n\n```sh\nmake docker_test\n```\n\nTo run the integration tests locally:\n\n```sh\nmake integration_tests\n```\n\n### Clean\n\nTo remove the virtual environment and build artifacts:\n\n```sh\nmake clean\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepam%2Fai-dial-adapter-bedrock","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepam%2Fai-dial-adapter-bedrock","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepam%2Fai-dial-adapter-bedrock/lists"}