{"id":28533452,"url":"https://github.com/epam/ai-dial-adapter-openai","last_synced_at":"2026-01-22T18:12:19.099Z","repository":{"id":198007913,"uuid":"688399252","full_name":"epam/ai-dial-adapter-openai","owner":"epam","description":"The project implements AI DIAL API for language models from Azure OpenAI","archived":false,"fork":false,"pushed_at":"2025-07-02T14:27:50.000Z","size":1124,"stargazers_count":13,"open_issues_count":10,"forks_count":5,"subscribers_count":17,"default_branch":"development","last_synced_at":"2025-07-02T15:36:42.625Z","etag":null,"topics":["ai-dial","llm"],"latest_commit_sha":null,"homepage":"https://epam-rail.com","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/epam.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":"SECURITY.md","support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-09-07T09:12:32.000Z","updated_at":"2025-07-02T14:25:35.000Z","dependencies_parsed_at":"2023-12-21T19:57:08.107Z","dependency_job_id":"b3bf2061-f5b8-40e6-afed-0f84f6bfd494","html_url":"https://github.com/epam/ai-dial-adapter-openai","commit_stats":null,"previous_names":["epam/ai-dial-adapter-openai"],"tags_count":39,"template":false,"template_full_name":null,"purl":"pkg:github/epam/ai-dial-adapter-openai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-openai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-openai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-openai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-openai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/epam","download_url":"https://codeload.github.com/epam/ai-dial-adapter-openai/tar.gz/refs/heads/development","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/epam%2Fai-dial-adapter-openai/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264142578,"owners_count":23563524,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai-dial","llm"],"created_at":"2025-06-09T17:07:06.442Z","updated_at":"2026-01-22T18:12:19.085Z","avatar_url":"https://github.com/epam.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e\n  DIAL OpenAI Adapter\n\u003c/h1\u003e\n\u003cp align=\"center\"\u003e\n  \u003cp align=\"center\"\u003e\n  \u003ca href=\"https://dialx.ai/\"\u003e\n    \u003cimg src=\"https://dialx.ai/logo/dialx_logo.svg\" alt=\"About DIALX\"\u003e\n  \u003c/a\u003e\n\u003c/p\u003e\n\u003ch4 align=\"center\"\u003e\n  \u003ca href=\"https://discord.gg/ukzj9U9tEe\"\u003e\n    \u003cimg src=\"https://img.shields.io/static/v1?label=DIALX%20Community%20on\u0026message=Discord\u0026color=blue\u0026logo=Discord\u0026style=flat-square\" alt=\"Discord\"\u003e\n  \u003c/a\u003e\n\u003c/h4\u003e\n\n- [Overview](#overview)\n- [Chat completions deployments](#chat-completions-deployments)\n  - [Supported upstream chat APIs](#supported-upstream-chat-apis)\n    - [Azure OpenAI Chat Completions API (Last generation API)](#azure-openai-chat-completions-api-last-generation-api)\n    - [Azure OpenAI Chat Completions API (Next generation API)](#azure-openai-chat-completions-api-next-generation-api)\n    - [Azure OpenAI Responses API (Next generation API)](#azure-openai-responses-api-next-generation-api)\n    - [Azure AI Foundry Chat Completions API](#azure-ai-foundry-chat-completions-api)\n    - [Azure OpenAI Images API](#azure-openai-images-api)\n    - [Azure OpenAI Video API](#azure-openai-video-api)\n    - [Azure Audio API](#azure-audio-api)\n      - [Text-to-speech models (TTS)](#text-to-speech-models-tts)\n      - [Speech-to-text models (STT)](#speech-to-text-models-stt)\n    - [OpenAI Platform Chat Completions API](#openai-platform-chat-completions-api)\n    - [OpenAI Completions API](#openai-completions-api)\n    - [Mistral Chat Completion API](#mistral-chat-completion-api)\n  - [Tokenization of chat completion requests/responses](#tokenization-of-chat-completion-requestsresponses)\n    - [How to minimize adapter-side tokenization](#how-to-minimize-adapter-side-tokenization)\n    - [Tokenization algorithm](#tokenization-algorithm)\n      - [Text tokenization](#text-tokenization)\n      - [Image tokenization](#image-tokenization)\n- [Embedding deployments](#embedding-deployments)\n  - [Supported upstream embedding APIs](#supported-upstream-embedding-apis)\n    - [Azure OpenAI Embeddings API (Last generation API)](#azure-openai-embeddings-api-last-generation-api)\n    - [Azure OpenAI Embeddings API (Next generation API)](#azure-openai-embeddings-api-next-generation-api)\n    - [Azure multimodal embeddings](#azure-multimodal-embeddings)\n    - [OpenAI Platform Embeddings API](#openai-platform-embeddings-api)\n- [Environment Variables](#environment-variables)\n  - [Categories of deployments](#categories-of-deployments)\n  - [Other variables](#other-variables)\n- [Configurable models](#configurable-models)\n  - [DALL-E / GPT Image 1](#dall-e--gpt-image-1)\n    - [Forward compatibility](#forward-compatibility)\n  - [Models based on Responses API](#models-based-on-responses-api)\n    - [Reasoning configuration](#reasoning-configuration)\n- [Load balancing](#load-balancing)\n- [Prompt caching](#prompt-caching)\n- [API versioning](#api-versioning)\n- [Server performance configuration](#server-performance-configuration)\n- [Deployment](#deployment)\n  - [Private CAs and self-signed certificates](#private-cas-and-self-signed-certificates)\n    - [Docker](#docker)\n- [Development](#development)\n  - [Development environment](#development-environment)\n  - [IDE configuration](#ide-configuration)\n  - [Make on Windows](#make-on-windows)\n  - [Run](#run)\n  - [Lint](#lint)\n  - [Test](#test)\n  - [Clean](#clean)\n\n---\n\n## Overview\n\nLLM Adapters unify the APIs of respective LLMs to align with the Unified Protocol of DIAL Core. Each Adapter operates within a dedicated container. Multi-modality allows supporting non-textual communications such as image-to-text, text-to-image, file transfers and more.\n\nThe project implements [AI DIAL API](https://dialx.ai/dial_api) for language models from [Azure OpenAI](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models).\n\n![ai-dial-core](https://docs.dialx.ai/assets/images/adapters-62587fb74cfb1c4225c20c08273ec5bc.svg)\n\n---\n\n## Chat completions deployments\n\nThe adapter is able to convert certain upstream APIs to the [DIAL Chat Completions API](https://dialx.ai/dial_api#operation/sendChatCompletionRequest) *(which is an extension of Azure [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat))*.\n\nChat Completions deployments are exposed via the endpoint:\n\n```text\nPOST ${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\n```\n\n### Supported upstream chat APIs\n\n#### Azure OpenAI Chat Completions API (Last generation API)\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/deployments/${AZURE_OPENAI_DEPLOYMENT_ID}/chat/completions\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nThere are three free variables in the config related to deployment ids.\nEach of these variables corresponds to an HTTP request initiated by the DIAL client:\n\n1. `DIAL_DEPLOYMENT_ID` - it's the deployment id visible to the DIAL Client via DIAL deployment listing. The client will be using the id to call the model by sending the request `POST ${DIAL_CORE_ORIGIN}/openai/deployments/${DIAL_DEPLOYMENT_ID}/chat/completions`\n2. `ADAPTER_DEPLOYMENT_ID` - the deployment id the OpenAI adapter receives when DIAL Core calls `POST ${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions`. Use this identifier in environment variables that define [deployment categories](#categories-of-deployments).\n3. `AZURE_OPENAI_DEPLOYMENT_ID` - the Azure OpenAI deployment called by the OpenAI adapter.\n\n```mermaid\nsequenceDiagram\n    autonumber\n    actor U as DIAL Client\n    participant C as DIAL Core\n    participant A as OpenAI Adapter\n    participant AZ as Azure OpenAI\n    participant OP as OpenAI Platform\n\n    Note over U,C: DIAL_DEPLOYMENT_ID\n    U-\u003e\u003eC: POST /openai/deployments/\u003cbr\u003e${DIAL_DEPLOYMENT_ID}/chat/completions\n\n    Note over C,A: ADAPTER_DEPLOYMENT_ID\n    C-\u003e\u003eA: POST ${ADAPTER_ORIGIN}/openai/deployments/\u003cbr\u003e${ADAPTER_DEPLOYMENT_ID}/chat/completions\n\n    alt Azure OpenAI upstream\n        Note over A,AZ: AZURE_OPENAI_DEPLOYMENT_ID\n        A-\u003e\u003eAZ: POST https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/\u003cbr\u003eopenai/deployments/${AZURE_OPENAI_DEPLOYMENT_ID}/\u003cbr\u003echat/completions\n        Note right of A: Auth: api-key (if provided) or Azure AD via DefaultAzureCredential\n        AZ--\u003e\u003eA: JSON or SSE stream\n    else OpenAI Platform upstream\n        A-\u003e\u003eOP: POST https://api.openai.com/v1/chat/completions\u003cbr\u003e(with \"model\"=${OPENAI_MODEL_NAME}, api-key)\n        OP--\u003e\u003eA: JSON or SSE stream\n    end\n\n    A--\u003e\u003eC: Normalized response (headers/stream)\n    C--\u003e\u003eU: Response to client\n```\n\nTypically these three variables share the same value (the Azure OpenAI deployment name). They may differ if you expose multiple DIAL deployments that call the same Azure OpenAI endpoint but [configured](#configurable-models) differently.\n\nThe [DefaultAzureCredential](https://learn.microsoft.com/en-us/python/api/azure-identity/azure.identity.defaultazurecredential?view=azure-python) is used to authenticate requests to Azure when an API key is not provided in the upstream configuration.\n\n#### Azure OpenAI Chat Completions API (Next generation API)\n\nThe Next generation API (aka [v1 API](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/api-version-lifecycle?tabs=key#next-generation-api)) doesn't include the deployment id in the URL:\n\n- Last generation API: `POST https://SERVICE_NAME.openai.azure.com/openai/deployments/gpt-4o/chat/completions`\n- Next generation API: `POST https://SERVICE_NAME.openai.azure.com/openai/v1/chat/completions`\n\nThe DIAL configuration changes accordingly:\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${AZURE_OPENAI_DEPLOYMENT_ID}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/v1/chat/completions\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nBecause the deployment ID is not included in the upstream URL, specify it in the `overrideName` field. If this field is missing, the model name takes the value of the `model` field from the original chat completion request (if present), otherwise `${ADAPTER_DEPLOYMENT_ID}`.\n\n#### Azure OpenAI Responses API (Next generation API)\n\nCertain advanced features of OpenAI models, such as [reasoning summary](https://platform.openai.com/docs/guides/reasoning#reasoning-summaries), are only accessible via Responses API and not accessible via Chat Completions API.\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${AZURE_OPENAI_DEPLOYMENT_ID}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/v1/responses\",\n          \"key\": \"${API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nAs in other cases where the upstream URL omits a deployment id, specify it in the `overrideName` field.\n\nThe last generation API is also supported via an URLs in the following format:\n\n```text\n\"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/responses\"\n```\n\n#### Azure AI Foundry Chat Completions API\n\nCertain LLM models like `gpt-oss-120b` or `Mistral-Large-2411` can only be deployed to an Azure AI Foundry service. They are accessible via:\n\n- Azure AI model inference endpoint or\n- Azure OpenAI endpoint\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config (Azure AI model inference endpoint)\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${AZURE_AI_FOUNDRY_DEPLOYMENT_ID}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_AI_FOUNDRY_SERVICE_NAME}.services.ai.azure.com/models/chat/completions\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config (Azure OpenAI endpoint)\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${AZURE_AI_FOUNDRY_DEPLOYMENT_ID}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_AI_FOUNDRY_SERVICE_NAME}.openai.azure.com/openai/deployments/gpt-oss-120b/chat/completions\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n#### Azure [OpenAI Images API](https://platform.openai.com/docs/api-reference/images/create)\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/deployments/${AZURE_OPENAI_DEPLOYMENT_ID}/images/generations\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nThe supported upstream models are `dall-e-3` and `gpt-image-1`. These are the values that `AZURE_OPENAI_DEPLOYMENT_ID` variable can take.\n\n\u003e [!IMPORTANT]\n\u003e The DALL·E 3 adapter deployment must be declared in `DALLE3_DEPLOYMENTS` env variable, and GPT-Image 1 deployment - in `GPT_IMAGE_1_DEPLOYMENTS`.\n\n#### [Azure OpenAI Video API](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/video-generation-quickstart)\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${AZURE_OPENAI_DEPLOYMENT_ID}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/v1/video/generations\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nThe supported upstream models are `sora`. This is the value that `AZURE_OPENAI_DEPLOYMENT_ID` variable can take.\n\nThe video generation models support configuration via the `custom_fields.configuration` field in the chat completion request:\n\n```json\n{\n  \"model\": \"sora\",\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"A cat playing with a ball of yarn\"\n    }\n  ],\n  \"custom_fields\": {\n    \"configuration\": {\n      \"width\": 480,\n      \"height\": 480,\n      \"n_seconds\": 5,\n      \"n_variants\": 1\n    }\n  }\n}\n```\n\nWidth and height are defaulted to 480x480 if not specified.\n\nFind the details in the [Azure API specification](https://github.com/Azure/azure-rest-api-specs/blob/691227eb0ccbaf57f8fda11e52692b4dd8600ccb/specification/ai/data-plane/OpenAI.v1/azure-v1-preview-generated.yaml#L6730-L6763).\n\n\u003e [!NOTE]\n\u003e `n_variants\u003e1` results in multiple video attachments to a **single chat completion choice**.\n\n\u003e [!IMPORTANT]\n\u003e Prompt tokens in the usage are set to zero.\n\u003e Completion tokens are set to the overall number of seconds in the generated video(s).\n\n#### Azure Audio API\n\nThe adapter supports models connected via [Azure Audio API](https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure#audio-api).\n\n##### Text-to-speech models (TTS)\n\nSet `AZURE_DEPLOYMENT_ID` variable to one of the [text-to-speech models](https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure#text-to-speech-models-preview) supported by Azure Audio API:\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${AZURE_AUDIO_API_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_SERVICE_NAME}.(openai|cognitiveservices).azure.com/openai/deployments/${AZURE_DEPLOYMENT_ID/audio/speech\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nAt the moment of writing, these are: `tts`, `tts-hd`, and `gpt-4o-mini-tts`.\n\nThe adapter takes the last user message as a text prompt and sends it to the upstream as `input` parameter. The input text is limited to 4096 characters. The text is being translated into speech audio by the upstream model. The audio file is returned as an attachment in the chat completion response.\n\nSystem instructions are used to set the tone of the synthesized speech.\n\nThe adapter supports the following configuration for the TTS models:\n\n```text\n{\n  \"instruction\": \"Speak in a cheerful tone.\", # optional, sets the tone; appended the system message from the chat completion request\n  \"voice\": \"allow\", # one of the preset voices\n  \"speed\": 1.0, # speech speed multiplier\n  \"response_format\": \"mp3\" # one of the supported audio formats\n}\n```\n\nFind the configuration details in the [Azure specification](https://github.com/Azure/azure-rest-api-specs/blob/4c5ec9b4e0b961799cc11f6051f240d18f093c38/specification/cognitiveservices/data-plane/AzureOpenAI/inference/preview/2025-04-01-preview/inference.yaml#L5287-L5323) or in the [OpenAI Platform specification](https://platform.openai.com/docs/api-reference/audio/createSpeech?api-mode=chat).\n\nThe usage is computed in the following way:\n\n1. `gpt-4o-mini-tts` - prompt tokens are computed using `gpt-4o` tiktoken algorithm. Completion tokens are set to zero.\n2. `tts` and `tts-hd` - there is no official documentation on the pricing for these models. Tokenizer for `gpt-4o` model will be used as a default for prompt tokens calculation. Completion tokens are set to zero.\n\n##### Speech-to-text models (STT)\n\nSet `AZURE_DEPLOYMENT_ID` variable to one of the [speech-to-text models](https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure#speech-to-text-models) supported by Azure Audio API:\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${AZURE_AUDIO_API_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_SERVICE_NAME}.(openai|cognitiveservices).azure.com/openai/deployments/${AZURE_DEPLOYMENT_ID/audio/transcriptions\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nAt the moment of writing, these are: `whisper`, `gpt-4o-transcribe`, `gpt-4o-mini-transcribe`, and `gpt-4o-transcribe-diarize`.\n\nThe adapter takes an audio attachment from the last user message and pass it to the transcription model. The transcription is return as a text in the chat completion response.\n\nSystem instructions are used to set the [prompt](https://platform.openai.com/docs/api-reference/audio/createTranscription#audio-createtranscription-prompt) parameter in the Transcription API request.\n\nThe usage is computed in the following way:\n\n1. `gpt-4o-*` models return audio tokens in the `usage.prompt_tokens` field and text tokens - in `usage.completion_tokens`.\n2. `whisper` models return duration of the given audio file in seconds in `usage.prompt_tokens` and zero in `usage.completion_tokens`.\n\n#### OpenAI Platform [Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create)\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${OPENAI_MODEL_NAME}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://api.openai.com/v1/chat/completions\",\n          \"key\": \"${API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nNote the difference from the Azure OpenAI configuration:\n\n- The API key is required.\n- Added `overrideName` to specify the upstream OpenAI model name. The upstream URL does not include the model name *(unlike Azure)*, so we pass it via `overrideName`. If this field is missing, the model name takes the value of the `model` field from the original chat completion request *(if present)*, otherwise `${ADAPTER_DEPLOYMENT_ID}`.\n\n#### OpenAI Completions API\n\nThe adapter also supports **legacy** [Completions API](https://platform.openai.com/docs/api-reference/completions/create) both for Azure-style upstream endpoints and OpenAI Platform-style endpoints:\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${OPENAI_MODEL_NAME}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://api.openai.com/v1/completions\",\n          \"key\": \"${API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n#### Mistral Chat Completion API\n\nThe Mistral Platform provides [Chat Completions API](https://docs.mistral.ai/api/#tag/chat), therefore, it could be connected to via the adapter:\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"overrideName\": \"${MISTRAL_MODEL_NAME}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${MISTRAL_MODEL_NAME}/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://api.mistral.ai/v1/chat/completions\",\n          \"key\": \"${MISTRAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nWhere `MISTRAL_MODEL_NAME` is one of the available [models](https://docs.mistral.ai/getting-started/models/models_overview/) on the Platform.\n\n### Tokenization of chat completion requests/responses\n\nThe adapter guarantees that all chat completion responses include token-usage information *(the number of prompt and completion tokens consumed)*.\n\nHowever, by default neither Azure OpenAI nor OpenAI Platform returns token usage for streaming requests *(those with `stream=true`)*.\n\nTherefore, the adapter tokenizes both the request and the response when the upstream doesn’t provide usage. Adapter-side tokenization is also required when the request includes `max_prompt_tokens` - the maximum number of tokens to which the incoming request is truncated before being sent upstream.\n\n#### How to minimize adapter-side tokenization\n\nThe tokenization algorithm is CPU-heavy and may throttle requests under high load. Therefore, it’s important to minimize cases where tokenization is required.\n\nAzure OpenAI and OpenAI Platform return token usage for streaming requests when the [`include_usage`](https://platform.openai.com/docs/api-reference/chat/create#chat-create-stream_options) option is enabled in the chat completion request. We recommend setting this option in the DIAL Core configuration via the `defaults` field to reduce the adapter’s CPU usage:\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"...\",\n      \"upstreams\": [\"...\"],\n      \"defaults\": {\n        \"stream_options\": {\n          \"include_usage\": true\n        }\n      }\n    }\n  }\n}\n```\n\n#### Tokenization algorithm\n\nHow does the adapter know which deployment requires which tokenization algorithm?\n\nThe adapter does not perform tokenization for:\n\n1. deployments registered in `DATABRICKS_DEPLOYMENTS` and `MISTRAL_DEPLOYMENTS` env vars. It's expected upstream for these deployments are going to return the token usage.\n2. deployments supported by the following APIs:\n   1. legacy Completions API\n   2. Images API\n   3. Responses API\n\nFor other deployments, tokenization is determined as follows.\n\n\u003e [!IMPORTANT]\n\u003e Adapter-side tokenization of documents, audio, and video files isn’t currently supported. Such multimodal content is counted as zero tokens.\n\n##### Text tokenization\n\nThe adapter is using the [tiktoken](https://github.com/openai/tiktoken) library as a tokenizer for OpenAI models.\n\n`TIKTOKEN_MODEL_MAPPING` env variable defines a mapping from adapter deployment ids to the model identifies which are know to [tiktoken](https://github.com/openai/tiktoken/blob/main/tiktoken/model.py).\n\nIf deployment id is missing from `TIKTOKEN_MODEL_MAPPING`, then the deployment id itself will be used to find a tokenizer in `tiktoken`. You can check if the deployment id is compatible with `tiktoken` by running the command `python -c \"from tiktoken.model import encoding_name_for_model as e; print(e('my-deployment-name'))\"`.\n\nFinally, if the deployment id is neither declared in `TIKTOKEN_MODEL_MAPPING`, nor is it compatible with `tiktoken`, then the tokenizer for `gpt-4o` model will be used as a default. It's a reasonable default since the corresponding `o200k_base` tokenizer is used for the [majority](https://github.com/openai/tiktoken/blob/0.12.0/tiktoken/model.py#L8-L16) of the latest OpenAI models.\n\n##### Image tokenization\n\nIf a deployment is registered in `GPT4O_DEPLOYMENTS` or `GPT4O_MINI_DEPLOYMENTS`, the corresponding image-tokenization algorithm described in [the Azure documentation](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/overview#image-input-tokens) is used.\n\nOtherwise, images aren’t tokenized — the image tokens are assumed to be 0.\n\n---\n\n## Embedding deployments\n\nThe adapter is able to convert certain upstream APIs to the [DIAL Embeddings API](https://dialx.ai/dial_api#operation/sendEmbeddingsRequest) *(which is an extension of Azure [OpenAI Embeddings API](https://platform.openai.com/docs/api-reference/embeddings/create))*.\n\nEmbeddings deployments are exposed via the endpoint:\n\n```text\nPOST ${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/embeddings\n```\n\n### Supported upstream embedding APIs\n\n#### Azure OpenAI Embeddings API (Last generation API)\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"embedding\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/embeddings\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/deployments/${AZURE_OPENAI_DEPLOYMENT_ID}/embeddings\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n#### Azure OpenAI Embeddings API (Next generation API)\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"embedding\",\n      \"overrideName\": \"${AZURE_OPENAI_DEPLOYMENT_ID}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/embeddings\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${AZURE_OPENAI_SERVICE_NAME}.openai.azure.com/openai/v1/embeddings\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n#### Azure multimodal embeddings\n\nThe adapter supports [Azure Multimodal embeddings](https://learn.microsoft.com/en-us/azure/ai-services/computer-vision/concept-image-retrieval).\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"embedding\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/embeddings\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://${COMPUTER_VISION_SERVICE_NAME}.cognitiveservices.azure.com\",\n          \"key\": \"${OPTIONAL_API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n\u003e [!IMPORTANT]\n\u003e `${ADAPTER_DEPLOYMENT_ID}` must be added to the env variable `AZURE_AI_VISION_DEPLOYMENTS` to enable the embeddings deployment.\n\nThe multimodal embeddings model supports text and images as inputs.\n\nSince the original OpenAI embeddings API only support text inputs, the image inputs should be passed in the `custom_input` request field as URL or in base64-encoded format:\n\n```sh\ncurl -X POST \"${DIAL_CORE_ORIGIN}/deployments/${DIAL_DEPLOYMENT_ID}/embeddings\" -v \\\n  -H \"api-key:${DIAL_API_KEY}\" \\\n  -H \"content-type:application/json\" \\\n  -d '{\"input\": [\"cat\", \"fish\"], \"custom_input\": [{\"type\": \"image/png\", \"url\": \"https://learn.microsoft.com/azure/ai-services/computer-vision/media/quickstarts/presentation.png\"}]}'\n```\n\nThe response will contain three embedding vectors, each corresponding to one of the inputs in the original request.\n\n#### OpenAI Platform [Embeddings API](https://platform.openai.com/docs/api-reference/embeddings/create)\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"${DIAL_DEPLOYMENT_ID}\": {\n      \"type\": \"embedding\",\n      \"overrideName\": \"${OPENAI_MODEL_NAME}\",\n      \"endpoint\": \"${ADAPTER_ORIGIN}/openai/deployments/${ADAPTER_DEPLOYMENT_ID}/embeddings\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://api.openai.com/v1/embeddings\",\n          \"key\": \"${API_KEY}\"\n        }\n      ]\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n---\n\n## Environment Variables\n\nCopy `.env.example` to `.env` and customize it for your environment.\n\n### Categories of deployments\n\nThe following variables cluster all deployments into the groups of deployments which share the same API and the same tokenization algorithm.\n\n|Variable|Default|Description|\n|---|---|---|\n|DALLE3_DEPLOYMENTS|``|Comma-separated list of deployments that support DALL-E 3 API. Example: `dall-e-3,dalle3,dall-e`|\n|DALLE3_AZURE_API_VERSION|2024-02-01|The API version for requests to the Azure DALL·E 3 API|\n|GPT_IMAGE_1_DEPLOYMENTS|``|Comma-separated list of deployments that support GPT-Image 1 API. Example: `gpt-image-1`|\n|GPT_IMAGE_1_AZURE_API_VERSION|2024-02-01|The API version for requests to the Azure GPT-Image 1 API|\n|MISTRAL_DEPLOYMENTS|``|Comma-separated list of deployments that support Mistral Large Azure API. Example: `mistral-large-azure,mistral-large`|\n|DATABRICKS_DEPLOYMENTS|``|Comma-separated list of Databricks chat completion deployments. Example: `databricks-dbrx-instruct,databricks-mixtral-8x7b-instruct,databricks-llama-2-70b-chat`|\n|GPT4O_DEPLOYMENTS|``|Comma-separated list of GPT-4o chat completion deployments. Example: `gpt-4o-2024-05-13`|\n|GPT4O_MINI_DEPLOYMENTS|``|Comma-separated list of GPT-4o mini chat completion deployments. Example: `gpt-4o-mini-2024-07-18`|\n|AZURE_AI_VISION_DEPLOYMENTS|``|Comma-separated list of Azure AI Vision embedding deployments. The endpoint of the deployment is expected to point to the Azure service: `https://\u003cservice-name\u003e.cognitiveservices.azure.com/`|\n|AUDIO_AZURE_API_VERSION|2025-03-01-preview|The API version for requests to the [Azure Audio API](#azure-audio-api) endpoints.|\n\nDeployments that do not fall into any of the categories are considered to support text-to-text chat completion OpenAI API or text embeddings OpenAI API.\n\n### Other variables\n\n|Variable|Default|Description|\n|---|---|---|\n|LOG_LEVEL|INFO|Log level. Use DEBUG for dev purposes and INFO in prod|\n|TIKTOKEN_MODEL_MAPPING|`{}`|A JSON dictionary from the request deployment id to a [tiktoken model name](https://github.com/openai/tiktoken/blob/main/tiktoken/model.py). It's used for [tokenization](#tokenization-of-chat-completion-requestsresponses) of chat completion requests on the adapter side. Example: `{\"my-gpt-deployment\":\"gpt-3.5-turbo\",\"my-gpt-o3-deployment\":\"o3\"}`. The tokenizer for `gpt-4o` is used as a default.|\n|DIAL_USE_FILE_STORAGE|False|Save image model artifacts to DIAL File storage (DALL-E images are uploaded to the DIAL file storage and its base64 encodings are replaced with links to the storage)|\n|DIAL_URL||URL of the core DIAL server (required when `DIAL_USE_FILE_STORAGE=True`)|\n|NON_STREAMING_DEPLOYMENTS|``|Comma-separated list of deployments that do not support streaming. The adapter will emulate streaming by calling the model and converting its response into a single-chunk stream. Example: `\"o1-mini,o1-preview\"`|\n|ACCESS_TOKEN_EXPIRATION_WINDOW|10|The Azure access token is renewed this many seconds before its actual expiration time. The buffer ensures that the token does not expire in the middle of an operation due to processing time and potential network delays.|\n|AZURE_OPEN_AI_SCOPE||Provided scope of access token to Azure OpenAI services. Default: `https://cognitiveservices.azure.com/.default`|\n|API_VERSIONS_MAPPING|`{}`|Mapping of API versions for requests to the Azure OpenAI Chat Completions API. Example: `{\"2023-03-15-preview\": \"2023-05-15\", \"\": \"2024-02-15-preview\"}`. An empty key sets the default API version when the user does not pass one in the request. Find the details in the section about [API versioning](#api-versioning).|\n|ELIMINATE_EMPTY_CHOICES|False|When enabled, the response stream is guaranteed to exclude chunks with an empty list of choices. This is useful when a DIAL client doesn't support such chunks. An empty list of choices can be generated by Azure OpenAI in at least two cases: (1) when the **Content filter** is not disabled, Azure includes [prompt filter results](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/content-filter?tabs=warning%2Cuser-prompt%2Cpython-new#prompt-annotation-message) in the first chunk with an empty list of choices; (2) when `stream_options.include_usage` is enabled, the last chunk contains usage data and an empty list of choices.|\n|WEB_CONCURRENCY|1|Number of [worker](https://www.uvicorn.org/deployment/#built-in) processes to spawn in the Uvicorn server. Find the details in the section about [performance](#server-performance-configuration).|\n|THREAD_POOL_SIZE||The size of a thread pool for CPU-heavy tasks such as tokenization and image analysis. The [default](https://github.com/python/cpython/blob/3.11/Lib/concurrent/futures/thread.py#L142) is `min(32, #logicalCPUs + 4)`. Find the details in the section about [performance](#server-performance-configuration).|\n\n---\n\n## Configurable models\n\nCertain models support configuration via the `$ADAPTER_ORIGIN/openai/deployments/$DEPLOYMENT_NAME/configuration` endpoint.\n\nGET request to this endpoint returns the schema of the model configuration in [JSON Schema](https://json-schema.org/) format.\n\nSuch models expect the `custom_fields.configuration` field of the `chat/completions` request to contain a JSON value conforming to that schema.\nThe `custom_fields.configuration` field is optional **if and only if** every field in the schema is also optional.\n\nThe configuration can be preset in the DIAL Core config via the `defaults` parameter:\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"my-deployment-id\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"$ADAPTER_ORIGIN/openai/deployments/my-deployment-id/chat/completions\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"$AZURE_OPENAI_SERVICE_ORIGIN/openai/deployments/openai-deployment-id/chat/completions\"\n        }\n      ],\n      \"defaults\": {\n        \"custom_fields\": {\n            \"configuration\": $MODEL_CONFIGURATION_OBJECT\n        }\n      }\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nThis is convenient when major model features can be enabled via configuration *(e.g., web search or reasoning)* and you want a deployment where these features are permanently enabled.\n\nDIAL Core will enrich requests with the configuration specified in `defaults`, so the client doesn’t need to provide it with each chat completion request.\n\n### DALL-E / GPT Image 1\n\nOpenAI image generation models accept configurations with parameters specific for image generation such as image size, style, and quality.\n\nThe latest supported parameters can be found in the official OpenAI documentation for models capable of [image generation](https://platform.openai.com/docs/api-reference/images/create) or in the Azure OpenAI [API documentation](https://learn.microsoft.com/en-us/azure/ai-services/openai/reference#image-generation).\n\nAlternatively, the configuration schema can be retrieved programmatically from the `/configuration` endpoint. However, this schema may lag behind the official one (see [Forward compatibility](#forward-compatibility)).\n\nAn example of DALL-E 3 request with configured style and image size:\n\n\u003cdetails\u003e\u003csummary\u003eRequest\u003c/summary\u003e\n\n```json\n{\n  \"model\": \"dall-e-3\",\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"forest meadow\"\n    }\n  ],\n  \"custom_fields\": {\n    \"configuration\": {\n      \"size\": \"1024x1024\",\n      \"style\": \"vivid\"\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nSimilarly, the configuration could be preset on the per-deployment basis in the DIAL Core config:\n\n\u003cdetails\u003e\u003csummary\u003eDIAL Core Config\u003c/summary\u003e\n\n```json\n{\n  \"models\": {\n    \"dial-dall-e-3\": {\n      \"type\": \"chat\",\n      \"description\": \"...\",\n      \"endpoint\": \"...\",\n      \"defaults\": {\n        \"custom_fields\": {\n          \"configuration\": {\n            \"size\": \"1024x1024\",\n            \"style\": \"vivid\"\n          }\n        }\n      }\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nSo that the end user doesn't have to attach configuration to each chat completion request. It will be applied automatically *(if missing)* by the DIAL Core for all incoming requests to this deployment.\n\n#### Forward compatibility\n\nThe configuration schema in the adapter isn't fixed and allows for extra fields and arbitrary parameter values. This enables forward compatibility with the future versions of the image generation API.\n\nLet's say the next version of GPT Image model introduces support of a negative prompt *(which isn't currently supported)*. It still will be possible to use a version of OpenAI adapter that is ignorant of the latest developments in the GPT Image API thanks to the permissive configuration schema.\n\n\u003cdetails\u003e\u003csummary\u003eRequest\u003c/summary\u003e\n\n```json\n{\n  \"model\": \"gpt-image-1\",\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"forest meadow\"\n    }\n  ],\n  \"custom_fields\": {\n    \"configuration\": {\n      \"negative_prompt\": \"trees\"\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\n### Models based on Responses API\n\nThe [Responses API](https://platform.openai.com/docs/api-reference/responses) provides more features than [Chat Completions API](https://platform.openai.com/docs/api-reference/chat/create). Some of these features could be enabled via a configuration fields in the chat completions request.\n\nThe JSON schema of the configuration is open which enables forward compatibility with the future developments in the Responses API.\n\n\u003e [!NOTE]\n\u003e Such a configuration is only possible for the models that are configured in the DIAL Core config to use Responses API upstream endpoints.\n\n#### Reasoning configuration\n\nThe [reasoning](https://platform.openai.com/docs/guides/reasoning) and the [reasoning summary](https://platform.openai.com/docs/guides/reasoning#reasoning-summaries) could be enabled via the configuration like this one:\n\n\u003cdetails\u003e\u003csummary\u003eRequest\u003c/summary\u003e\n\n```json\n{\n  \"model\": \"gpt-5-2025-08-07\",\n  \"messages\": [\n    {\n      \"role\": \"user\",\n      \"content\": \"Write a bash script that takes a matrix represented as a string with format \\\"[1,2],[3,4],[5,6]\\\" and prints the transpose in the same format.\"\n    }\n  ],\n  \"custom_fields\": {\n    \"configuration\": {\n      \"reasoning\": {\n        \"effort\": \"medium\",\n        \"summary\": \"auto\"\n      }\n    }\n  }\n}\n```\n\n\u003c/details\u003e\n\nHere `custom_fields.configuration.reasoning` is an object which is being passed to the Response API as the [reasoning](https://platform.openai.com/docs/api-reference/responses/create#responses_create-reasoning) parameter.\n\n\u003e [!IMPORTANT]\n\u003e Not all models support reasoning. Consult with the [documentation](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/how-to/reasoning?tabs=gpt-5%2Cpython-secure%2Cpy) before enabling reasoning.\n\n---\n\n## Load balancing\n\nThe adapter supports multiple upstream definitions in the DIAL Core config:\n\n```json\n{\n  \"models\": {\n    \"gpt-4o-2024-11-20\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"http://$OPENAI_ADAPTER_ORIGIN/openai/deployments/gpt-4o-2024-11-20/chat/completions\",\n      \"displayName\": \"GPT-4o\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://$AZURE_OPENAI_SERVICE_NAME1.openai.azure.com/openai/deployments/gpt-4o-2024-11-20/chat/completions\"\n        },\n        {\n          \"endpoint\": \"https://$AZURE_OPENAI_SERVICE_NAME2.openai.azure.com/openai/deployments/gpt-4o-2024-11-20/chat/completions\"\n        },\n        {\n          \"endpoint\": \"https://$AZURE_OPENAI_SERVICE_NAME3.openai.azure.com/openai/deployments/gpt-4o-2024-11-20/chat/completions\"\n        }\n      ]\n    }\n  }\n}\n```\n\n---\n\n## Prompt caching\n\n[Prompt caching](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/prompt-caching) can be enabled via the `autoCachingSupported` flag in the DIAL Core config.\n\n```json\n{\n  \"models\": {\n    \"gpt-4o-2024-11-20\": {\n      \"type\": \"chat\",\n      \"endpoint\": \"http://$OPENAI_ADAPTER_ORIGIN/openai/deployments/gpt-4o-2024-11-20/chat/completions\",\n      \"displayName\": \"GPT-4o\",\n      \"upstreams\": [\n        {\n          \"endpoint\": \"https://$AZURE_OPENAI_SERVICE_NAME1.openai.azure.com/openai/deployments/gpt-4o-2024-11-20/chat/completions\"\n        },\n        {\n          \"endpoint\": \"https://$AZURE_OPENAI_SERVICE_NAME2.openai.azure.com/openai/deployments/gpt-4o-2024-11-20/chat/completions\"\n        },\n        {\n          \"endpoint\": \"https://$AZURE_OPENAI_SERVICE_NAME3.openai.azure.com/openai/deployments/gpt-4o-2024-11-20/chat/completions\"\n        }\n      ],\n      \"features\": {\n        \"autoCachingSupported\": true\n      }\n    }\n  }\n}\n```\n\n\u003e [!IMPORTANT]\n\u003e Verify that the deployment actually supports [prompt caching](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/prompt-caching#supported-models) before enabling it.\n\n---\n\n## API versioning\n\nThe adapter provides an Azure-flavour of the OpenAI Chat Completions API.\n\nAzure’s API is a variant of the OpenAI Platform API. The key differences are the deployment ID in the path and the required [`api-version`](https://learn.microsoft.com/en-us/azure/ai-foundry/openai/reference#rest-api-versioning) query parameter:\n\n```txt\nOpenAI Platform: POST https://api.openai.com/v1/chat/completions\nAzure OpenAI:    POST https://YOUR_RESOURCE_NAME.openai.azure.com/openai/deployments/YOUR_DEPLOYMENT_NAME/chat/completions?api-version=2024-06-01\n```\n\nThe `api-version` parameter tracks API changes, and the OpenAI SDK requires it.\n\nConsider an application calling Azure OpenAI via DIAL. You typically pin an Azure OpenAI API version *(usually the latest)*. Over time, new API versions ship with new features, and SDKs add support for them. This means the application developer must bump both the SDK version and the Azure OpenAI API version - adding maintenance overhead.\n\nMoreover, some Azure OpenAI API versions are retired, breaking applications that still depend on them.\n\nIn practice, most changes between API versions have been backward-compatible, so clients generally want to use the latest version.\n\nGiven that the API largely evolves in a backward-compatible way, we introduced `API_VERSIONS_MAPPING` to reduce version-management burden:\n\n1. **Map deprecated to current versions** so DIAL apps don’t break:\n\n   ```txt\n   DIAL Client:\n     client = AsyncAzureOpenAI(api_version=\"2023-01-01-preview\", ...)\n     response = await client.chat.completions.create(...)\n\n   OpenAI Adapter:\n     API_VERSIONS_MAPPING={\"2023-01-01-preview\":\"2025-06-01\"}\n   ```\n\n2. **Define a default version** by mapping the empty string to the latest version. This delegates tracking of the latest API version to DIAL:\n\n   ```txt\n   DIAL Client:\n     client = AsyncAzureOpenAI(api_version=\"\", ...)\n     response = await client.chat.completions.create(...)\n\n   OpenAI Adapter:\n     API_VERSIONS_MAPPING={\"\":\"2025-06-01\"}\n   ```\n\nKeeping the mapping current is the DIAL operations team’s responsibility, not the application developer’s.\n\n\u003e [!NOTE]\n\u003e API version is irrelevant for the upstreams that use Response API or v1 Chat Completions API, since these APIs aren't versioned.\n\n---\n\n## Server performance configuration\n\nThere are two environment variables that control server performance:\n\n1. `WEB_CONCURRENCY` *(default = 1)* — the number of worker processes spawned by [uvicorn](https://www.uvicorn.org/deployment/#running-from-the-command-line). Workers run independently; the parent uvicorn process handles load balancing across them. The OS schedules workers on different CPU cores, enabling true parallelism. This matters when the server performs CPU-intensive work, primarily request/response [tokenization](#tokenization-of-chat-completion-requestsresponses). For full CPU utilization, set this to the number of **logical CPUs**. However, the default of 1 is fine if you don’t expect much CPU load (see [minimizing tokenization](#how-to-minimize-adapter-side-tokenization)).\n\n2. `THREAD_POOL_SIZE` *(default = logical CPUs + 4)* — the size of the thread pool used for CPU-heavy tasks (currently, only request/response [tokenization](#tokenization-of-chat-completion-requestsresponses)). This effectively caps how many CPU-bound tasks can run simultaneously: no more than `THREAD_POOL_SIZE` at a time. Note that this does not block requests without CPU-heavy work (e.g., health checks or embeddings requests).\n\n---\n\n## Deployment\n\n### Private CAs and self-signed certificates\n\nThe Docker container supports trusting private Certificate Authorities (CAs) and self-signed certificates for all outbound TLS connections.\n\nTo enable this, provide your CA certificates at runtime and opt in via `USE_SYSTEM_CA_CERTS` environment variable.\n\n#### Docker\n\nRun the container with:\n\n1. `USE_SYSTEM_CA_CERTS` set to any non-empty value,\n2. a directory containing one or more `*.crt` files (PEM format) mounted at `/certificates` (read-only is fine)\n\n```sh\ndocker run --rm \\\n  -e USE_SYSTEM_CA_CERTS=1 \\\n  -v \"$PWD/certs:/certificates:ro\" \\\n  epam/ai-dial-adapter-openai:development\n```\n\nWhen enabled, the container builds a temporary trust store on startup that combines the system CA bundle with all certificates found in `/certificates/*.crt`.\n\n---\n\n## Development\n\n### Development environment\n\nThis project uses [Python\u003e=3.11](https://www.python.org/downloads/) and [Poetry\u003e=2.1.1](https://python-poetry.org/) as a dependency manager.\n\nCheck out Poetry's [documentation on how to install it](https://python-poetry.org/docs/#installation) on your system before proceeding.\n\nTo install requirements:\n\n```sh\npoetry install\n```\n\nThis will install all requirements for running the package, linting, formatting and tests.\n\n### IDE configuration\n\nThe recommended IDE is [VS Code](https://code.visualstudio.com/).\nOpen the project in VS Code and install the recommended extensions.\nVS Code is configured to use PEP-8 compatible formatter [Black](https://black.readthedocs.io/en/stable/index.html).\n\nAlternatively you can use [PyCharm](https://www.jetbrains.com/pycharm/).\nSet up the Black in PyCharm [manually](https://black.readthedocs.io/en/stable/integrations/editors.html#pycharm-intellij-idea) or\ninstall PyCharm\u003e=2023.2 with [built-in Black support](https://blog.jetbrains.com/pycharm/2023/07/2023-2/#black).\n\n### Make on Windows\n\nAs of now, Windows distributions do not include the make tool. To run make commands, the tool can be installed using\nthe following command (since [Windows 10](https://learn.microsoft.com/en-us/windows/package-manager/winget/)):\n\n```sh\nwinget install GnuWin32.Make\n```\n\nFor convenience, the tool folder can be added to the PATH environment variable as `C:\\Program Files (x86)\\GnuWin32\\bin`.\nThe command definitions inside Makefile should be cross-platform to keep the development environment setup simple.\n\n### Run\n\nRun the development server locally:\n\n```sh\nmake serve\n```\n\nRun the server from a Docker container:\n\n```sh\nmake docker_serve\n```\n\n### Lint\n\nRun the linting before committing:\n\n```sh\nmake lint\n```\n\nTo auto-fix formatting issues run:\n\n```sh\nmake format\n```\n\n### Test\n\nRun unit tests locally:\n\n```sh\nmake test\n```\n\n### Clean\n\nTo remove the virtual environment and build artifacts:\n\n```sh\nmake clean\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepam%2Fai-dial-adapter-openai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fepam%2Fai-dial-adapter-openai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fepam%2Fai-dial-adapter-openai/lists"}