{"id":48362866,"url":"https://github.com/deepset-ai/fastapi-openai-compat","last_synced_at":"2026-04-05T13:04:30.461Z","repository":{"id":337390257,"uuid":"1153393461","full_name":"deepset-ai/fastapi-openai-compat","owner":"deepset-ai","description":"FastAPI router factory for OpenAI-compatible Chat Completion endpoints","archived":false,"fork":false,"pushed_at":"2026-04-01T10:19:53.000Z","size":197,"stargazers_count":7,"open_issues_count":0,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-04-01T12:26:02.697Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/deepset-ai.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2026-02-09T08:49:00.000Z","updated_at":"2026-04-01T10:19:58.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/deepset-ai/fastapi-openai-compat","commit_stats":null,"previous_names":["deepset-ai/fastapi-openai-compat"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/deepset-ai/fastapi-openai-compat","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Ffastapi-openai-compat","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Ffastapi-openai-compat/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Ffastapi-openai-compat/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Ffastapi-openai-compat/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/deepset-ai","download_url":"https://codeload.github.com/deepset-ai/fastapi-openai-compat/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/deepset-ai%2Ffastapi-openai-compat/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31436307,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-05T08:13:15.228Z","status":"ssl_error","status_checked_at":"2026-04-05T08:13:11.839Z","response_time":75,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-04-05T13:04:27.381Z","updated_at":"2026-04-05T13:04:30.451Z","avatar_url":"https://github.com/deepset-ai.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# fastapi-openai-compat\n\n[![PyPI - Version](https://img.shields.io/pypi/v/fastapi-openai-compat.svg)](https://pypi.org/project/fastapi-openai-compat)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/fastapi-openai-compat.svg)](https://pypi.org/project/fastapi-openai-compat)\n[![Tests](https://github.com/deepset-ai/fastapi-openai-compat/actions/workflows/tests.yml/badge.svg)](https://github.com/deepset-ai/fastapi-openai-compat/actions/workflows/tests.yml)\n\nFastAPI router factory for OpenAI-compatible Chat Completions, Responses, and Files upload endpoints.\n\nProvides configurable router factories for OpenAI-style APIs, with support for\nstreaming (SSE), non-streaming responses, tool calling, reasoning content,\nconfigurable hooks, custom chunk mapping, and callback-driven file upload handling.\n\n## Table of contents\n\n- [Installation](#installation)\n- [Quick start](#quick-start)\n- [A note on dict-based types](#a-note-on-dict-based-types)\n- **Chat Completions API**\n  - [The `run_completion` callable](#the-run_completion-callable)\n  - [Response types](#response-types) -- [string](#returning-a-string) \u0026#183; [generator](#streaming-with-a-generator) \u0026#183; [ChatCompletion](#returning-a-chatcompletion)\n  - [Tool calling](#tool-calling) -- [ChatCompletion](#returning-chatcompletion-directly) \u0026#183; [StreamingChunk](#automatic-streamingchunk-support)\n  - [Reasoning content](#reasoning-content)\n  - [Custom SSE events](#custom-sse-events)\n  - [Hooks](#hooks) -- [transformer](#transformer-hooks) \u0026#183; [observer](#observer-hooks)\n  - [Custom chunk mapping](#custom-chunk-mapping)\n- **Responses API**\n  - [Quick start](#quick-start-1)\n  - [The `run_response` callable](#the-run_response-callable)\n  - [Streaming text](#streaming-text)\n  - [Streaming function calls](#streaming-function-calls)\n  - [Streaming reasoning](#streaming-reasoning)\n  - [Returning a Response object](#returning-a-response-object)\n  - [Combining with chat completions](#combining-with-chat-completions)\n  - [Hooks](#hooks-1)\n- [Examples](#examples)\n- [API reference](#api-reference) -- [`create_chat_completion_router`](#create_chat_completion_router) \u0026#183; [`create_models_router`](#create_models_router) \u0026#183; [`create_responses_router`](#create_responses_router) \u0026#183; [`create_files_router`](#create_files_router-minimal-files-upload-support)\n\n## Installation\n\n```bash\npip install fastapi-openai-compat\n```\n\nWith Haystack `StreamingChunk` support:\n\n```bash\npip install fastapi-openai-compat[haystack]\n```\n\n## Quick start\n\nCreate an OpenAI-compatible Chat Completions server in a few lines. Both sync and async\ncallables are supported -- sync callables are automatically executed in a thread pool\nso they never block the async event loop.\n\n```python\nfrom fastapi import FastAPI\nfrom fastapi_openai_compat import CompletionResult, MessageParam, create_chat_completion_router\n\ndef list_models() -\u003e list[str]:\n    return [\"my-pipeline\"]\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    # Your (potentially blocking) pipeline execution logic here\n    return \"Hello from Haystack!\"\n\napp = FastAPI()\nrouter = create_chat_completion_router(\n    list_models=list_models,\n    run_completion=run_completion,\n)\napp.include_router(router)\n```\n\nAsync callables work the same way:\n\n```python\nasync def list_models() -\u003e list[str]:\n    return [\"my-pipeline\"]\n\nasync def run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    return \"Hello from Haystack!\"\n```\n\n## A note on dict-based types\n\nThe type aliases `MessageParam`, `InputItem`, and `OutputItem` are all\n`dict[str, Any]` -- **not** Pydantic models.  This is a deliberate design\nchoice:\n\n- **Forward-compatibility.** OpenAI regularly adds new message types, content\n  part types, and input/output item types.  A strict union of Pydantic models\n  would reject unknown types and require a library release for every API\n  change.  Plain dicts let your callbacks handle new types immediately.\n- **Pass-through design.** This library validates the request *envelope*\n  (correct top-level structure) and passes the inner items through to your\n  callback unchanged.  Domain-specific validation belongs in your callback or\n  a pre-hook, not in the transport layer.\n- **Consistency.** Both Chat Completions (`messages`) and Responses API\n  (`input_items`, `output`) follow the same pattern, so you have a single\n  mental model for both.\n\nThe aliases exist to give you IDE hints and self-documenting signatures.\nEach alias's docstring lists the common dict shapes you'll encounter -- check\nthem in your IDE or in the\n[source](src/fastapi_openai_compat/chat_completions/models.py).\n\n## The `run_completion` callable\n\nThe `run_completion` callable receives three arguments:\n\n| Argument   | Type                  | Description |\n|------------|-----------------------|-------------|\n| `model`    | `str`                 | The model name from the request (e.g. `\"my-pipeline\"`). |\n| `messages` | `list[MessageParam]`  | The conversation history in OpenAI format (see [A note on dict-based types](#a-note-on-dict-based-types)). |\n| `body`     | `dict`                | The full request body, including all extra parameters (e.g. `temperature`, `max_tokens`, `stream`, `metadata`, `tools`). |\n\nThe request model accepts any additional fields beyond `model`, `messages`, and `stream`.\nThese extra parameters are forwarded as-is in the `body` dict, so you can use them\nhowever you need without any library changes.\n\nFor example, you can access `metadata` and any other extra field from `body`:\n\n```python\nimport time\nfrom fastapi_openai_compat import ChatCompletion, Choice, Message, MessageParam, CompletionResult\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    metadata = body.get(\"metadata\", {})\n    temperature = body.get(\"temperature\", 1.0)\n    request_id = metadata.get(\"request_id\", \"unknown\")\n\n    return ChatCompletion(\n        id=f\"resp-{request_id}\",\n        object=\"chat.completion\",\n        created=int(time.time()),\n        model=model,\n        choices=[\n            Choice(\n                index=0,\n                message=Message(role=\"assistant\", content=\"Hello!\"),\n                finish_reason=\"stop\",\n            )\n        ],\n        metadata={\"request_id\": request_id, \"temperature_used\": temperature},\n    )\n```\n\nA client can then send:\n\n```bash\ncurl http://localhost:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"model\": \"my-pipeline\",\n    \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}],\n    \"temperature\": 0.7,\n    \"metadata\": {\"request_id\": \"abc-123\", \"user_tier\": \"premium\"}\n  }'\n```\n\nThe `metadata` field in the response works because `ChatCompletion` also allows extra fields,\nso you can attach any additional data to the response object.\n\nThe return type determines how the response is formatted:\n\n| Return type        | Behavior |\n|--------------------|----------|\n| `str`              | Wrapped automatically into a `ChatCompletion` response. |\n| `Generator`        | Each yielded chunk is converted to a `chat.completion.chunk` SSE message. |\n| `AsyncGenerator`   | Same as `Generator`, but async. |\n| `ChatCompletion`   | Returned as-is for full control over the response. |\n\n## Response types\n\n### Returning a string\n\nThe simplest option -- return a plain string and the library wraps it as a\ncomplete `ChatCompletion` response automatically:\n\n```python\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    last_msg = messages[-1][\"content\"]\n    return f\"You said: {last_msg}\"\n```\n\n### Streaming with a generator\n\nReturn a generator to stream responses token by token via SSE.\nEach yielded string is automatically wrapped into a `chat.completion.chunk` message --\nyou only need to yield the text content, the library handles the SSE wire format.\nA `finish_reason=\"stop\"` sentinel is appended automatically at the end of the stream.\n\nYour `run_completion` should check `body.get(\"stream\", False)` to decide whether\nto return a generator or a plain string:\n\n```python\nfrom collections.abc import Generator\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    last_msg = messages[-1][\"content\"]\n\n    if body.get(\"stream\", False):\n        def stream() -\u003e Generator[str, None, None]:\n            for word in last_msg.split():\n                yield word + \" \"\n        return stream()\n\n    return f\"You said: {last_msg}\"\n```\n\nAsync generators work the same way:\n\n```python\nfrom collections.abc import AsyncGenerator\n\nasync def run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    async def stream() -\u003e AsyncGenerator[str, None]:\n        for word in [\"Hello\", \" from\", \" Haystack\", \"!\"]:\n            yield word\n    return stream()\n```\n\n### Returning a ChatCompletion\n\nFor full control over the response (e.g. custom `usage`, `finish_reason`, or `system_fingerprint`),\nreturn a `ChatCompletion` object directly:\n\n```python\nimport time\nfrom fastapi_openai_compat import ChatCompletion, Choice, Message, CompletionResult\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    return ChatCompletion(\n        id=\"resp-1\",\n        object=\"chat.completion\",\n        created=int(time.time()),\n        model=model,\n        choices=[\n            Choice(\n                index=0,\n                message=Message(role=\"assistant\", content=\"Hello!\"),\n                finish_reason=\"stop\",\n            )\n        ],\n        usage={\"prompt_tokens\": 10, \"completion_tokens\": 5, \"total_tokens\": 15},\n    )\n```\n\n## Tool calling\n\n### Returning ChatCompletion directly\n\nFor tool calls and other advanced responses, return a `ChatCompletion` directly\nfrom `run_completion` for full control over the response structure:\n\n```python\nimport time\nfrom fastapi_openai_compat import ChatCompletion, Choice, Message, CompletionResult\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    return ChatCompletion(\n        id=\"resp-1\",\n        object=\"chat.completion\",\n        created=int(time.time()),\n        model=model,\n        choices=[\n            Choice(\n                index=0,\n                message=Message(\n                    role=\"assistant\",\n                    content=None,\n                    tool_calls=[{\n                        \"id\": \"call_1\",\n                        \"type\": \"function\",\n                        \"function\": {\"name\": \"get_weather\", \"arguments\": '{\"city\": \"Paris\"}'},\n                    }],\n                ),\n                finish_reason=\"tool_calls\",\n            )\n        ],\n    )\n```\n\nStreaming tool calls work the same way -- yield `ChatCompletion` chunk objects\nfrom your generator and the library serializes them directly as SSE:\n\n```python\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    def stream():\n        yield ChatCompletion(\n            id=\"resp-1\", object=\"chat.completion.chunk\",\n            created=int(time.time()), model=model,\n            choices=[Choice(index=0, delta=Message(\n                role=\"assistant\",\n                tool_calls=[{\"index\": 0, \"id\": \"call_1\", \"type\": \"function\",\n                             \"function\": {\"name\": \"get_weather\", \"arguments\": \"\"}}],\n            ))],\n        )\n        yield ChatCompletion(\n            id=\"resp-1\", object=\"chat.completion.chunk\",\n            created=int(time.time()), model=model,\n            choices=[Choice(index=0, delta=Message(\n                role=\"assistant\",\n                tool_calls=[{\"index\": 0, \"function\": {\"arguments\": '{\"city\": \"Paris\"}'}}],\n            ))],\n        )\n        yield ChatCompletion(\n            id=\"resp-1\", object=\"chat.completion.chunk\",\n            created=int(time.time()), model=model,\n            choices=[Choice(index=0, delta=Message(role=\"assistant\"), finish_reason=\"tool_calls\")],\n        )\n    return stream()\n```\n\n### Automatic StreamingChunk support\n\nWhen using Haystack's `StreamingChunk` (requires `pip install fastapi-openai-compat[haystack]`),\ntool call deltas and finish reasons are handled automatically via duck typing:\n\n```python\nfrom haystack.dataclasses import StreamingChunk\nfrom haystack.dataclasses.streaming_chunk import ToolCallDelta\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    def stream():\n        yield StreamingChunk(\n            content=\"\",\n            tool_calls=[ToolCallDelta(\n                index=0, id=\"call_1\",\n                tool_name=\"get_weather\", arguments='{\"city\": \"Paris\"}',\n            )],\n            index=0,\n        )\n        yield StreamingChunk(content=\"\", finish_reason=\"tool_calls\")\n    return stream()\n```\n\nThe library automatically:\n\n- Converts `ToolCallDelta` objects to OpenAI wire format (`tool_calls[].function.name/arguments`)\n- Propagates `finish_reason` from chunks (e.g. `\"stop\"`, `\"tool_calls\"`, `\"length\"`)\n- Only auto-appends `finish_reason=\"stop\"` if no chunk already carried a finish reason\n- Works via duck typing -- any object with `tool_calls` and `finish_reason` attributes is supported\n\n## Reasoning content\n\nWhen streaming, chunks with a `reasoning` attribute are emitted as\n`reasoning_content` on the message delta (the\n[DeepSeek convention](https://api-docs.deepseek.com/guides/reasoning_model),\nwidely adopted by OpenAI-compatible clients). This is detected via duck\ntyping -- any object with a `.reasoning` attribute works.\n\nWith Haystack's `StreamingChunk` and `ReasoningContent`:\n\n```python\nfrom haystack.dataclasses import StreamingChunk\nfrom haystack.dataclasses.streaming_chunk import ReasoningContent\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    def stream():\n        yield StreamingChunk(content=\"\", reasoning=ReasoningContent(reasoning_text=\"Let me think...\"))\n        yield StreamingChunk(content=\"\", reasoning=ReasoningContent(reasoning_text=\" The answer is 42.\"))\n        yield StreamingChunk(content=\"The answer is 42.\")\n    return stream()\n```\n\nThe resulting SSE stream contains chunks with `reasoning_content` on the delta\nfor reasoning tokens, followed by regular `content` deltas for the visible\nanswer:\n\n```json\n{\"choices\": [{\"delta\": {\"reasoning_content\": \"Let me think...\", \"content\": null}}]}\n{\"choices\": [{\"delta\": {\"reasoning_content\": \" The answer is 42.\", \"content\": null}}]}\n{\"choices\": [{\"delta\": {\"content\": \"The answer is 42.\"}}]}\n```\n\nAny custom object works too -- the library checks for `chunk.reasoning` and\nextracts text via `.reasoning_text` (falling back to `str()`):\n\n```python\nclass ReasoningChunk:\n    def __init__(self, reasoning_text: str):\n        self.reasoning = type(\"R\", (), {\"reasoning_text\": reasoning_text})()\n\ndef stream():\n    yield ReasoningChunk(\"Thinking step 1...\")\n    yield \"Final answer\"\n```\n\n## Custom SSE events\n\nYou can yield custom SSE events alongside regular chat completion chunks. This is useful\nfor sending side-channel data to clients like [Open WebUI](https://openwebui.com) --\nstatus updates, notifications, source citations, etc.\n\nAny object with a `.to_event_dict()` method is recognized as a custom event and serialized\nas `data: {\"event\": {...}}` in the SSE stream. Custom events don't interfere with\nchat completion chunks or the `finish_reason` tracking.\n\n```python\nfrom collections.abc import Generator\nfrom fastapi_openai_compat import CompletionResult\n\nclass StatusEvent:\n    def __init__(self, description: str, done: bool = False):\n        self.description = description\n        self.done = done\n\n    def to_event_dict(self) -\u003e dict:\n        return {\"type\": \"status\", \"data\": {\"description\": self.description, \"done\": self.done}}\n\ndef run_completion(model: str, messages: list[MessageParam], body: dict) -\u003e CompletionResult:\n    def stream() -\u003e Generator[str | StatusEvent, None, None]:\n        yield StatusEvent(\"Processing your request...\")\n        for word in [\"Hello\", \" from\", \" Haystack\", \"!\"]:\n            yield word\n        yield StatusEvent(\"Done\", done=True)\n    return stream()\n```\n\nThis works via duck typing -- any object implementing `to_event_dict() -\u003e dict` is supported.\nThe protocol is compatible with [Hayhooks' Open WebUI events](https://deepset-ai.github.io/hayhooks/).\n\n## Hooks\n\nYou can inject pre/post hooks to modify requests and results (transformer hooks)\nor to observe them without modification (observer hooks). Both sync and async\nhooks are supported.\n\n### Transformer hooks\n\nReturn a modified value to transform the request or result:\n\n```python\nfrom fastapi_openai_compat import ChatRequest, CompletionResult\n\nasync def pre_hook(request: ChatRequest) -\u003e ChatRequest:\n    # e.g. inject system prompts, validate, rate-limit\n    return request\n\nasync def post_hook(result: CompletionResult) -\u003e CompletionResult:\n    # e.g. transform, filter\n    return result\n\nrouter = create_chat_completion_router(\n    list_models=list_models,\n    run_completion=run_completion,\n    pre_hook=pre_hook,\n    post_hook=post_hook,\n)\n```\n\n### Observer hooks\n\nReturn `None` to observe without modifying (useful for logging, metrics, etc.):\n\n```python\ndef log_request(request: ChatRequest) -\u003e None:\n    print(f\"Request for model: {request.model}\")\n\ndef log_result(result: CompletionResult) -\u003e None:\n    print(f\"Got result type: {type(result).__name__}\")\n\nrouter = create_chat_completion_router(\n    list_models=list_models,\n    run_completion=run_completion,\n    pre_hook=log_request,\n    post_hook=log_result,\n)\n```\n\n## Custom chunk mapping\n\nBy default the router handles plain `str` chunks and objects with a `.content`\nattribute (e.g. Haystack `StreamingChunk`). If your pipeline streams a different\ntype, provide a `chunk_mapper` to extract text content:\n\n```python\nfrom dataclasses import dataclass\n\n@dataclass\nclass MyChunk:\n    text: str\n    score: float\n\ndef my_mapper(chunk: MyChunk) -\u003e str:\n    return chunk.text\n\nrouter = create_chat_completion_router(\n    list_models=list_models,\n    run_completion=run_completion,\n    chunk_mapper=my_mapper,\n)\n```\n\nThis works with any object -- dataclasses, dicts, Pydantic models, etc.:\n\n```python\ndef dict_mapper(chunk: dict) -\u003e str:\n    return chunk[\"payload\"]\n```\n\n## Responses API\n\nThe Responses API uses named SSE events (matching the\n[OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses))\ninstead of the `data:`-only format used by chat completions.\n\n### Quick start\n\n```python\nfrom fastapi import FastAPI\nfrom fastapi_openai_compat import InputItem, ResponseResult, create_responses_router\n\ndef list_models() -\u003e list[str]:\n    return [\"my-pipeline\"]\n\ndef run_response(model: str, input_items: list[InputItem], body: dict) -\u003e ResponseResult:\n    return \"Hello from the Responses API!\"\n\napp = FastAPI()\napp.include_router(\n    create_responses_router(\n        list_models=list_models,\n        run_response=run_response,\n        include_models_endpoints=True,\n    )\n)\n```\n\n### The `run_response` callable\n\nThe `run_response` callable receives three arguments:\n\n| Argument       | Type               | Description |\n|----------------|--------------------|-------------|\n| `model`        | `str`              | The model name from the request. |\n| `input_items`  | `list[InputItem]`  | Normalized input items (see [A note on dict-based types](#a-note-on-dict-based-types)). String shorthand is converted to a message item; `None` becomes `[]`. |\n| `body`         | `dict`             | The full request body, including all extra parameters (e.g. `temperature`, `tools`, `instructions`). |\n\nThe return type determines how the response is formatted:\n\n| Return type      | Behavior |\n|------------------|----------|\n| `str`            | Wrapped into a `Response` with a single text output message. |\n| `Generator`      | Each yielded chunk is emitted as named SSE events (`response.output_text.delta`, etc.). |\n| `AsyncGenerator` | Same as `Generator`, but async. |\n| `Response`       | Returned as-is for full control over the response. |\n\n### Streaming text\n\nReturn a generator to stream text via named SSE events. Each yielded string\nbecomes a `response.output_text.delta` event. The library handles all the\nsurrounding lifecycle events (`response.created`, `response.in_progress`,\n`response.output_item.added`, `response.completed`, etc.) automatically.\n\n```python\nfrom collections.abc import Generator\n\ndef run_response(model: str, input_items: list[InputItem], body: dict) -\u003e ResponseResult:\n    if body.get(\"stream\", False):\n        def stream() -\u003e Generator[str, None, None]:\n            for word in [\"Hello\", \" from\", \" streaming\", \"!\"]:\n                yield word\n        return stream()\n    return \"Hello!\"\n```\n\n### Streaming function calls\n\nYield objects with `function_call_id`, `function_call_name`, and\n`function_call_arguments` attributes to stream function call events.\nThe library emits `response.function_call_arguments.delta` events during\nstreaming and a `response.output_item.done` event when the call completes.\n\n```python\nclass FunctionCallChunk:\n    def __init__(self, *, call_id: str, name: str | None, arguments: str | None):\n        self.function_call_id = call_id\n        self.function_call_name = name\n        self.function_call_arguments = arguments\n\ndef run_response(model: str, input_items: list[InputItem], body: dict) -\u003e ResponseResult:\n    def stream():\n        yield FunctionCallChunk(call_id=\"call_1\", name=\"get_weather\", arguments='{\"city\":')\n        yield FunctionCallChunk(call_id=\"call_1\", name=None, arguments=' \"Paris\"}')\n    return stream()\n```\n\n### Streaming reasoning\n\nYield chunks with a `reasoning` attribute to stream reasoning output items.\nThe library emits the proper OpenAI\n[reasoning summary events](https://platform.openai.com/docs/api-reference/responses-streaming)\n(`response.reasoning_summary_part.added`, `response.reasoning_summary_text.delta`,\n`response.reasoning_summary_text.done`, `response.reasoning_summary_part.done`)\nand produces a `type: \"reasoning\"` output item with a `summary` array in the\ncompleted response.\n\n```python\nfrom haystack.dataclasses import StreamingChunk\nfrom haystack.dataclasses.streaming_chunk import ReasoningContent\n\ndef run_response(model: str, input_items: list[InputItem], body: dict) -\u003e ResponseResult:\n    def stream():\n        yield StreamingChunk(content=\"\", reasoning=ReasoningContent(reasoning_text=\"Step 1: \"))\n        yield StreamingChunk(content=\"\", reasoning=ReasoningContent(reasoning_text=\"analyze the input.\"))\n        yield StreamingChunk(content=\"Here is the answer.\")\n    return stream()\n```\n\nThe completed `Response` object includes the reasoning item before the message:\n\n```json\n{\n  \"output\": [\n    {\"type\": \"reasoning\", \"summary\": [{\"type\": \"summary_text\", \"text\": \"Step 1: analyze the input.\"}]},\n    {\"type\": \"message\", \"content\": [{\"type\": \"output_text\", \"text\": \"Here is the answer.\"}]}\n  ]\n}\n```\n\n### Returning a Response object\n\nFor full control, return a `Response` directly:\n\n```python\nimport time\nimport uuid\nfrom fastapi_openai_compat import Response\n\ndef run_response(model: str, input_items: list[InputItem], body: dict) -\u003e ResponseResult:\n    return Response(\n        id=f\"resp_{uuid.uuid4().hex}\",\n        created_at=int(time.time()),\n        model=model,\n        output=[{\n            \"id\": f\"msg_{uuid.uuid4().hex}\",\n            \"type\": \"message\",\n            \"status\": \"completed\",\n            \"role\": \"assistant\",\n            \"content\": [{\"type\": \"output_text\", \"text\": \"Hello!\", \"annotations\": []}],\n        }],\n    )\n```\n\n### Combining with chat completions\n\nWhen mounting both routers in the same app, use a dedicated `create_models_router`\nto avoid duplicate `/v1/models` endpoints:\n\n```python\nfrom fastapi import FastAPI\nfrom fastapi_openai_compat import (\n    create_chat_completion_router,\n    create_models_router,\n    create_responses_router,\n)\n\napp = FastAPI()\napp.include_router(create_models_router(list_models=list_models))\napp.include_router(\n    create_chat_completion_router(\n        list_models=list_models,\n        run_completion=run_completion,\n        include_models_endpoints=False,\n    )\n)\napp.include_router(\n    create_responses_router(\n        list_models=list_models,\n        run_response=run_response,\n        include_models_endpoints=False,\n    )\n)\n```\n\n### Hooks\n\nPre/post hooks work the same way as chat completions. The pre-hook receives\na `ResponseRequest` and the post-hook receives a `ResponseResult`:\n\n```python\nfrom fastapi_openai_compat import ResponseRequest\n\nasync def pre_hook(request: ResponseRequest) -\u003e ResponseRequest:\n    # e.g. inject instructions, validate, rate-limit\n    return request\n\nrouter = create_responses_router(\n    list_models=list_models,\n    run_response=run_response,\n    pre_hook=pre_hook,\n)\n```\n\n## Examples\n\nThe [`examples/`](examples/) folder contains ready-to-run servers:\n\n- **[`basic.py`](examples/basic.py)** -- Minimal echo server, no external API keys required.\n- **[`haystack_chat.py`](examples/haystack_chat.py)** -- Haystack `OpenAIChatGenerator` with streaming support.\n- **[`responses_basic.py`](examples/responses_basic.py)** -- Responses API text + streaming + function call demo.\n- **[`responses_with_files.py`](examples/responses_with_files.py)** -- Responses API with `/v1/files` upload + `input_file.file_id`.\n\nSee the [examples README](examples/README.md) for setup and usage instructions.\n\n## API reference\n\nThis library implements endpoints compatible with the [OpenAI Chat Completions API](https://platform.openai.com/docs/api-reference/chat), the [OpenAI Responses API](https://platform.openai.com/docs/api-reference/responses), and a minimal Files upload router.\n\n### `create_chat_completion_router`\n\n`create_openai_router` is still available as a backward-compatible alias.\n\n```python\ncreate_chat_completion_router(\n    *,\n    list_models,\n    run_completion,\n    pre_hook=None,\n    post_hook=None,\n    chunk_mapper=default_chunk_mapper,\n    owned_by=\"custom\",\n    tags=None,\n    include_models_endpoints=True,\n) -\u003e APIRouter\n```\n\n| Parameter        | Type                      | Description |\n|------------------|---------------------------|-------------|\n| `list_models`    | `Callable -\u003e list[str]`   | Returns available model/pipeline names. |\n| `run_completion` | `Callable -\u003e CompletionResult` | Runs a chat completion given `(model, messages: list[MessageParam], body)`. |\n| `pre_hook`       | `Callable` or `None`      | Called before `run_completion`. Receives `ChatRequest`, returns modified request (transformer) or `None` (observer). |\n| `post_hook`      | `Callable` or `None`      | Called after `run_completion`. Receives `CompletionResult`, returns modified result (transformer) or `None` (observer). |\n| `chunk_mapper`   | `Callable[[Any], str]`    | Converts streamed chunks to strings. Default handles `str` and `.content` attribute. |\n| `owned_by`       | `str`                     | Value for the `owned_by` field in model objects. Defaults to `\"custom\"`. |\n| `tags`           | `list[str]` or `None`     | OpenAPI tags for the generated endpoints. Defaults to `[\"openai\"]`. |\n| `include_models_endpoints` | `bool`          | If true, includes `/v1/models` and `/models` in this router. Defaults to `True`. |\n\n### Endpoints\n\nWith `include_models_endpoints=True` (default), the router exposes:\n\n| Method | Path                        | Description |\n|--------|-----------------------------|-------------|\n| `GET`  | `/v1/models`                | List available models. |\n| `POST` | `/v1/chat/completions`      | Create a chat completion (streaming or non-streaming). |\n| `GET`  | `/models`                   | Alias for `/v1/models`. |\n| `POST` | `/chat/completions`         | Alias for `/v1/chat/completions`. |\n\n### `create_models_router`\n\n```python\ncreate_models_router(\n    *,\n    list_models,\n    owned_by=\"custom\",\n    tags=None,\n    operation_id_prefix=\"openai\",\n) -\u003e APIRouter\n```\n\nUse this when composing multiple routers and you want a single owner for\n`/v1/models`. See [Combining with chat completions](#combining-with-chat-completions)\nfor a full example.\n\n### `create_responses_router`\n\n```python\ncreate_responses_router(\n    *,\n    list_models,\n    run_response,\n    pre_hook=None,\n    post_hook=None,\n    chunk_mapper=default_chunk_mapper,\n    owned_by=\"custom\",\n    tags=None,\n    include_models_endpoints=False,\n) -\u003e APIRouter\n```\n\n| Parameter                 | Type                      | Description |\n|---------------------------|---------------------------|-------------|\n| `list_models`             | `Callable -\u003e list[str]`   | Returns available model/pipeline names. |\n| `run_response`            | `Callable -\u003e ResponseResult` | Runs a Responses request given `(model, input_items: list[InputItem], body)`. |\n| `pre_hook`                | `Callable` or `None`      | Called before `run_response`. Receives `ResponseRequest`, returns modified request (transformer) or `None` (observer). |\n| `post_hook`               | `Callable` or `None`      | Called after `run_response`. Receives `ResponseResult`, returns modified result (transformer) or `None` (observer). |\n| `chunk_mapper`            | `Callable[[Any], str]`    | Converts streamed non-string chunks to strings. |\n| `owned_by`                | `str`                     | Value for `owned_by` in model objects when models endpoints are enabled. Defaults to `\"custom\"`. |\n| `tags`                    | `list[str]` or `None`     | OpenAPI tags for generated endpoints. Defaults to `[\"openai\"]`. |\n| `include_models_endpoints`| `bool`                    | If true, includes `/v1/models` and `/models` in this router. Defaults to `False` to avoid conflicts when combined with chat or a dedicated models router. |\n\nResponses router endpoints:\n\n| Method | Path            | Description |\n|--------|-----------------|-------------|\n| `POST` | `/v1/responses` | Create a Responses API response (streaming or non-streaming). |\n| `POST` | `/responses`    | Alias for `/v1/responses`. |\n\n### `create_files_router` (minimal Files upload support)\n\n```python\ncreate_files_router(\n    *,\n    run_file_upload,\n    tags=None,\n) -\u003e APIRouter\n```\n\nThe `run_file_upload` callback receives:\n\n- `filename`: uploaded filename, if present\n- `content_type`: uploaded content type, if present\n- `content`: full uploaded file bytes\n- `purpose`: multipart form `purpose` field\n\nIt can return either:\n\n- `FileObject`\n- `dict` matching the `FileObject` schema\n\nThe router exposes:\n\n| Method | Path         | Description |\n|--------|--------------|-------------|\n| `POST` | `/v1/files`  | Upload a file (`files.create(...)` compatible). |\n| `POST` | `/files`     | Alias for `/v1/files`. |\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepset-ai%2Ffastapi-openai-compat","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeepset-ai%2Ffastapi-openai-compat","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeepset-ai%2Ffastapi-openai-compat/lists"}