{"id":36419187,"url":"https://github.com/softwaremill/sttp-ai","last_synced_at":"2026-04-10T11:04:30.006Z","repository":{"id":152939933,"uuid":"622889115","full_name":"softwaremill/sttp-ai","owner":"softwaremill","description":"Scala Client for AI models","archived":false,"fork":false,"pushed_at":"2026-04-08T15:25:12.000Z","size":1369,"stargazers_count":90,"open_issues_count":16,"forks_count":16,"subscribers_count":15,"default_branch":"master","last_synced_at":"2026-04-08T17:26:27.067Z","etag":null,"topics":["ai","api","claude","client","openai","scala"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/softwaremill.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2023-04-03T09:13:10.000Z","updated_at":"2026-04-08T15:26:15.000Z","dependencies_parsed_at":null,"dependency_job_id":"9d0f0308-0a3a-40b6-aef0-83c1c7ccd635","html_url":"https://github.com/softwaremill/sttp-ai","commit_stats":{"total_commits":233,"total_committers":17,"mean_commits":"13.705882352941176","dds":0.7639484978540773,"last_synced_commit":"abe808975edf01ca6a6ce65132aae8a3d7a38af3"},"previous_names":["softwaremill/sttp-ai"],"tags_count":42,"template":false,"template_full_name":"softwaremill/sbt-template","purl":"pkg:github/softwaremill/sttp-ai","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fsttp-ai","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fsttp-ai/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fsttp-ai/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fsttp-ai/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/softwaremill","download_url":"https://codeload.github.com/softwaremill/sttp-ai/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/softwaremill%2Fsttp-ai/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31639526,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-10T07:40:12.752Z","status":"ssl_error","status_checked_at":"2026-04-10T07:40:11.664Z","response_time":98,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ai","api","claude","client","openai","scala"],"created_at":"2026-01-11T17:02:13.004Z","updated_at":"2026-04-10T11:04:29.968Z","avatar_url":"https://github.com/softwaremill.png","language":"Scala","funding_links":[],"categories":[],"sub_categories":[],"readme":"![sttp-ai](https://github.com/softwaremill/sttp-ai/raw/master/banner.png)\n\n[![Ideas, suggestions, problems, questions](https://img.shields.io/badge/Discourse-ask%20question-blue)](https://softwaremill.community/c/open-source)\n[![CI](https://github.com/softwaremill/sttp-ai/workflows/CI/badge.svg)](https://github.com/softwaremill/sttp-ai/actions?query=workflow%3ACI+branch%3Amaster)\n\n[![sttp.ai:core](https://maven-badges.sml.io/sonatype-central/com.softwaremill.sttp.ai/core_3/badge.svg?subject=sttp.ai:core)](https://maven-badges.sml.io/sonatype-central/com.softwaremill.sttp.ai/core_3/)\n[![sttp.ai:openai](https://maven-badges.sml.io/sonatype-central/com.softwaremill.sttp.ai/openai_3/badge.svg?subject=sttp.ai:openai)](https://maven-badges.sml.io/sonatype-central/com.softwaremill.sttp.ai/openai_3/)\n[![sttp.ai:claude](https://maven-badges.sml.io/sonatype-central/com.softwaremill.sttp.ai/claude_3/badge.svg?subject=sttp.ai:claude)](https://maven-badges.sml.io/sonatype-central/com.softwaremill.sttp.ai/claude_3/)\n\nsttp is a family of Scala HTTP-related projects, and currently includes:\n\n* [sttp client](https://github.com/softwaremill/sttp): The Scala HTTP client you always wanted!\n* [sttp tapir](https://github.com/softwaremill/tapir): Typed API descRiptions\n* sttp ai: this project. Non-official Scala client wrapper for OpenAI, Claude (Anthropic), and OpenAI-compatible APIs. Use the power of ChatGPT and Claude inside your code!\n\n## Table of Contents\n\n- [Intro](#intro)\n- [Quickstart](#quickstart)\n  - [OpenAI/OpenAI-compatible APIs](#for-openaiopenai-compatible-apis)\n  - [Claude (Anthropic) API](#for-claude-anthropic-api)\n- [OpenAI API](#openai-api)\n  - [Basic Usage](#basic-usage-openai)\n  - [Streaming](#streaming-openai)\n  - [Structured Outputs/JSON Schema](#structured-outputsjson-schema-support)\n  - [Function/Tool Calling](#generating-json-schema-from-case-class)\n- [Claude API](#claude-api)\n  - [Features](#claude-features)\n  - [Basic Usage](#basic-usage-claude)\n  - [Configuration](#claude-configuration)\n  - [Messages API](#claude-messages-api)\n  - [Tool Calling](#claude-tool-calling)\n  - [Streaming](#claude-streaming)\n  - [Models API](#claude-models-api)\n  - [Error Handling](#claude-error-handling)\n  - [Key Differences from OpenAI](#key-differences-from-openai-api)\n  - [Synchronous Claude Client](#synchronous-claude-client)\n- [OpenAI-Compatible APIs](#openai-compatible-apis)\n- [Examples](#examples)\n- [Contributing](#contributing)\n- [Commercial Support](#commercial-support)\n- [Copyright](#copyright)\n\n## Intro\n\nsttp-ai uses sttp client to describe requests and responses used in OpenAI, Claude (Anthropic), and OpenAI-compatible endpoints.\n\n## Quickstart\n\n### For OpenAI/OpenAI-compatible APIs\n\nAdd the following dependency:\n\n```sbt\n\"com.softwaremill.sttp.ai\" %% \"openai\" % \"0.4.3\"\n```\n\n### For Claude (Anthropic) API\n\nAdd the following dependency:\n\n```sbt\n\"com.softwaremill.sttp.ai\" %% \"claude\" % \"0.4.3\"\n\n// For streaming support, add one or more:\n\"com.softwaremill.sttp.ai\" %% \"claude-streaming-fs2\" % \"0.4.3\"    // cats-effect/fs2\n\"com.softwaremill.sttp.ai\" %% \"claude-streaming-zio\" % \"0.4.3\"    // ZIO\n\"com.softwaremill.sttp.ai\" %% \"claude-streaming-akka\" % \"0.4.3\"   // Akka Streams (Scala 2.13 only)\n\"com.softwaremill.sttp.ai\" %% \"claude-streaming-pekko\" % \"0.4.3\"  // Pekko Streams\n\"com.softwaremill.sttp.ai\" %% \"claude-streaming-ox\" % \"0.4.3\"    // Ox direct-style (Scala 3 only)\n```\n\nsttp-openai is available for Scala 2.13 and Scala 3\n\n## OpenAI API\n\nOpenAI API Official Documentation: https://platform.openai.com/docs/api-reference/completions\n\nExamples are runnable using [scala-cli](https://scala-cli.virtuslab.org).\n\n### Basic Usage (OpenAI)\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::openai:0.4.3\n\nimport sttp.ai.openai.OpenAISyncClient\nimport sttp.ai.openai.requests.completions.chat.ChatRequestResponseData.ChatResponse\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.{ChatBody, ChatCompletionModel}\nimport sttp.ai.openai.requests.completions.chat.message.*\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    val apiKey = System.getenv(\"OPENAI_KEY\")\n    val openAI = OpenAISyncClient(apiKey)\n\n    // Create body of Chat Completions Request\n    val bodyMessages: Seq[Message] = Seq(\n      Message.UserMessage(\n        content = Content.TextContent(\"Hello!\"),\n      )\n    )\n\n    // use ChatCompletionModel.CustomChatCompletionModel(\"gpt-some-future-version\")\n    // for models not yet supported here\n    val chatRequestBody: ChatBody = ChatBody(\n      model = ChatCompletionModel.GPT4oMini,\n      messages = bodyMessages\n    )\n\n    // be aware that calling `createChatCompletion` may throw an OpenAIException\n    // e.g. AuthenticationException, RateLimitException and many more\n    val chatResponse: ChatResponse = openAI.createChatCompletion(chatRequestBody)\n\n    println(chatResponse)\n    /*\n        ChatResponse(\n         chatcmpl-79shQITCiqTHFlI9tgElqcbMTJCLZ,chat.completion,\n         1682589572,\n         gpt-4o-mini,\n         Usage(10,10,20),\n         List(\n           Choices(\n             Message(assistant, Hello there! How can I assist you today?), stop, 0)\n           )\n         )\n    */\n```\n\n## Claude API\n\nThis module provides **native support for Anthropic's Claude API** within the sttp-openai library. Unlike OpenAI compatibility layers, this provides direct access to Claude's unique features and API structure.\n\n### Claude Features\n\n- ✅ **Native Claude API support** - Direct Claude API integration, not compatibility layer\n- ✅ **ContentBlock structure** - Support for Claude's rich message content blocks (text, images)\n- ✅ **Proper Authentication** - Uses `x-api-key` and `anthropic-version` headers\n- ✅ **Messages API** - Complete `/v1/messages` endpoint implementation\n- ✅ **Models API** - List available Claude models via `/v1/models`\n- ✅ **Streaming Support** - Server-Sent Events streaming for all effect systems (fs2, ZIO, Akka, Pekko, Ox)\n- ✅ **Tool Calling** - Native Claude tool calling support\n- ✅ **Image Support** - Multi-modal inputs via ContentBlock with base64 encoding\n- ✅ **Comprehensive Error Handling** - Claude-specific exception hierarchy\n- ✅ **System Messages** - Proper system message handling via `system` parameter\n- ✅ **Cross-platform** - Support for Scala 2.13 and Scala 3\n\n### Basic Usage (Claude)\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::claude:0.4.3\n\nimport sttp.ai.claude.*\nimport sttp.ai.claude.config.ClaudeConfig\nimport sttp.ai.claude.models.{ContentBlock, Message}\nimport sttp.ai.claude.requests.MessageRequest\nimport sttp.client4.*\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    // Create an instance of ClaudeClient using your Anthropic API key\n    // Set ANTHROPIC_API_KEY environment variable or pass it directly\n    val config = ClaudeConfig.fromEnv  // reads ANTHROPIC_API_KEY\n    val backend: SyncBackend = DefaultSyncBackend()\n    val client = ClaudeClient(config)\n\n    // Create a simple message\n    val messages = List(\n      Message.user(List(ContentBlock.TextContent(\"Hello Claude! What's the weather like today?\")))\n    )\n\n    val request = MessageRequest.simple(\n      model = \"claude-3-haiku-20240307\",  // Fast, cost-effective model\n      messages = messages,\n      maxTokens = 500\n    )\n\n    // Send the request (returns Either[ClaudeException, MessageResponse])\n    val response = client.createMessage(request).send(backend)\n\n    response.body match {\n      case Right(messageResponse) =\u003e\n        messageResponse.content.foreach {\n          case ContentBlock.TextContent(text) =\u003e println(text)\n          case _ =\u003e // Handle other content types if needed\n        }\n        println(s\"Usage: ${messageResponse.usage}\")\n      case Left(error) =\u003e\n        println(s\"Claude API Error: ${error.getMessage}\")\n    }\n\n    backend.close()\n```\n\n**Key differences from OpenAI:**\n- Uses `ContentBlock` instead of simple strings for rich content (text, images)\n- Separate system parameter instead of system role messages\n- Different authentication headers (`x-api-key` + `anthropic-version`)\n- Native Claude model names (e.g., `claude-3-haiku-20240307`)\n\n### Claude Configuration\n\n```scala\ncase class ClaudeConfig(\n  apiKey: String,                                    // Your Anthropic API key\n  anthropicVersion: String = \"2023-06-01\",          // API version header\n  baseUrl: Uri = \"https://api.anthropic.com\",       // API base URL\n  timeout: Duration = 60.seconds,                   // Request timeout\n  maxRetries: Int = 3,                             // Max retry attempts\n  organization: Option[String] = None               // Optional organization ID\n)\n```\n\n**Environment Variables:**\n- `ANTHROPIC_API_KEY` - Your API key (required)\n- `ANTHROPIC_VERSION` - API version (optional, defaults to \"2023-06-01\")\n- `ANTHROPIC_BASE_URL` - Custom base URL (optional)\n\n### Claude Messages API\n\n#### Basic Text Conversation\n\n```scala\nval messages = List(\n  Message.user(List(ContentBlock.text(\"What is the capital of France?\"))),\n  Message.assistant(List(ContentBlock.text(\"The capital of France is Paris.\"))),\n  Message.user(List(ContentBlock.text(\"What about Italy?\")))\n)\n\nval request = MessageRequest.simple(\n  model = \"claude-3-sonnet-20240229\",\n  messages = messages,\n  maxTokens = 1000\n)\n```\n\n#### System Messages\n\nUnlike OpenAI, Claude uses a separate `system` parameter instead of system role messages:\n\n```scala\nval request = MessageRequest.withSystem(\n  model = \"claude-3-sonnet-20240229\",\n  system = \"You are a helpful assistant that always responds in French.\",\n  messages = List(Message.user(List(ContentBlock.text(\"Hello!\")))),\n  maxTokens = 1000\n)\n```\n\n#### Image Support\n\n```scala\nimport java.util.Base64\nimport java.nio.file.{Files, Paths}\n\n// Read and encode image\nval imageBytes = Files.readAllBytes(Paths.get(\"image.jpg\"))\nval base64Image = Base64.getEncoder.encodeToString(imageBytes)\n\nval messages = List(\n  Message.user(List(\n    ContentBlock.text(\"What do you see in this image?\"),\n    ContentBlock.image(\n      mediaType = \"image/jpeg\",\n      data = base64Image\n    )\n  ))\n)\n\nval request = MessageRequest.simple(\n  model = \"claude-3-sonnet-20240229\",\n  messages = messages,\n  maxTokens = 1000\n)\n```\n\n#### Advanced Parameters\n\n```scala\nval request = MessageRequest(\n  model = \"claude-3-sonnet-20240229\",\n  messages = messages,\n  maxTokens = 4000,\n  temperature = Some(0.7),           // Creativity (0.0 - 1.0)\n  topP = Some(0.9),                  // Nucleus sampling\n  topK = Some(40),                   // Top-k sampling\n  stopSequences = Some(List(\"\\n\\n\")), // Stop generation at sequences\n  system = Some(\"Be concise and helpful.\"),\n  tools = Some(tools)                // Tool calling support\n)\n```\n\n### Claude Tool Calling\n\n```scala\nimport sttp.ai.claude.models.{Tool, ToolInputSchema, PropertySchema}\n\nval weatherTool = Tool(\n  name = \"get_weather\",\n  description = \"Get current weather for a location\",\n  inputSchema = ToolInputSchema(\n    `type` = \"object\",\n    properties = Map(\n      \"location\" -\u003e PropertySchema(`type` = \"string\", description = Some(\"City name\")),\n      \"unit\" -\u003e PropertySchema(`type` = \"string\", enum = Some(List(\"celsius\", \"fahrenheit\")))\n    ),\n    required = Some(List(\"location\"))\n  )\n)\n\nval request = MessageRequest.withTools(\n  model = \"claude-3-sonnet-20240229\",\n  messages = List(Message.user(List(ContentBlock.text(\"What's the weather in Paris?\")))),\n  maxTokens = 1000,\n  tools = List(weatherTool)\n)\n```\n\n### Claude Streaming\n\n#### Using fs2 (cats-effect)\n\n```scala\nimport sttp.ai.claude.streaming.fs2.*\nimport sttp.client4.httpclient.fs2.HttpClientFs2Backend\nimport cats.effect.IO\n\nval backend = HttpClientFs2Backend[IO]()\n\n// Extension method for streaming\nval streamRequest = client.createMessageAsBinaryStream(backend.capabilities.streams, request)\n\nstreamRequest\n  .send(backend)\n  .map(_.map(_.parseSSE.parseClaudeStreamResponse))\n  .flatMap {\n    case Right(stream) =\u003e\n      stream\n        .evalTap(response =\u003e IO.println(response.delta.text.getOrElse(\"\")))\n        .compile\n        .drain\n    case Left(error) =\u003e\n      IO.println(s\"Error: $error\")\n  }\n```\n\n#### Using ZIO\n\n```scala\nimport sttp.ai.claude.streaming.zio.*\nimport sttp.client4.httpclient.zio.HttpClientZioBackend\nimport zio.*\n\nval backend = HttpClientZioBackend()\n\nval program = for {\n  streamRequest \u003c- ZIO.succeed(client.createMessageAsBinaryStream(backend.capabilities.streams, request))\n  result \u003c- streamRequest.send(backend)\n  _ \u003c- result match {\n    case Right(stream) =\u003e\n      stream\n        .parseSSE\n        .parseClaudeStreamResponse\n        .tap(response =\u003e Console.printLine(response.delta.text.getOrElse(\"\")))\n        .runDrain\n    case Left(error) =\u003e\n      Console.printLine(s\"Error: $error\")\n  }\n} yield ()\n```\n\n#### Using Ox (Scala 3)\n\n```scala\nimport sttp.ai.claude.streaming.ox.*\nimport sttp.client4.ox.OxHttpClientBackend\nimport ox.*\n\nval backend = OxHttpClientBackend()\n\nval streamRequest = client.createMessageAsBinaryStream(backend.capabilities.streams, request)\n\nval result = streamRequest.send(backend)\nresult match {\n  case Right(stream) =\u003e\n    stream\n      .parseSSE\n      .parseClaudeStreamResponse\n      .tap(response =\u003e println(response.delta.text.getOrElse(\"\")))\n      .runDrain()\n  case Left(error) =\u003e\n    println(s\"Error: $error\")\n}\n```\n\n### Claude Models API\n\n```scala\nval modelsRequest = client.listModels()\nval models = modelsRequest.send(backend)\n\nmodels match {\n  case Right(response) =\u003e\n    response.data.foreach(model =\u003e println(s\"${model.id} - ${model.displayName.getOrElse(\"N/A\")}\"))\n  case Left(error) =\u003e\n    println(s\"Error: $error\")\n}\n```\n\n**Common Claude models** (use `listModels()` for current list):\n\n- `claude-3-sonnet-20240229` - Balanced performance and speed\n- `claude-3-opus-20240229` - Highest capability model\n- `claude-3-haiku-20240307` - Fastest model\n- `claude-instant-1.2` - Legacy fast model\n\n### Claude Error Handling\n\nClaude-specific exception hierarchy:\n\n```scala\nimport sttp.ai.claude.ClaudeExceptions.*\n\nclient.createMessage(request).send(backend) match {\n  case Right(response) =\u003e // Success\n    handleResponse(response)\n  case Left(error) =\u003e error match {\n    case _: AuthenticationException =\u003e // Invalid API key\n      println(\"Authentication failed - check your API key\")\n    case _: RateLimitException =\u003e // Rate limited\n      println(\"Rate limited - please wait before retrying\")\n    case _: InvalidRequestException =\u003e // Malformed request\n      println(\"Invalid request - check your parameters\")\n    case _: PermissionException =\u003e // Access denied\n      println(\"Permission denied for this resource\")\n    case _: APIException =\u003e // Other API error\n      println(s\"API error: ${error.getMessage}\")\n    case _: DeserializationClaudeException =\u003e // JSON parsing error\n      println(\"Failed to parse response\")\n  }\n}\n```\n\n### Key Differences from OpenAI API\n\n| Feature | Claude API | OpenAI API |\n|---------|------------|------------|\n| **Message Content** | `ContentBlock` arrays | Simple strings |\n| **System Messages** | `system` parameter | Role-based message |\n| **Authentication** | `x-api-key` + `anthropic-version` headers | `Authorization` header |\n| **Image Input** | ContentBlock with base64 | URL or base64 in content |\n| **Tool Calling** | Native tool structure | Function calling |\n| **Streaming** | Server-Sent Events | Server-Sent Events |\n| **Model Names** | `claude-3-sonnet-20240229` | `gpt-4` |\n\n### Synchronous Claude Client\n\nFor blocking operations, use `ClaudeSyncClient`:\n\n```scala\nimport sttp.ai.claude.ClaudeSyncClient\n\nval syncClient = new ClaudeSyncClient(config)\n\n// Throws ClaudeException on error\ntry {\n  val response = syncClient.createMessage(request)\n  println(response.content.head.text.getOrElse(\"\"))\n} catch {\n  case e: ClaudeException =\u003e println(s\"Error: ${e.getMessage}\")\n}\n```\n\n## OpenAI-Compatible APIs\n\n### To use Ollama or Grok (OpenAI-compatible APIs)\n\nOllama with sync backend:\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::openai:0.4.3\n\nimport sttp.model.Uri.*\nimport sttp.ai.openai.OpenAISyncClient\nimport sttp.ai.openai.requests.completions.chat.ChatRequestResponseData.ChatResponse\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.{ChatBody, ChatCompletionModel}\nimport sttp.ai.openai.requests.completions.chat.message.*\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    // Create an instance of OpenAISyncClient providing any api key\n    // and a base url of locally running instance of ollama\n    val openAI: OpenAISyncClient = OpenAISyncClient(\"ollama\", uri\"http://localhost:11434/v1\")\n\n    // Create body of Chat Completions Request\n    val bodyMessages: Seq[Message] = Seq(\n      Message.UserMessage(\n        content = Content.TextContent(\"Hello!\"),\n      )\n    )\n\n    val chatRequestBody: ChatBody = ChatBody(\n      // assuming one has already executed `ollama pull mistral` in console\n      model = ChatCompletionModel.CustomChatCompletionModel(\"mistral\"),\n      messages = bodyMessages\n    )\n\n    // be aware that calling `createChatCompletion` may throw an OpenAIException\n    // e.g. AuthenticationException, RateLimitException and many more\n    val chatResponse: ChatResponse = openAI.createChatCompletion(chatRequestBody)\n\n    println(chatResponse)\n  /*\n    ChatResponse(\n      chatcmpl-650,\n      List(\n        Choices(\n          Message(Assistant, \"\"\"Hello there! How can I help you today?\"\"\", List(), None),\n          \"stop\",\n          0\n        )\n      ),\n      1714663831,\n      \"mistral\",\n      \"chat.completion\",\n      Usage(0, 187, 187),\n      Some(\"fp_ollama\")\n    )\n  */\n```\n\nGrok with cats-effect based backend:\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::openai:0.4.3\n//\u003e using dep com.softwaremill.sttp.client4::cats:4.0.0-M17\n\nimport cats.effect.IO\nimport cats.effect.unsafe.implicits.global\nimport sttp.client4.httpclient.cats.HttpClientCatsBackend\nimport sttp.model.Uri.*\nimport sttp.ai.openai.OpenAI\nimport sttp.ai.openai.OpenAIExceptions.OpenAIException\nimport sttp.ai.openai.requests.completions.chat.ChatRequestResponseData.ChatResponse\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.{ChatBody, ChatCompletionModel}\nimport sttp.ai.openai.requests.completions.chat.message.*\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    val apiKey = System.getenv(\"OPENAI_KEY\")\n    val openAI = new OpenAI(apiKey, uri\"https://api.groq.com/openai/v1\")\n\n    val bodyMessages: Seq[Message] = Seq(\n      Message.UserMessage(\n        content = Content.TextContent(\"Hello!\"),\n      )\n    )\n\n    val chatRequestBody: ChatBody = ChatBody(\n      model = ChatCompletionModel.CustomChatCompletionModel(\"gemma-7b-it\"),\n      messages = bodyMessages\n    )\n\n    val program = HttpClientCatsBackend.resource[IO]().use { backend =\u003e\n      val response: IO[Either[OpenAIException, ChatResponse]] =\n        openAI\n          .createChatCompletion(chatRequestBody)\n          .send(backend)\n          .map(_.body)\n      val rethrownResponse: IO[ChatResponse] = response.rethrow\n      val redeemedResponse: IO[String] = rethrownResponse.redeem(\n        error =\u003e error.getMessage,\n        chatResponse =\u003e chatResponse.toString\n      )\n      redeemedResponse.flatMap(IO.println)\n    }\n\n    program.unsafeRunSync()\n  /*\n    ChatResponse(\n      \"chatcmpl-e0f9f78c-5e74-494c-9599-da02fa495ff8\",\n      List(\n        Choices(\n          Message(Assistant, \"Hello! 👋 It's great to hear from you. What can I do for you today? 😊\", List(), None),\n          \"stop\",\n          0\n        )\n      ),\n      1714667435,\n      \"gemma-7b-it\",\n      \"chat.completion\",\n      Usage(16, 21, 37),\n      Some(\"fp_f0c35fc854\")\n    )\n  */\n```\n\n#### Available client implementations:\n\n* `OpenAISyncClient` which provides high-level methods to interact with OpenAI. All the methods send requests synchronously and are blocking, might throw `OpenAIException`\n* `OpenAI` which provides raw sttp-client4 `Request`s and parses `Response`s as `Either[OpenAIException, A]`\n\nIf you want to make use of other effects, you have to use `OpenAI` and pass the chosen backend directly to `request.send(backend)` function.\n\nTo customize a request when using the `OpenAISyncClient`, e.g. by adding a header, or changing the timeout (via request options), you can use the `.customizeRequest` method on the client.\n\nExample below uses `HttpClientCatsBackend` as a backend, make sure to [add it to the dependencies](https://sttp.softwaremill.com/en/latest/backends/catseffect.html)\nor use backend of your choice.\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::openai:0.4.3\n//\u003e using dep com.softwaremill.sttp.client4::cats:4.0.0-M17\n\nimport cats.effect.IO\nimport cats.effect.unsafe.implicits.global\nimport sttp.client4.httpclient.cats.HttpClientCatsBackend\nimport sttp.ai.openai.OpenAI\nimport sttp.ai.openai.OpenAIExceptions.OpenAIException\nimport sttp.ai.openai.requests.completions.chat.ChatRequestResponseData.ChatResponse\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.{ChatBody, ChatCompletionModel}\nimport sttp.ai.openai.requests.completions.chat.message.*\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    val apiKey = System.getenv(\"OPENAI_KEY\")\n    val openAI = new OpenAI(apiKey)\n\n    val bodyMessages: Seq[Message] = Seq(\n      Message.UserMessage(\n        content = Content.TextContent(\"Hello!\"),\n      )\n    )\n\n    val chatRequestBody: ChatBody = ChatBody(\n      model = ChatCompletionModel.GPT35Turbo,\n      messages = bodyMessages\n    )\n\n    val program = HttpClientCatsBackend.resource[IO]().use { backend =\u003e\n      val response: IO[Either[OpenAIException, ChatResponse]] =\n        openAI\n          .createChatCompletion(chatRequestBody)\n          .send(backend)\n          .map(_.body)\n      val rethrownResponse: IO[ChatResponse] = response.rethrow\n      val redeemedResponse: IO[String] = rethrownResponse.redeem(\n        error =\u003e error.getMessage,\n        chatResponse =\u003e chatResponse.toString\n      )\n      redeemedResponse.flatMap(IO.println)\n    }\n\n    program.unsafeRunSync()\n  /*\n    ChatResponse(\n      chatcmpl-79shQITCiqTHFlI9tgElqcbMTJCLZ,chat.completion,\n      1682589572,\n      gpt-3.5-turbo-0301,\n      Usage(10,10,20),\n      List(\n        Choices(\n          Message(assistant, Hello there! How can I assist you today?), stop, 0)\n        )\n      )\n    )\n  */\n```\n\n### Streaming (OpenAI)\n\n#### Create completion with streaming:\n\nTo enable streaming support for the Chat Completion API using server-sent events, you must include the appropriate\ndependency for your chosen streaming library. We provide support for the following libraries: _fs2_, _ZIO_, _Akka / Pekko Streams_ and _Ox_.\n\nFor example, to use `fs2` add the following dependency \u0026 import:\n\n```scala\n// sbt dependency\n\"com.softwaremill.sttp.ai\" %% \"fs2\" % \"0.4.3\"\n\n// import \nimport sttp.ai.openai.streaming.fs2.*\n```\n\nExample below uses `HttpClientFs2Backend` as a backend:\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::fs2:0.4.3\n\nimport cats.effect.IO\nimport cats.effect.unsafe.implicits.global\nimport fs2.Stream\nimport sttp.client4.httpclient.fs2.HttpClientFs2Backend\nimport sttp.ai.openai.OpenAI\nimport sttp.ai.openai.streaming.fs2.*\nimport sttp.ai.openai.OpenAIExceptions.OpenAIException\nimport sttp.ai.openai.requests.completions.chat.ChatChunkRequestResponseData.ChatChunkResponse\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.{ChatBody, ChatCompletionModel}\nimport sttp.ai.openai.requests.completions.chat.message.*\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    val apiKey = System.getenv(\"OPENAI_KEY\")\n    val openAI = new OpenAI(apiKey)\n\n    val bodyMessages: Seq[Message] = Seq(\n      Message.UserMessage(\n        content = Content.TextContent(\"Hello!\"),\n      )\n    )\n\n    val chatRequestBody: ChatBody = ChatBody(\n      model = ChatCompletionModel.GPT35Turbo,\n      messages = bodyMessages\n    )\n\n    val program = HttpClientFs2Backend.resource[IO]().use { backend =\u003e\n      val response: IO[Either[OpenAIException, Stream[IO, ChatChunkResponse]]] =\n        openAI\n          .createStreamedChatCompletion[IO](chatRequestBody)\n          .send(backend)\n          .map(_.body)\n\n      response\n        .flatMap {\n          case Left(exception) =\u003e IO.println(exception.getMessage)\n          case Right(stream)   =\u003e stream.evalTap(IO.println).compile.drain\n        }\n    }\n\n    program.unsafeRunSync()\n  /*\n    ...\n    ChatChunkResponse(\n      \"chatcmpl-8HEZFNDmu2AYW8jVvNKyRO4W4KcO8\",\n      \"chat.completion.chunk\",\n      1699118265,\n      \"gpt-3.5-turbo-0613\",\n      List(\n        Choices(\n          Delta(None, Some(\"Hi\"), None),\n          null,\n          0\n        )\n      )\n    )\n    ...\n    ChatChunkResponse(\n      \"chatcmpl-8HEZFNDmu2AYW8jVvNKyRO4W4KcO8\",\n      \"chat.completion.chunk\",\n      1699118265,\n      \"gpt-3.5-turbo-0613\",\n      List(\n        Choices(\n          Delta(None, Some(\" there\"), None),\n          null,\n          0\n        )\n      )\n    )\n    ...\n   */\n```\n\nTo use direct-style streaming (requires Scala 3) add the following dependency \u0026 import:\n\n```scala\n// sbt dependency\n\"com.softwaremill.sttp.ai\" %% \"ox\" % \"0.4.3\"\n\n// import \nimport sttp.ai.openai.streaming.ox.*\n```\n\nExample code:\n\n```scala\n//\u003e using dep com.softwaremill.sttp.ai::ox:0.4.3\n\nimport ox.*\nimport ox.either.orThrow\nimport sttp.client4.DefaultSyncBackend\nimport sttp.ai.openai.OpenAI\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.{ChatBody, ChatCompletionModel}\nimport sttp.ai.openai.requests.completions.chat.message.*\nimport sttp.ai.openai.streaming.ox.*\n\nobject Main extends OxApp:\n  override def run(args: Vector[String])(using Ox): ExitCode =\n    val apiKey = System.getenv(\"OPENAI_KEY\")\n    val openAI = new OpenAI(apiKey)\n    \n    val bodyMessages: Seq[Message] = Seq(\n      Message.UserMessage(\n        content = Content.TextContent(\"Hello!\")\n      )\n    )\n    \n    val chatRequestBody: ChatBody = ChatBody(\n      model = ChatCompletionModel.GPT35Turbo,\n      messages = bodyMessages\n    )\n    \n    val backend = useCloseableInScope(DefaultSyncBackend())\n    openAI\n      .createStreamedChatCompletion(chatRequestBody)\n      .send(backend)\n      .body // this gives us an Either[OpenAIException, Flow[ChatChunkResponse]]\n      .orThrow // we choose to throw any exceptions and fail the whole app\n      .runForeach(el =\u003e println(el.orThrow))\n    \n    ExitCode.Success\n```\n\nSee also the [ChatProxy](https://github.com/softwaremill/sttp-openai/blob/master/examples/src/main/scala/examples/ChatProxy.scala) example application.\n\n#### Structured Outputs/JSON Schema support\n\nTo take advantage of [OpenAI's Structured Outputs](https://platform.openai.com/docs/guides/structured-outputs/introduction)\nand support for JSON Schema, you can use `ResponseFormat.JsonSchema` when creating a completion.\n\nThe example below produces a JSON object:\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::openai:0.4.3\n\nimport scala.collection.immutable.ListMap\nimport sttp.apispec.{Schema, SchemaType}\nimport sttp.ai.openai.OpenAISyncClient\nimport sttp.ai.openai.requests.completions.chat.ChatRequestResponseData.ChatResponse\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.{ChatBody, ChatCompletionModel, ResponseFormat}\nimport sttp.ai.openai.requests.completions.chat.message.*\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    val apiKey = System.getenv(\"OPENAI_KEY\")\n    val openAI = OpenAISyncClient(apiKey)\n\n    val jsonSchema: Schema =\n      Schema(SchemaType.Object).copy(properties =\n        ListMap(\n          \"steps\" -\u003e Schema(SchemaType.Array).copy(items =\n            Some(Schema(SchemaType.Object).copy(properties =\n              ListMap(\n                \"explanation\" -\u003e Schema(SchemaType.String),\n                \"output\" -\u003e Schema(SchemaType.String)\n              )\n            ))\n          ),\n          \"finalAnswer\" -\u003e Schema(SchemaType.String)\n        ),\n      )\n\n    val responseFormat: ResponseFormat.JsonSchema =\n      ResponseFormat.JsonSchema(\n        name = \"mathReasoning\",\n        strict = Some(true),\n        schema = Some(jsonSchema),\n        description = None\n      )\n\n    val bodyMessages: Seq[Message] = Seq(\n      Message.SystemMessage(content = \"You are a helpful math tutor. Guide the user through the solution step by step.\"),\n      Message.UserMessage(content = Content.TextContent(\"How can I solve 8x + 7 = -23\"))\n    )\n\n    // Create body of Chat Completions Request, using our JSON Schema as the `responseFormat`\n    val chatRequestBody: ChatBody = ChatBody(\n      model = ChatCompletionModel.GPT4oMini,\n      messages = bodyMessages,\n      responseFormat = Some(responseFormat)\n    )\n\n    val chatResponse: ChatResponse = openAI.createChatCompletion(chatRequestBody)\n\n    println(chatResponse.choices)\n  /*\n    List(\n      Choices(\n        Message(\n          Assistant,\n          {\n            \"steps\": [\n              {\"explanation\": \"Start with the original equation: 8x + 7 = -23\", \"output\": \"8x + 7 = -23\"},\n              {\"explanation\": \"Subtract 7 from both sides to isolate the term with x.\", \"output\": \"8x + 7 - 7 = -23 - 7\"},\n              {\"explanation\": \"This simplifies to: 8x = -30\", \"output\": \"8x = -30\"},\n              {\"explanation\": \"Now, divide both sides by 8 to solve for x.\", \"output\": \"x = -30 / 8\"},\n              {\"explanation\": \"Simplify -30 / 8 to its simplest form. Both the numerator and denominator can be divided by 2.\", \"output\": \"x = -15 / 4\"}\n            ],\n            \"finalAnswer\": \"x = -15/4\"\n          },\n          List(),\n          None\n        ),\n        stop,\n        0\n      )\n    )\n  */\n```\n\n##### Deriving a JSON Schema with tapir\n\nTo derive the same math reasoning schema used above, you can use\n[Tapir's support for generating a JSON schema from a Tapir schema](https://tapir.softwaremill.com/en/latest/docs/json-schema.html):\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.tapir::tapir-apispec-docs:1.11.7\n\nimport sttp.apispec.{Schema =\u003e ASchema}\nimport sttp.tapir.Schema\nimport sttp.tapir.docs.apispec.schema.TapirSchemaToJsonSchema\nimport sttp.tapir.generic.auto.*\n\ncase class Step(\n  explanation: String,\n  output: String\n)\n\ncase class MathReasoning(\n  steps: List[Step],\n  finalAnswer: String\n)\n\nval tSchema = implicitly[Schema[MathReasoning]]\n\nval jsonSchema: ASchema = TapirSchemaToJsonSchema(\n  tSchema,\n  markOptionsAsNullable = true\n)\n```\n\n#### Generating JSON Schema from case class\n\nWe can also generate JSON Schema directly from case class, without defining the schema manually.\n\nIn the example below I define such use case. User tries to book a flight, using function tool. The flow looks as follows:\n- User sends a message with the request to book a flight and provides function tool, which means that there is a function on a client side which 'knows' how to book a flight. Within this call it is necessary to provide Json Schema to define function arguments.\n- Assistant sends a message with arguments created based on Json Schema provided in the first step.\n- User calls custom function with arguments sent by Assistant before.\n- User sends result from the function call to Assistant.\n- Assistant sends a final result to User.\n\nThe key point here is using `FunctionTool.withSchema[T]` method. With this method, Json Schema can be automatically generated using TapirSchemaToJsonSchema functionality. All we need to do is to define case class with [Tapir Schema](https://tapir.softwaremill.com/en/latest/endpoint/schemas.html) defined for it.\n\nAnother helpful feature is adding possibility to create ToolMessage object passing object instead of String, which will be automatically serialized to Json. All you have to do is just define SnakePickle.Writer for specific class.\n\nWith all this in mind please remember that it is still required to deserialized arguments, which are sent back by Assistant to call our function.\n\n```scala mdoc:compile-only\n//\u003e using dep com.softwaremill.sttp.ai::openai:0.4.3\n\nimport sttp.ai.openai.OpenAISyncClient\nimport sttp.ai.core.json.SnakePickle\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.ChatBody\nimport sttp.ai.openai.requests.completions.chat.ChatRequestBody.ChatCompletionModel.GPT4oMini\nimport sttp.ai.openai.requests.completions.chat.ToolCall.FunctionToolCall\nimport sttp.ai.openai.requests.completions.chat.message.Content.TextContent\nimport sttp.ai.openai.requests.completions.chat.message.Message.{AssistantMessage, ToolMessage, UserMessage}\nimport sttp.ai.openai.requests.completions.chat.message.Tool.FunctionTool\nimport sttp.tapir.generic.auto.*\n\ncase class Passenger(name: String, age: Int)\n\nobject Passenger:\n  given SnakePickle.Reader[Passenger] = SnakePickle.macroR[Passenger]\n\ncase class FlightDetails(passenger: Passenger, departureCity: String, destinationCity: String)\n\nobject FlightDetails:\n  given SnakePickle.Reader[FlightDetails] = SnakePickle.macroR[FlightDetails]\n\ncase class BookedFlight(confirmationNumber: String, status: String)\n\nobject BookedFlight:\n  given SnakePickle.Writer[BookedFlight] = SnakePickle.macroW[BookedFlight]\n\nobject Main:\n  def main(args: Array[String]): Unit =\n    val apiKey = System.getenv(\"OPENAI_KEY\")\n    val openAI = OpenAISyncClient(apiKey)\n\n    val initialRequestMessage = Seq(UserMessage(content = TextContent(\"I want to book a flight from London to Tokyo for Jane Doe, age 34\")))\n\n    // Request created using FunctionTool.withSchema, all we need to do here is just define the type. The schema is automatically generated using a macro, available via the `sttp.tapir.generic.auto.*` import.\n    val givenRequest = ChatBody(\n      model = GPT4oMini,\n      messages = initialRequestMessage,\n      tools = Some(Seq(\n        FunctionTool.withSchema[FlightDetails](\n          name = \"book_flight\",\n          description = Some(\"Books a flight for a passenger with full details\")))\n      )\n    )\n\n    val initialRequestResult = openAI.createChatCompletion(givenRequest)\n\n    println(initialRequestResult.choices)\n    /*\n      List(\n        Choices(\n          Message(\n            null,\n            None,\n            List(\n              FunctionToolCall(\n                Some(call_XZNvfldLQTa1f7aMInswpTMS),\n                FunctionCall(\n                  {\n                    \"passenger\": {\n                      \"name\": \"Jane Doe\",\n                      \"age\": 34\n                    },\n                    \"departureCity\": \"London\",\n                    \"destinationCity\": \"Tokyo\"\n                  },\n                  Some(book_flight)\n                )\n              )\n            ),\n            Assistant,\n            None,\n            None\n          ),\n          tool_calls,\n          0,\n          None\n        )\n      )\n      */\n\n    // Helper function that mimics external function definition\n    def bookFlight(flightDetails: FlightDetails): BookedFlight =\n      println(flightDetails)\n      BookedFlight(confirmationNumber = \"123456\", status = \"confirmed\")\n\n    // Tool calls list (in this example it is just single tool call, but there may be multiple), which is necessary to build message list for second request.\n    val toolCalls = initialRequestResult.choices.head.message.toolCalls\n\n    val functionToolCall = toolCalls.head match\n      case functionToolCall: FunctionToolCall =\u003e functionToolCall\n\n    // Function arguments are manually deserialized, 'bookFlight' function mimic external function definition.\n    val bookedFlight = bookFlight(SnakePickle.read[FlightDetails](functionToolCall.function.arguments))\n\n    val secondRequest = givenRequest.copy(\n      messages = initialRequestMessage\n        :+ AssistantMessage(content = \"\", toolCalls = toolCalls)\n        // ToolMessage created using object instead of String with Json representation of object.\n        :+ ToolMessage(toolCallId = functionToolCall.id.get, content = bookedFlight)\n    )\n\n    val finalResult = openAI.createChatCompletion(secondRequest)\n\n    println(finalResult.choices)\n    /*\n      List(\n        Choices(\n          Message(\n            \"The flight from London to Tokyo for Jane Doe, age 34, has been successfully booked. The confirmation number is **123456** and the status is **confirmed**.\",\n            None,\n            List(),\n            Assistant,\n            None,\n            None\n          ),\n          stop,\n          0,\n          None\n        )\n      )\n      */\n```\n\n## Contributing\n\nIf you have a question, or hit a problem, feel free to post on our community https://softwaremill.community/c/open-source/\n\nOr, if you encounter a bug, something is unclear in the code or documentation, don't hesitate and open an issue on GitHub.\n\nFor running integration tests against the real OpenAI API, see [Integration Testing Guide](INTEGRATION_TESTING.md).\n\n## Commercial Support\n\nWe offer commercial support for sttp and related technologies, as well as development services. [Contact us](https://softwaremill.com) to learn more about our offer!\n\n## Copyright\n\nCopyright (C) 2023-2025 SoftwareMill [https://softwaremill.com](https://softwaremill.com).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoftwaremill%2Fsttp-ai","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsoftwaremill%2Fsttp-ai","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsoftwaremill%2Fsttp-ai/lists"}