https://github.com/lzell/aiproxyswift
Swift client for AI providers. Can make requests straight to the provider or proxied through our API key protection backend
https://github.com/lzell/aiproxyswift
client openai replicate swift
Last synced: 4 months ago
JSON representation
Swift client for AI providers. Can make requests straight to the provider or proxied through our API key protection backend
- Host: GitHub
- URL: https://github.com/lzell/aiproxyswift
- Owner: lzell
- License: mit
- Created: 2024-06-11T22:03:40.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2025-10-18T19:39:31.000Z (7 months ago)
- Last Synced: 2025-10-21T07:54:20.182Z (7 months ago)
- Topics: client, openai, replicate, swift
- Language: Swift
- Homepage: https://www.aiproxy.com
- Size: 4.14 MB
- Stars: 361
- Watchers: 6
- Forks: 71
- Open Issues: 14
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-ChatGPT-repositories - AIProxySwift - Swift client for AI providers. Can make requests straight to the provider or proxied through our API key protection backend (CLIs)
README
# About
Use this library to adopt AI APIs in your app. Swift clients for the following providers are
included:
- OpenAI
- Gemini
- Anthropic
- Stability AI
- DeepL
- Together AI
- Replicate
- ElevenLabs
- Fal
- Groq
- Perplexity
- Mistral
- EachAI
- OpenRouter
- DeepSeek
- Fireworks AI
- Brave
Your initialization code determines whether requests go straight to the provider or are
protected through the [AIProxy](https://www.aiproxy.com) backend.
We only recommend making requests straight to the provider during prototyping and for BYOK
use-cases.
Requests that are protected through AIProxy have five levels of security applied to keep your API
key secure and your AI bill predictable:
- Certificate pinning
- DeviceCheck verification
- Split key encryption
- Per user rate limits
- Per IP rate limits
# Installation
## Installation using Xcode
1. From within your Xcode project, select `File > Add Package Dependencies`
2. Punch `github.com/lzell/aiproxyswift` into the package URL bar, and select the 'main' branch
as the dependency rule. Alternatively, you can choose specific releases if you'd like to have finer control of when your dependency gets updated.
## Installation using cocoapods
Add to your podfile:
pod "AIProxy"
Then, from shell:
pod install
## How to configure the package for use with AIProxy
We recommend using the AIProxy option `useStableID` to rate limit usage across an app store user's account on multiple devices.
To enable this, please first add support for iCloud's key-value storage:
1. Tap on your project in Xcode's project tree
2. Select your target in the secondary sidebar
3. Tap on Signing & Capabilities > Add Capability > iCloud
4. Check the 'Key-Value storage' checkbox
During your app's launch, call `AIProxy.configure`. Using this method, you can specify:
- the log level that you'd like to see in your Xcode console from the AIProxy lib
- whether to print request/response bodies to Xcode's console, which is useful for debugging or contributing to the library
- whether to resolve DNS queries [using Cloudflare's DoT](https://developers.cloudflare.com/1.1.1.1/encryption/dns-over-tls/) (recommended)
- whether to use stable identifiers as client IDs (recommended)
In a SwiftUI app, call `AIProxy.configure` in your app's composition root:
```swift
import AIProxy
@main
struct MyApp: App {
init() {
AIProxy.configure(
logLevel: .debug,
printRequestBodies: false,
printResponseBodies: false,
resolveDNSOverTLS: true,
useStableID: true
)
}
// ...
}
```
In a UIKit app, call `AIProxy.configure` in applicationDidFinishLaunching:
```swift
import AIProxy
@UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?
func application(_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
AIProxy.configure(
logLevel: .debug,
printRequestBodies: false,
printResponseBodies: false,
resolveDNSOverTLS: true,
useStableID: true
)
// ...
return true
}
// ...
}
```
## How to configure the AIProxy backend for use with your project
See the [AIProxy integration video](https://www.aiproxy.com/docs/integration-guide.html).
Note that this is not required if you are shipping an app where the customers provide their own
API keys (known as BYOK for "bring your own key").
If you are shipping an app using a personal or company API key, we highly recommend setting up
AIProxy as an alternative to building, monitoring, and maintaining your own backend.
# How to update the package
- If you set the dependency rule to `main` during installation, then you can ensure the package is
up to date by right clicking on the package and selecting 'Update Package'
- If you selected a version-based rule, inspect the rule in the 'Package Dependencies' section
of your project settings:
Once the rule is set to include the release version that you'd like to bring in, Xcode should
update the package automatically. If it does not, right click on the package in the project
tree and select 'Update Package'.
# How to contribute to the package
Your additions to AIProxySwift are welcome! I like to develop the library while working in an
app that depends on it:
1. Fork the repo
2. Clone your fork
3. Open your app in Xcode
4. Remove AIProxySwift from your app (since this is likely referencing a remote lib)
5. Go to `File > Add Package Dependencies`, and in the bottom left of that popup there is a button "Add local"
6. Tap "Add local" and then select the folder where you cloned AIProxySwift on your disk.
If you do that, then you can modify the source to AIProxySwift right from within your Xcode project for your app.
Once you're happy with your changes, open a PR here.
# Example usage
* [OpenAI](#openai)
* [Gemini](#gemini)
* [Anthropic](#anthropic)
* [Stability AI](#stability-ai)
* [DeepL](#deepl)
* [Together AI](#together-ai)
* [Replicate](#replicate)
* [ElevenLabs](#elevenlabs)
* [Fal](#fal)
* [Groq](#groq)
* [Perplexity](#perplexity)
* [Mistral](#mistral)
* [EachAI](#eachai)
* [OpenRouter](#openrouter)
* [DeepSeek](#deepseek)
* [Fireworks AI](#fireworks-ai)
* [Brave](#brave)
* [Advanced Settings](#advanced-settings)
## OpenAI
### Get a non-streaming chat completion from OpenAI:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAIChatCompletionRequestBody(
model: "gpt-5.2",
messages: [
.system(content: .text("You are a friendly assistant")),
.user(content: .text("hello world"))
],
reasoningEffort: .noReasoning
)
do {
let response = try await openAIService.chatCompletionRequest(
body: requestBody,
secondsToWait: 120
)
print(response.choices.first?.message.content ?? "")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create OpenAI chat completion: \(error)")
}
```
### How to make a buffered chat completion to OpenAI with extended timeout
This is useful for `o1` and `o3` models.
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAIChatCompletionRequestBody(
model: "gpt-5.2",
messages: [
.developer(content: .text("You are a coding assistant")),
.user(content: .text("Build a ruby service that writes latency stats to redis on each request"))
],
reasoningEffort: .high
)
do {
let response = try await openAIService.chatCompletionRequest(
body: requestBody,
secondsToWait: 300
)
print(response.choices.first?.message.content ?? "")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch let err as URLError where err.code == URLError.timedOut {
print("Request to OpenAI for a reasoning request timed out")
} catch let err as URLError where [.notConnectedToInternet, .networkConnectionLost].contains(err.code) {
print("Could not complete OpenAI reasoning request. Please check your internet connection")
} catch {
print("Could not complete OpenAI reasoning request: \(error)")
}
```
### Get a streaming chat completion from OpenAI:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAIChatCompletionRequestBody(
model: "gpt-5.2",
messages: [.user(content: .text("hello world"))]
reasoningEffort: .noReasoning
)
do {
let stream = try await openAIService.streamingChatCompletionRequest(
body: requestBody,
secondsToWait: 60
)
for try await chunk in stream {
print(chunk.choices.first?.delta.content ?? "")
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create OpenAI streaming chat completion: \(error)")
}
```
### How to include history in chat completion requests to OpenAI
Use this approach to have a conversation with ChatGPT. All previous chat messages, whether
issued by the user or the assistant (chatGPT), are fed back into the model on each request.
As an alternative, you can use the new ChatGPT Responses API to hold the entire history by passing in the previousResponseId
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
// We'll start the conversation by asking about the color of a blackberry.
// There is no prior history, so we only send up a single user message.
//
// You can optionally include a .system message to give the model
// instructions on how it should behave.
let userMessage1: OpenAIChatCompletionRequestBody.Message = .user(
content: .text("What color is a blackberry?")
)
// Create the first chat completion.
var completion1: OpenAIChatCompletionResponseBody? = nil
do {
completion1 = try await openAIService.chatCompletionRequest(body: .init(
model: "gpt-5-mini",
messages: [
userMessage1
]
))
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not get first chat completion: \(error)")
}
// Get the contents of the model's first response:
guard let assistantContent1 = completion1?.choices.first?.message.content else {
print("Completion1: ChatGPT did not respond with any assistant content")
return
}
print("Completion1: \(assistantContent1)")
// Continue the conversation by asking about a strawberry.
// If the history were absent from the request, ChatGPT would respond with general facts.
// By including the history, the model continues the conversation, understanding that we
// are specifically interested in the strawberry's color.
let userMessage2: OpenAIChatCompletionRequestBody.Message = .user(
content: .text("And a strawberry?")
)
// Create the second chat completion, note the `messages` array.
var completion2: OpenAIChatCompletionResponseBody? = nil
do {
completion2 = try await openAIService.chatCompletionRequest(body: .init(
model: "gpt-5-mini",
messages: [
userMessage1,
.assistant(content: .text(assistantContent1)),
userMessage2
]
))
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not get second chat completion: \(error)")
}
// Get the contents of the model's second response:
guard let assistantContent2 = completion2?.choices.first?.message.content else {
print("Completion2: ChatGPT did not respond with any assistant content")
return
}
print("Completion2: \(assistantContent2)")
```
### Send a multi-modal chat completion request to OpenAI:
On macOS, use `NSImage(named:)` in place of `UIImage(named:)`
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let image = UIImage(named: "myImage") else {
print("Could not find an image named 'myImage' in your app assets")
return
}
guard let imageURL = AIProxy.encodeImageAsURL(image: image, compressionQuality: 0.6) else {
print("Could not encode image as data URL")
return
}
do {
let response = try await openAIService.chatCompletionRequest(body: .init(
model: "gpt-5.2",
messages: [
.system(
content: .text("Tell me what you see")
),
.user(
content: .parts(
[
.text("What do you see?"),
.imageURL(imageURL, detail: .auto)
]
)
)
]
))
print(response.choices.first?.message.content ?? "")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create OpenAI multi-modal chat completion: \(error)")
}
```
### How to generate an image with DALLE
This snippet will print out the URL of an image generated with `dall-e-3`:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
do {
let response = try await openAIService.createImageRequest(
body: .init(
prompt: "a skier",
model: .dallE3
),
secondsToWait: 300
)
print(response.data.first?.url ?? "")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not create an image with DALLE 3: \(error)")
}
```
### How to generate an image with OpenAI's gpt-image-1
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
do {
let response = try await openAIService.createImageRequest(
body: .init(
prompt: "a skier",
model: .gptImage1
),
secondsToWait: 300
)
guard let base64Data = response.data.first?.b64JSON,
let imageData = Data(base64Encoded: base64Data),
let image = UIImage(data: imageData) else {
print("Could not create a UIImage out of the base64 returned by OpenAI")
return
}
// Do something with 'image'
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not create OpenAI image generation: \(error)")
}
```
### How to make a high fidelity image edit with OpenAI's gpt-image-1
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let openAIService = getOpenAIService(config)
guard let image = UIImage(named: "my-image") else {
print("Could not find an image named 'my-image' in your app assets")
return
}
guard let jpegData = AIProxy.encodeImageAsJpeg(image: image, compressionQuality: 0.5) else {
print("Could not encode image as jpeg")
return
}
let requestBody = OpenAICreateImageEditRequestBody(
images: [.jpeg(jpegData)],
prompt: "Change the coffee cup to red",
inputFidelity: .high,
model: .gptImage1
)
do {
let response = try await openAIService.createImageEditRequest(
body: requestBody,
secondsToWait: 300
)
guard let base64Data = response.data.first?.b64JSON,
let imageData = Data(base64Encoded: base64Data),
let editedImage = UIImage(data: imageData) else {
print("Could not create a UIImage out of the base64 returned by OpenAI")
return
}
// Do something with 'editedImage' here
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not create OpenAI edit image generation: \(error)")
}
```
### How to upload multiple images for use in an image edit with OpenAI's gpt-image-1
- This snippet uploads two images to `gpt-image-1`, transfering the material of one to the other.
- One image is uploaded as a png and the other as a jpeg.
- The output quality is chosen to be `.low` for speed of generation.
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let image1 = UIImage(named: "my-first-image") else {
print("Could not find an image named 'my-first-image' in your app assets")
return
}
guard let image2 = UIImage(named: "my-second-image") else {
print("Could not find an image named 'my-second-image' in your app assets")
return
}
guard let jpegData = AIProxy.encodeImageAsJpeg(image: image1, compressionQuality: 0.4) else {
print("Could not convert image to jpeg")
return
}
guard let pngData = image2.pngData() else {
print("Could not convert image to png")
return
}
do {
let response = try await openAIService.createImageEditRequest(
body: .init(
images: [
.jpeg(jpegData),
.png(pngData)
],
prompt: "Transfer the material of the second image to the first",
model: .gptImage1,
quality: .low
),
secondsToWait: 300
)
guard let base64Data = response.data.first?.b64JSON,
let imageData = Data(base64Encoded: base64Data),
let image = UIImage(data: imageData) else {
print("Could not create a UIImage out of the base64 returned by OpenAI")
return
}
// Do something with 'image'
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not create OpenAI edit image generation: \(error)")
}
```
### How to make a web search chat completion call with OpenAI
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAIChatCompletionRequestBody(
model: "gpt-4o-mini-search-preview",
messages: [.user(
content: .text("what is Apple's stock price today?")
)],
webSearchOptions: .init(
searchContextSize: .low,
userLocation: nil
)
)
do {
let response = try await openAIService.chatCompletionRequest(
body: requestBody,
secondsToWait: 60
)
print(response.choices.first?.message.content ?? "")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not make a web search completion call with OpenAI: \(error)")
}
```
### How to ensure OpenAI returns JSON as the chat message content
If you need to enforce a strict JSON contract, please use Structured Outputs (the next example)
instead of this approach. This approach is referred to as 'JSON mode' in the OpenAI docs, and
is the predecessor to Structured Outputs.
JSON mode is enabled with `responseFormat: .jsonObject`, while Structured Outputs is enabled
with `responseFormat: .jsonSchema`.
If you use JSON mode, set `responseFormat` *and* specify in the prompt that OpenAI should
return JSON only:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
do {
let requestBody = OpenAIChatCompletionRequestBody(
model: "gpt-4o",
messages: [
.system(content: .text("Return valid JSON only")),
.user(content: .text("Return alice and bob in a list of names"))
],
responseFormat: .jsonObject
)
let response = try await openAIService.chatCompletionRequest(body: requestBody)
print(response.choices.first?.message.content ?? "")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create OpenAI chat completion in JSON mode: \(error)")
}
```
### How to use OpenAI structured outputs (JSON schemas) in a chat response
This example prompts chatGPT to construct a color palette and conform to a strict JSON schema
in its response:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
do {
let schema: [String: AIProxyJSONValue] = [
"type": "object",
"properties": [
"colors": [
"type": "array",
"items": [
"type": "object",
"properties": [
"name": [
"type": "string",
"description": "A descriptive name to give the color"
],
"hex_code": [
"type": "string",
"description": "The hex code of the color"
]
],
"required": ["name", "hex_code"],
"additionalProperties": false
]
]
],
"required": ["colors"],
"additionalProperties": false
]
let requestBody = OpenAIChatCompletionRequestBody(
model: "gpt-5.2",
messages: [
.system(content: .text("Return valid JSON only, and follow the specified JSON structure")),
.user(content: .text("Return a peaches and cream color palette"))
],
responseFormat: .jsonSchema(
name: "palette_creator",
description: "A list of colors that make up a color pallete",
schema: schema,
strict: true
)
)
let response = try await openAIService.chatCompletionRequest(body: requestBody)
print(response.choices.first?.message.content ?? "")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create OpenAI chat completion with structured outputs: \(error)")
}
```
### How to use OpenAI structured outputs with a function call
This implements the example in OpenAI's new [function calling guide](https://platform.openai.com/docs/guides/function-calling).
For more examples, see the [original structured outputs announcement](https://openai.com/index/introducing-structured-outputs-in-the-api).
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
func getWeather(location: String?) -> String {
// Fill this with your native function logic.
// Using a stub for this example.
return "Sunny and 65 degrees"
}
// We'll start the conversation by asking about the weather.
// There is no prior history, so we only send up a single user message.
//
// You can optionally include a .system message to give the model
// instructions on how it should behave.
let userMessage: OpenAIChatCompletionRequestBody.Message = .user(
content: .text("What is the weather in SF?")
)
var completion1: OpenAIChatCompletionResponseBody? = nil
do {
completion1 = try await openAIService.chatCompletionRequest(body: .init(
model: "gpt-5.2",
messages: [
userMessage
],
tools: [
.function(
name: "get_weather",
description: "Get current temperature for a given location.",
parameters: [
"type": "object",
"properties": [
"location": [
"type": "string",
"description": "City and country e.g. Bogotá, Colombia"
]
],
"required": ["location"],
"additionalProperties": false
],
strict: true
)
]
))
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not get first chat completion: \(error)")
}
// Get the contents of the model's first response:
guard let toolCall = completion1?.choices.first?.message.toolCalls?.first else {
print("Completion1: ChatGPT did not respond with a tool call")
return
}
// Invoke the function call natively.
guard toolCall.function.name == "get_weather" else {
print("We only know how to get the weather")
return
}
let weather = getWeather(location: toolCall.function.arguments?["location"] as? String)
// Pass the results of the function call back to OpenAI.
// We create a second chat completion, note the `messages` array in
// the completion request. It passes back up:
// 1. the original user message
// 2. the response from the assistant, which told us to call the get_weather function
// 3. the result of our `getWeather` invocation
let toolMessage: OpenAIChatCompletionRequestBody.Message = .tool(
content: .text(weather),
toolCallID: toolCall.id
)
var completion2: OpenAIChatCompletionResponseBody? = nil
do {
completion2 = try await openAIService.chatCompletionRequest(
body: .init(
model: "gpt-5.2",
messages: [
userMessage,
.assistant(
toolCalls: [
.init(
id: toolCall.id,
function: .init(
name: toolCall.function.name,
arguments: toolCall.function.argumentsRaw
)
)
]
),
toolMessage
]
)
)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not get second chat completion: \(error)")
}
// Get the contents of the model's second response:
guard let assistantContent2 = completion2?.choices.first?.message.content else {
print("Completion2: ChatGPT did not respond with any assistant content")
return
}
print(assistantContent2)
// Prints: "The weather in San Francisco is sunny with a temperature of 65 degrees Fahrenheit."
```
### How to stream structured outputs tool calls with OpenAI
This example it taken from OpenAI's [function calling guide](https://platform.openai.com/docs/guides/function-calling).
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAIChatCompletionRequestBody(
model: "gpt-5.2",
messages: [
.user(content: .text("What is the weather like in Paris today?")),
],
tools: [
.function(
name: "get_weather",
description: "Get current temperature for a given location.",
parameters: [
"type": "object",
"properties": [
"location": [
"type": "string",
"description": "City and country e.g. Bogotá, Colombia"
],
],
"required": ["location"],
"additionalProperties": false
],
strict: true
),
]
)
do {
let stream = try await openAIService.streamingChatCompletionRequest(body: requestBody)
for try await chunk in stream {
guard let delta = chunk.choices.first?.delta else {
continue
}
// If the model decided to call a function, this branch will be entered:
if let toolCall = delta.toolCalls?.first {
if let functionName = toolCall.function?.name {
print("ChatGPT wants to call function \(functionName) with arguments...")
}
print(toolCall.function?.arguments ?? "")
}
// If the model decided to chat, this branch will be entered:
if let content = delta.content {
print(content)
}
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not make a streaming tool call to OpenAI: \(error)")
}
```
### How to transcribe audio with OpenAI
1. Record an audio file in quicktime and save it as "helloworld.m4a"
2. Add the audio file to your Xcode project. Make sure it's included in your target: select your audio file in the project tree, type `cmd-opt-0` to open the inspect panel, and view `Target Membership`
3. Run this snippet:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let url = Bundle.main.url(forResource: "helloworld", withExtension: "m4a"),
let fileContents = try? Data(contentsOf: url)
else {
print("Could not read file contents of helloworld.m4a")
return
}
do {
let requestBody = OpenAICreateTranscriptionRequestBody(
file: fileContents,
model: "gpt-4o-mini-transcribe",
responseFormat: "text"
)
let response = try await openAIService.createTranscriptionRequest(
body: requestBody,
secondsToWait: 120
)
print(response.text)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not transcribe audio with OpenAI: \(error)")
}
```
### How to get Whisper word-level timestamps in an audio transcription with OpenAI
1. Record an audio file in quicktime and save it as "helloworld.m4a"
2. Add the audio file to your Xcode project. Make sure it's included in your target: select your audio file in the project tree, type `cmd-opt-0` to open the inspect panel, and view `Target Membership`
3. Run this snippet:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let url = Bundle.main.url(forResource: "helloworld", withExtension: "m4a"),
let fileContents = try? Data(contentsOf: url)
else {
print("Could not read file contents of helloworld.m4a")
return
}
do {
let requestBody = OpenAICreateTranscriptionRequestBody(
file: fileContents,
model: "whisper-1",
responseFormat: "verbose_json",
timestampGranularities: [.word, .segment]
)
let response = try await openAIService.createTranscriptionRequest(body: requestBody)
if let words = response.words {
for word in words {
print("\(word.word) from \(word.start) to \(word.end)")
}
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get word-level timestamps from OpenAI: \(error)")
}
```
### How to use OpenAI text-to-speech
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
do {
let requestBody = OpenAITextToSpeechRequestBody(
input: "Hello world",
voice: .nova
)
let mpegData = try await openAIService.createTextToSpeechRequest(body: requestBody)
// Do not use a local `let` or `var` for AVAudioPlayer.
// You need the lifecycle of the player to live beyond the scope of this function.
// Instead, use file scope or set the player as a member of a reference type with long life.
// For example, at the top of this file you may define:
//
// fileprivate var audioPlayer: AVAudioPlayer? = nil
//
// And then use the code below to play the TTS result:
audioPlayer = try AVAudioPlayer(data: mpegData)
audioPlayer?.prepareToPlay()
audioPlayer?.play()
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create OpenAI TTS audio: \(error)")
}
```
### How to classify text as potentially harmful with OpenAI moderations
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAIModerationRequestBody(
input: [
.text("is this bad"),
],
model: "omni-moderation-latest"
)
do {
let response = try await openAIService.moderationRequest(body: requestBody)
print("Is this content flagged: \(response.results.first?.flagged ?? false)")
//
// For a more detailed assessment of the input content, inspect:
//
// response.results.first?.categories
//
// and
//
// response.results.first?.categoryScores
//
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not perform moderation request to OpenAI")
}
```
### How to classify images as potentially harmful with OpenAI moderations
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let image = NSImage(named: "myImage") else {
print("Could not find an image named 'myImage' in your app assets")
return
}
guard let imageURL = AIProxy.encodeImageAsURL(image: image, compressionQuality: 0.4) else {
print("Could not encode image as data URL")
return
}
let requestBody = OpenAIModerationRequestBody(
input: [
.image(imageURL)
],
model: "omni-moderation-latest"
)
do {
let response = try await openAIService.moderationRequest(body: requestBody)
print("Is this content flagged: \(response.results.first?.flagged ?? false)")
//
// For a more detailed assessment of the input content, inspect:
//
// response.results.first?.categories
//
// and
//
// response.results.first?.categoryScores
//
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not perform image moderation request to OpenAI")
}
```
### How to get embeddings using OpenAI
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAIEmbeddingRequestBody(
input: .text("hello world"),
model: "text-embedding-3-small"
)
// Or, for multiple embeddings from strings:
/*
let requestBody = OpenAIEmbeddingRequestBody(
input: .textArray([
"hello world",
"hola mundo"
]),
model: "text-embedding-3-small"
)
*/
// Or, for multiple embeddings from tokens:
/*
let requestBody = OpenAIEmbeddingRequestBody(
input: .intArray([0,1,2]),
model: "text-embedding-3-small"
)
*/
do {
let response = try await openAIService.embeddingRequest(body: requestBody)
print(
"""
The response contains \(response.embeddings.count) embeddings.
The first vector starts with \(response.embeddings.first?.vector.prefix(10) ?? [])
"""
)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not perform embedding request to OpenAI: \(error)")
}
```
### How to use realtime audio with OpenAI
Use this example to have a conversation with OpenAI's realtime models.
If you're building for iOS, see the [OpenAI realtime starter](https://github.com/lzell/OpenAIRealtimeSample) project for a jumping off point.
We recommend getting a basic chat completion with OpenAI working before attempting realtime.
Realtime is a more involved integration (as you can see from the code snippet below), and
getting a basic integration working first narrows down the source of any problem.
Take these steps to build and run an OpenAI realtime example:
1. Generate a new SwiftUI Xcode project
2. Add the `NSMicrophoneUsageDescription` key to your info.plist file
3. If macOS, tap your project > your target > Signing & Capabilities and add the following:
- App Sandbox > Outgoing Connections (client)
- App Sandbox > Audio Input
- Hardened Runtime > AudioInput
4. Replace the contents of `ContentView.swift` with the snippet below
5. Replace the placeholders in the snippet
- If connecting directly to OpenAI, replace `your-openai-key`
- If protecting your connection through AIProxy, replace `aiproxy-partial-key` and `aiproxy-service-url`
6. Set the `logLevel` argument of the `openAIService.realtimeSession` call to your desired level. If you leave
it set at `.debug`, then you'll see logs for all audio samples that we send and receive from OpenAI.
```swift
import SwiftUI
import AIProxy
struct ContentView: View {
let realtimeManager = RealtimeManager()
@State private var isRealtimeActive: Bool = false {
willSet {
if newValue {
startRealtime()
} else {
stopRealtime()
}
}
}
private func startRealtime() {
Task {
do {
try await realtimeManager.startConversation()
} catch {
print("Could not start OpenAI realtime: \(error)")
}
}
}
private func stopRealtime() {
Task {
await realtimeManager.stopConversation()
}
}
var body: some View {
VStack {
Button(isRealtimeActive ? "Stop OpenAI Realtime" : "Start OpenAI Realtime") {
self.isRealtimeActive.toggle()
}
}
}
}
@AIProxyActor
final class RealtimeManager {
private var realtimeSession: OpenAIRealtimeSession?
private var audioController: AudioController?
nonisolated init() {}
func startConversation() async throws {
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment to protect your connection through AIProxy */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
// Set to false if you want your user to speak first
let aiSpeaksFirst = true
let audioController = try await AudioController(modes: [.playback, .record])
let micStream = try audioController.micStream()
// Start the realtime session:
let configuration = OpenAIRealtimeSessionConfiguration(
inputAudioFormat: .pcm16,
inputAudioTranscription: .init(model: "whisper-1"),
instructions: "You are a tour guide of Yosemite national park",
maxResponseOutputTokens: .int(4096),
modalities: [.audio, .text],
outputAudioFormat: .pcm16,
temperature: 0.7,
turnDetection: .init(
type: .semanticVAD(eagerness: .medium)
),
voice: "shimmer"
)
let realtimeSession = try await openAIService.realtimeSession(
model: "gpt-4o-mini-realtime-preview-2024-12-17",
configuration: configuration,
logLevel: .debug
)
// Send audio from the microphone to OpenAI once OpenAI is ready for it:
var isOpenAIReadyForAudio = false
Task {
for await buffer in micStream {
if isOpenAIReadyForAudio, let base64Audio = AIProxy.base64EncodeAudioPCMBuffer(from: buffer) {
await realtimeSession.sendMessage(
OpenAIRealtimeInputAudioBufferAppend(audio: base64Audio)
)
}
}
}
// Listen for messages from OpenAI:
Task {
for await message in realtimeSession.receiver {
switch message {
case .error(_):
realtimeSession.disconnect()
case .sessionUpdated:
if aiSpeaksFirst {
await realtimeSession.sendMessage(OpenAIRealtimeResponseCreate())
} else {
isOpenAIReadyForAudio = true
}
case .responseAudioDelta(let base64String):
audioController.playPCM16Audio(base64String: base64String)
case .inputAudioBufferSpeechStarted:
audioController.interruptPlayback()
case .responseCreated:
isOpenAIReadyForAudio = true
default:
break
}
}
}
self.realtimeSession = realtimeSession
self.audioController = audioController
}
func stopConversation() {
self.audioController?.stop()
self.realtimeSession?.disconnect()
self.audioController = nil
self.realtimeSession = nil
}
}
```
### How to make a basic request using OpenAI's Responses API
Note: there is also a streaming version of this snippet below.
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateResponseRequestBody(
input: .text("hello world"),
model: "gpt-5.1",
reasoning: .init(effort: .noReasoning),
text: .init(verbosity: .high), // Optional: Use low verbosity for concise responses
previousResponseId: nil // Pass this on future requests to save chat history
)
do {
let response = try await openAIService.createResponse(requestBody: requestBody)
print(response.outputText)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a text response from OpenAI: \(error)")
}
```
### How to make a tool call request with OpenAI's Responses API
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let schema: [String: AIProxyJSONValue] = [
"type": "object",
"properties": [
"location": [
"type": "string",
"description": "City and country e.g. Bogotá, Colombia"
]
],
"required": ["location"],
"additionalProperties": false
]
let requestBody = OpenAICreateResponseRequestBody(
input: .text("What is the weather like in Paris today?"),
model: "gpt-4o",
tools: [
.function(
.init(
name: "get_weather",
parameters: schema
)
)
]
)
do {
let response = try await openAIService.createResponse(
requestBody: requestBody,
secondsToWait: 60
)
if case .functionCall(let functionCall) = response.output.first {
print("""
The model wants to call function: \(functionCall.name)
with arguments: \(functionCall.arguments)")
""")
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create a tool call response from OpenAI: \(error)")
}
```
### How to make a Structured Outputs request with OpenAI's Responses API
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let schema: [String: AIProxyJSONValue] = [
"type": "object",
"properties": [
"colors": [
"type": "array",
"items": [
"type": "object",
"properties": [
"name": [
"type": "string",
"description": "A descriptive name to give the color"
],
"hex_code": [
"type": "string",
"description": "The hex code of the color"
]
],
"required": ["name", "hex_code"],
"additionalProperties": false
]
]
],
"required": ["colors"],
"additionalProperties": false
]
let requestBody = OpenAICreateResponseRequestBody(
input: .items([
.message(role: .system, content: .text("You are a color palette generator")),
.message(role: .user, content: .text("Return a peaches and cream color palette"))
]),
model: "gpt-4o",
text: .init(
format: .jsonSchema(
name: "palette",
schema: schema,
description: "A list of colors that make up a color pallete",
strict: true
)
)
)
do {
let response = try await openAIService.createResponse(
requestBody: requestBody,
secondsToWait: 120
)
print(response.outputText)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a structured output response from OpenAI: \(error)")
}
```
### How to make a JSON mode request with OpenAI's Responses API
Please also see the Structured Outputs snippet above, which is a more modern way of getting a JSON response
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateResponseRequestBody(
input: .items([
.message(role: .system, content: .text("Return valid JSON only")),
.message(role: .user, content: .text("Return alice and bob in a list of names"))
]),
model: "gpt-4o",
text: .init(format: .jsonObject)
)
do {
let response = try await openAIService.createResponse(
requestBody: requestBody,
secondsToWait: 120
)
print(response.outputText)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a JSON mode response from OpenAI: \(error)")
}
```
### How to use an image as input (multi-modal) using OpenAI's Responses API
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let image = UIImage(named: "myImage") else {
print("Could not find an image named 'myImage' in your app assets")
return
}
guard let imageURL = AIProxy.encodeImageAsURL(image: image, compressionQuality: 0.5) else {
print("Could not encode image as data URL")
return
}
let requestBody = OpenAICreateResponseRequestBody(
input: .items(
[
.message(
role: .system,
content: .text("You are a visual assistant")
),
.message(
role: .user,
content: .list([
.text("What do you see?"),
.imageURL(imageURL)
])
)
]
),
model: "gpt-4o"
)
do {
let response = try await openAIService.createResponse(
requestBody: requestBody,
secondsToWait: 60
)
print(response.outputText)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create a multi-modal OpenAI Response: \(error)")
}
```
### How to make a web search call using OpenAI's Responses API
Note: there is also a streaming version of this snippet below.
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateResponseRequestBody(
input: .text("What is Apple's stock price today?"),
model: "gpt-4o",
tools: [
.webSearch(.init(searchContextSize: .low))
]
)
do {
let response = try await openAIService.createResponse(requestBody: requestBody)
print(response.outputText)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get web search result from OpenAI: \(error)")
}
```
### How to use a stored file in an OpenAI prompt using the Responses API
Note: This example is for including a file in a prompt. For searching through a file, see the vector store examples below.
Replace the `fileID` with the ID returned from the snippet `How to upload a file to OpenAI's file storage`
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateResponseRequestBody(
input: .items(
[
.message(role: .user, content: .list(
[
.file(fileID: "your-file-ID"),
.text("What is the purpose of this doc?")
])
)
]
),
model: "gpt-4o"
)
do {
let response = try await openAIService.createResponse(requestBody: requestBody)
print(response.outputText)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not prompt with file contents: \(error)")
}
```
### How to use image inputs in the OpenAI Responses API
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
// Example of using a local image
guard let garmentImage = NSImage(named: "tshirt"),
let garmentImageURL = AIProxy.encodeImageAsURL(image: garmentImage,
compressionQuality: 0.5) else {
print("Could not find an image named 'tshirt' in your app assets")
return
}
// Example of using a remote image
let remoteImageURL = URL(string: "https://www.aiproxy.com/assets/img/requests.png")!
let requestBody = OpenAICreateResponseRequestBody(
input: .items(
[
.message(
role: .user,
content: .list([
.text("What are in these images?"),
.imageURL(garmentImageURL),
.imageURL(remoteImageURL),
])
),
]
),
model: "gpt-4o-mini"
)
do {
let response = try await openAIService.createResponse(requestBody: requestBody)
print(response.outputText)
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not prompt with image inputs: \(error)")
}
```
### How to create a vector store with default chunking on OpenAI
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateVectorStoreRequestBody(
chunkingStrategy: .auto,
name: "my-vector-store"
)
do {
let vectorStore = try await openAIService.createVectorStore(
requestBody: requestBody,
secondsToWait: 60
)
print("Created vector store with id: \(vectorStore.id)")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create an OpenAI vector store: \(error)")
}
```
### How to create a vector store with specific chunking on OpenAI
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateVectorStoreRequestBody(
chunkingStrategy: .static(chunkOverlapTokens: 300, maxChunkSizeTokens: 700),
name: "my-vector-store"
)
do {
let vectorStore = try await openAIService.createVectorStore(
requestBody: requestBody,
secondsToWait: 60
)
print("Created vector store with id: \(vectorStore.id)")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create an OpenAI vector store: \(error)")
}
```
### How to upload a file to OpenAI's file storage
Add the file `The-Swift-Programming-Language.pdf` to your Xcode project tree.
This will upload the pdf to OpenAI for use in a future vector store request:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let localURL = Bundle.main.url(forResource: "The-Swift-Programming-Language", withExtension: "pdf"),
let pdfData = try? Data(contentsOf: localURL) else {
print("Drop The-Swift-Programming-Language.pdf file the project tree first.")
return
}
do {
let openAIFile = try await openAIService.uploadFile(
contents: pdfData,
name: "The-Swift-Programming-Language.pdf",
purpose: .userData,
secondsToWait: 300
)
print("""
File uploaded to OpenAI's media storage.
It will be available until \(openAIFile.expiresAt.flatMap {String($0)} ?? "forever")
Use it in subsequent requests with ID: \(openAIFile.id)
""")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not upload file to OpenAI: \(error)")
}
```
### How to add an uploaded file to an OpenAI vector store
You'll need two IDs for this snippet:
1. The file ID returned in the `How to upload a file to OpenAI's file storage` snippet
2. The vector store ID returned in the `How to create a vector store with default chunking on OpenAI` snippet
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let fileID = /* ID from the file upload example */
let vectorStoreID = /* ID from the vector store example */
let requestBody = OpenAICreateVectorStoreFileRequestBody(
fileID: fileID
)
do {
let vectorStoreFile = try await openAIService.createVectorStoreFile(
vectorStoreID: vectorStoreID,
requestBody: requestBody,
secondsToWait: 120
)
print("Created vector store file with id: \(vectorStoreFile.id ?? "unknown")")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create an OpenAI vector store file: \(error)")
}
```
### How to make a streaming request using OpenAI's Responses API
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateResponseRequestBody(
input: .text("hello world"),
model: "gpt-4o"
)
do {
let stream = try await openAIService.createStreamingResponse(requestBody: requestBody)
for try await event in stream {
switch event {
case .outputTextDelta(let outputTextDelta):
print(outputTextDelta.delta)
default:
break
}
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a streaming response from OpenAI: \(error)")
}
```
### How to make a streaming function call through OpenAI's Responses API
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let schema: [String: AIProxyJSONValue] = [
"type": "object",
"properties": [
"location": [
"type": "string",
"description": "City and country e.g. Bogotá, Colombia"
]
],
"required": ["location"],
"additionalProperties": false
]
let requestBody = OpenAICreateResponseRequestBody(
input: .text("What is the weather like in Paris today?"),
model: "gpt-4o",
tools: [
.function(
.init(
name: "get_weather",
parameters: schema
)
)
]
)
do {
let stream = try await openAIService.createStreamingResponse(requestBody: requestBody)
for try await event in stream {
switch event {
case .functionCallArgumentsDelta(let functionCallArgumentsDelta):
print(functionCallArgumentsDelta.delta)
default:
break
}
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a streaming response from OpenAI: \(error)")
}
```
### How to make a streaming web search call through OpenAI's Responses API
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = OpenAICreateResponseRequestBody(
input: .text("What is Apple's stock price today?"),
model: "gpt-4o",
tools: [
.webSearch(.init(searchContextSize: .low))
]
)
do {
let stream = try await openAIService.createStreamingResponse(requestBody: requestBody)
for try await event in stream {
switch event {
case .outputTextDelta(let outputTextDelta):
print(outputTextDelta.delta)
default:
break
}
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a text response from OpenAI: \(error)")
}
```
### How to make a streaming file search call through OpenAI's Responses API
To run this snippet, you'll first need to add your files to a vector store.
See the snippet above titled `How to add an uploaded file to an OpenAI vector store`.
Once your files are added and processed, you can run this snippet on your `vectorStoreID`.
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let vectorStoreID = /* your vector store ID */
let requestBody = OpenAICreateResponseRequestBody(
input: .text("How do I use 'async let'?"),
model: "gpt-4o",
tools: [
.fileSearch(
.init(
vectorStoreIDs: [
vectorStoreID
]
)
)
]
)
do {
let stream = try await openAIService.createStreamingResponse(requestBody: requestBody)
for try await event in stream {
switch event {
case .outputTextDelta(let outputTextDelta):
print(outputTextDelta.delta)
case .outputTextAnnotationAdded(let outputTextAnnotationAdded):
if case .fileCitation(let fileCitation) = outputTextAnnotationAdded.annotation {
print("Citing: \(fileCitation.filename) at index: \(fileCitation.index)")
}
default:
break
}
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a text response from OpenAI: \(error)")
}
```
### How to use prompt templates with the OpenAI Responses API
- Follow OpenAI's guide for [creating a prompt](https://platform.openai.com/docs/guides/prompting#create-a-prompt).
- Use the returned prompt ID in the snippet below
- Fill in the prompt variables as part of the request body (for example, I used the variable 'topic' below)
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let openAIService = AIProxy.openAIDirectService(
// unprotectedAPIKey: "your-openai-key"
// )
/* Uncomment for all other production use cases */
// let openAIService = AIProxy.openAIService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let templateID = ""
let requestBody = OpenAICreateResponseRequestBody(
model: "gpt-5",
prompt: .init(
id: templateID,
variables: [
"topic": .text("sandwiches")
],
version: "1"
)
)
do {
// Uncomment for the buffered case:
let response = try await openAIService.createResponse(
requestBody: requestBody,
secondsToWait: 60
)
print(response.outputText)
// Uncomment for the streaming case:
// let stream = try await openAIService.createStreamingResponse(
// requestBody: requestBody,
// secondsToWait: 60
// )
// for try await event in stream {
// switch event {
// case .outputTextDelta(let outputTextDelta):
// print(outputTextDelta.delta)
// default:
// break
// }
// }
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not get a text response from OpenAI: \(error)")
}
}
```
### How to use OpenAI through an Azure deployment
You can use all of the OpenAI snippets aboves with one change. Initialize the OpenAI service with:
```swift
import AIProxy
let openAIService = AIProxy.openAIService(
partialKey: "partial-key-from-your-developer-dashboard",
serviceURL: "service-url-from-your-developer-dashboard",
requestFormat: .azureDeployment(apiVersion: "2024-06-01")
)
```
***
## Gemini
### How to generate text content with Gemini
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [.text("How do I use product xyz?")]
)
],
systemInstruction: .init(parts: [.text("Introduce yourself as a customer support person")])
)
do {
let response = try await geminiService.generateContentRequest(
body: requestBody,
model: "gemini-2.5-flash",
secondsToWait: 60
)
for part in response.candidates?.first?.content?.parts ?? [] {
switch part {
case .text(let text):
print("Gemini sent: \(text)")
case .functionCall(name: let functionName, args: let arguments):
print("Gemini wants us to call function \(functionName) with arguments: \(arguments ?? [:])")
}
}
if let usage = response.usageMetadata {
print(
"""
Used:
\(usage.promptTokenCount ?? 0) prompt tokens
\(usage.cachedContentTokenCount ?? 0) cached tokens
\(usage.candidatesTokenCount ?? 0) candidate tokens
\(usage.totalTokenCount ?? 0) total tokens
"""
)
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create Gemini generate content request: \(error)")
}
```
### How to generate streaming text content with Gemini
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [.text("How do I use product xyz?")]
)
],
safetySettings: [
.init(category: .dangerousContent, threshold: .none),
.init(category: .civicIntegrity, threshold: .none),
.init(category: .harassment, threshold: .none),
.init(category: .hateSpeech, threshold: .none),
.init(category: .sexuallyExplicit, threshold: .none)
],
systemInstruction: .init(parts: [.text("Introduce yourself as a customer support person")])
)
do {
let stream = try await geminiService.generateStreamingContentRequest(
body: requestBody,
model: "gemini-2.5-flash",
secondsToWait: 60
)
for try await chunk in stream {
for part in chunk.candidates?.first?.content?.parts ?? [] {
if case .text(let text) = part {
print(text)
}
}
}
print("Gemini finished streaming")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not generate Gemini streaming content: \(error)")
}
```
### How to make a tool call with Gemini
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let functionParameters: [String: AIProxyJSONValue] = [
"type": "OBJECT",
"properties": [
"brightness": [
"description": "Light level from 0 to 100. Zero is off and 100 is full brightness.",
"type": "NUMBER"
],
"colorTemperature": [
"description": "Color temperature of the light fixture which can be `daylight`, `cool` or `warm`.",
"type": "STRING"
]
],
"required": [
"brightness",
"colorTemperature"
]
]
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [.text("Dim the lights so the room feels cozy and warm.")],
role: "user"
)
],
/* Uncomment this to enforce that a function is called regardless of prompt contents. */
// toolConfig: .init(
// functionCallingConfig: .init(
// allowedFunctionNames: ["controlLight"],
// mode: .anyFunction
// )
// ),
tools: [
.functionDeclarations(
[
.init(
name: "controlLight",
description: "Set the brightness and color temperature of a room light.",
parameters: functionParameters
)
]
)
]
)
do {
let response = try await geminiService.generateContentRequest(
body: requestBody,
model: "gemini-2.5-flash",
secondsToWait: 60
)
for part in response.candidates?.first?.content?.parts ?? [] {
switch part {
case .text(let text):
print("Gemini sent: \(text)")
case .functionCall(name: let functionName, args: let arguments):
print("Gemini wants us to call function \(functionName) with arguments: \(arguments ?? [:])")
}
}
if let usage = response.usageMetadata {
print(
"""
Used:
\(usage.promptTokenCount ?? 0) prompt tokens
\(usage.cachedContentTokenCount ?? 0) cached tokens
\(usage.candidatesTokenCount ?? 0) candidate tokens
\(usage.totalTokenCount ?? 0) total tokens
"""
)
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create Gemini tool (function) call: \(error)")
}
```
### How to make a google search grounding call with Gemini 2.0
It's important that you connect a GCP billing account to your Gemini API key to use this
feature. Otherwise, Gemini will return 429s for every call. You can connect your billing
account for the API keys you use [here](https://aistudio.google.com/app/apikey).
Consider applying to [google for startups](https://cloud.google.com/startup?hl=en) to gain
credits that you can put towards Gemini.
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [.text("What is the price of Google stock today")],
role: "user"
)
],
generationConfig: .init(
temperature: 0.7
),
systemInstruction: .init(
parts: [.text("You are a helpful assistant")]
),
tools: [
.googleSearch()
]
)
do {
let response = try await geminiService.generateContentRequest(
body: requestBody,
model: "gemini-2.5-flash",
secondsToWait: 60
)
for candidate in response.candidates ?? [] {
for part in candidate.content?.parts ?? [] {
if case .text(let text) = part {
print("Gemini sent: \(text)\n")
print("Gemini used \(candidate.groundingMetadata?.groundingChunks?.count ?? 0) grounding chunks")
print("Gemini used \(candidate.groundingMetadata?.groundingSupports?.count ?? 0) grounding supports")
}
}
for groundingChunk in candidate.groundingMetadata?.groundingChunks ?? [] {
if let url = groundingChunk.web?.url {
print("Grounding URL: \(url)")
}
}
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create Gemini google search grounding request: \(error)")
}
```
### How to transcribe audio with Gemini
Add a file called `helloworld.m4a` to your Xcode assets before running this sample snippet:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let url = Bundle.main.url(forResource: "helloworld", withExtension: "m4a") else {
print("Could not find an audio file named helloworld.m4a in your app bundle")
return
}
do {
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [
.text("""
Can you transcribe this interview, in the format of timecode, speaker, caption?
Use speaker A, speaker B, etc. to identify speakers.
"""),
.inline(data: try Data(contentsOf: url), mimeType: "audio/mp4")
]
)
]
)
let response = try await geminiService.generateContentRequest(
body: requestBody,
model: "gemini-2.5-flash",
secondsToWait: 60
)
for part in response.candidates?.first?.content?.parts ?? [] {
switch part {
case .text(let text):
print("Gemini transcript: \(text)")
case .functionCall(name: let functionName, args: let arguments):
print("Gemini wants us to call function \(functionName) with arguments: \(arguments ?? [:])")
}
}
if let usage = response.usageMetadata {
print(
"""
Used:
\(usage.promptTokenCount ?? 0) prompt tokens
\(usage.cachedContentTokenCount ?? 0) cached tokens
\(usage.candidatesTokenCount ?? 0) candidate tokens
\(usage.totalTokenCount ?? 0) total tokens
"""
)
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create transcript with Gemini: \(error)")
}
```
### How to use images in the prompt to Gemini
Add a file called 'my-image.jpg' to Xcode app assets. Then run this snippet:
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
guard let image = NSImage(named: "my-image") else {
print("Could not find an image named 'my-image' in your app assets")
return
}
guard let jpegData = AIProxy.encodeImageAsJpeg(image: image, compressionQuality: 0.6) else {
print("Could not encode image as Jpeg")
return
}
do {
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [
.text("What do you see?"),
.inline(
data: jpegData,
mimeType: "image/jpeg"
)
]
)
],
safetySettings: [
.init(category: .dangerousContent, threshold: .none),
.init(category: .civicIntegrity, threshold: .none),
.init(category: .harassment, threshold: .none),
.init(category: .hateSpeech, threshold: .none),
.init(category: .sexuallyExplicit, threshold: .none)
]
)
let response = try await geminiService.generateContentRequest(
body: requestBody,
model: "gemini-2.5-flash",
secondsToWait: 60
)
for part in response.candidates?.first?.content?.parts ?? [] {
switch part {
case .text(let text):
print("Gemini sees: \(text)")
case .functionCall(name: let functionName, args: let arguments):
print("Gemini wants us to call function \(functionName) with arguments: \(arguments ?? [:])")
}
}
if let usage = response.usageMetadata {
print(
"""
Used:
\(usage.promptTokenCount ?? 0) prompt tokens
\(usage.cachedContentTokenCount ?? 0) cached tokens
\(usage.candidatesTokenCount ?? 0) candidate tokens
\(usage.totalTokenCount ?? 0) total tokens
"""
)
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create Gemini generate content request: \(error)")
}
```
### How to upload a video file to Gemini temporary storage
Add a file called `my-movie.mov` to your Xcode assets before running this sample snippet.
If you use a file like `my-movie.mp4`, change the mime type from `video/quicktime` to `video/mp4` in the snippet below.
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
// Try to upload the zip file in Xcode assets
// Get the images to train with:
guard let movieAsset = NSDataAsset(name: "my-movie") else {
print("""
Drop my-movie.mov into Assets first.
""")
return
}
do {
let geminiFile = try await geminiService.uploadFile(
fileData: movieAsset.data,
mimeType: "video/quicktime"
)
print("""
Video file uploaded to Gemini's media storage.
It will be available for 48 hours.
Find it at \(geminiFile.uri.absoluteString)
""")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received non-200 status code: \(statusCode) with response body: \(responseBody)")
} catch {
print("Could not upload file to Gemini: \(error)")
}
```
### How to convert video contents to text with Gemini
Use the file URL returned from the snippet above.
```swift
import AIProxy
let fileURL = URL(string: "url-from-snippet-above")!
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [
.text("Dump the text content in markdown from this video"),
.file(
url: fileURL,
mimeType: "video/quicktime"
)
]
)
],
safetySettings: [
.init(category: .dangerousContent, threshold: .none),
.init(category: .civicIntegrity, threshold: .none),
.init(category: .harassment, threshold: .none),
.init(category: .hateSpeech, threshold: .none),
.init(category: .sexuallyExplicit, threshold: .none)
]
)
do {
let response = try await geminiService.generateContentRequest(
body: requestBody,
model: "gemini-2.5-flash",
secondsToWait: 60
)
for part in response.candidates?.first?.content?.parts ?? [] {
switch part {
case .text(let text):
print("Gemini transcript: \(text)")
case .functionCall(name: let functionName, args: let arguments):
print("Gemini wants us to call function \(functionName) with arguments: \(arguments ?? [:])")
}
}
if let usage = response.usageMetadata {
print(
"""
Used:
\(usage.promptTokenCount ?? 0) prompt tokens
\(usage.cachedContentTokenCount ?? 0) cached tokens
\(usage.candidatesTokenCount ?? 0) candidate tokens
\(usage.totalTokenCount ?? 0) total tokens
"""
)
}
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not create Gemini vision request: \(error)")
}
```
### How to delete a temporary file from Gemini storage
```swift
import AIProxy
let fileURL = URL(string: "url-from-snippet-above")!
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
do {
try await geminiService.deleteFile(fileURL: fileURL)
print("File deleted from \(fileURL.absoluteString)")
} catch AIProxyError.unsuccessfulRequest(let statusCode, let responseBody) {
print("Received \(statusCode) status code with response body: \(responseBody)")
} catch {
print("Could not delete file from Gemini temporary storage: \(error)")
}
```
### How to use structured ouputs with Gemini
```swift
import AIProxy
/* Uncomment for BYOK use cases */
// let geminiService = AIProxy.geminiDirectService(
// unprotectedAPIKey: "your-gemini-key"
// )
/* Uncomment for all other production use cases */
// let geminiService = AIProxy.geminiService(
// partialKey: "partial-key-from-your-developer-dashboard",
// serviceURL: "service-url-from-your-developer-dashboard"
// )
let schema: [String: AIProxyJSONValue] = [
"description": "List of recipes",
"type": "array",
"items": [
"type": "object",
"properties": [
"recipeName": [
"type": "string",
"description": "Name of the recipe",
"nullable": false
]
],
"required": ["recipeName"]
]
]
do {
let requestBody = GeminiGenerateContentRequestBody(
contents: [
.init(
parts: [
.text("List a few popular cookie recipes."),
]
)
],
generationConfig: .init(
responseMimeType: "application/json",
responseSchema: schema