https://github.com/getgrinta/swift-llm
Modern Swift LLM SDK with support for AI tools
https://github.com/getgrinta/swift-llm
ai ai-tools llm sdk swift
Last synced: 9 months ago
JSON representation
Modern Swift LLM SDK with support for AI tools
- Host: GitHub
- URL: https://github.com/getgrinta/swift-llm
- Owner: getgrinta
- License: apache-2.0
- Created: 2025-05-22T16:53:22.000Z (11 months ago)
- Default Branch: main
- Last Pushed: 2025-05-25T13:47:27.000Z (11 months ago)
- Last Synced: 2025-06-07T00:09:30.405Z (10 months ago)
- Topics: ai, ai-tools, llm, sdk, swift
- Language: Swift
- Homepage:
- Size: 7.5 MB
- Stars: 2
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# swift-llm
**Simple Swift library for interacting with Large Language Models (LLMs), featuring support for streaming responses and tool integration.**
[](https://github.com/getgrinta/swift-llm/actions/workflows/swift-test.yml)
[](https://swift.org)
[](https://developer.apple.com/ios/)
[](https://swift.org/package-manager/)
[](https://GitHub.com/getgrinta/swift-llm/releases/)
[](LICENSE)
`swift-llm` provides a modern, async/await-based interface to communicate with LLM APIs, making it easy to integrate advanced AI capabilities into your Swift applications.
## Features
- **Core LLM Interaction**: Send requests and receive responses from LLMs using `ChatMessage` arrays for rich conversational context.
- **Streaming Support**: Handle real-time streaming of LLM responses for a more interactive user experience (`AsyncThrowingStream`).
- **Multi-Standard SSE Parsing**: Supports Server-Sent Events (SSE) streams conforming to both OpenAI (`openAi`) and Vercel AI SDK (`vercel`) standards via the `SSETokenizer`.
- **Tool Integration**: Empower your LLM to use predefined tools to perform actions in parallel and gather information, enabling more complex and capable assistants (`TooledLLMClient`).
- **Customizable Requests**: Modify outgoing `URLRequest` objects using a `requestTransformer` closure, allowing for custom headers, body modifications, or different endpoints for stream/non-stream calls.
- **Typed Models**: Clear, `Codable` Swift structures for requests, responses, and tool definitions.
- **Modern Swift**: Built with Swift 6.1+ and leverages modern concurrency features.
- **Easy Integration**: Designed as a Swift Package Manager library.
## Requirements
- Swift 6.0 or later
- iOS 13.0 or later
## Installation
Add `swift-llm` as a dependency to your `Package.swift` file:
```swift
import PackageDescription
let package = Package(
name: "YourProjectName",
platforms: [.iOS(.v13)],
dependencies: [
.package(url: "https://github.com/getgrinta/swift-llm.git", from: "0.1.4")
],
targets: [
.target(
name: "YourProjectTarget",
dependencies: ["swift-llm"]
)
]
)
```
## Usage
### 1. Basic Chat
```swift
import SwiftLLM
let endpoint = "your_llm_api_endpoint"
let modelName = "your_model_name"
let bearerToken = "your_api_key"
// Construct messages - an array of ChatMessage objects
// Assuming ChatMessage(role: .user, content: "...")
let messages = [ChatMessage(role: .user, content: "What is the capital of France?")]
// Initialize the LLMClient with your endpoint, model, and API key
// Default standard is .openAi. For Vercel, specify: SSETokenizer.Standard.vercel
let llmClient = LLMClient(endpoint: endpoint, model: modelName, apiKey: bearerToken)
Task {
do {
print("Sending request...")
// Send messages and optionally set temperature (e.g., 0.7 for some creativity)
let response = try await llmClient.send(messages: messages, temperature: 0.7)
print("LLM Response: \(response.message)")
} catch {
print("Error during non-streaming chat: \(error.localizedDescription)")
}
}
```
### 2. Streaming
```swift
import SwiftLLM
// Initialize the LLMClient with your endpoint, model, and API key
let llmClient = LLMClient(endpoint: endpoint, model: modelName, apiKey: bearerToken)
let endpoint = "your_llm_api_endpoint"
let modelName = "your_model_name"
let bearerToken = "your_api_key"
let messages = [ChatMessage(role: .user, content: "Tell me a short story, stream it part by part.")]
Task {
do {
// Stream messages and optionally set temperature
let stream = try await llmClient.stream(messages: messages, temperature: 0.7)
print("Streaming response:")
for try await chunk in stream {
print(chunk.message, terminator: "") // Append chunks as they arrive
}
print() // Newline after stream finishes
} catch {
print("Error during streaming chat: \(error.localizedDescription)")
}
}
```
### 3. Using `TooledLLMClient`
First, define your tools:
```swift
import SwiftLLM
// Example Tool: Get Current Weather
let weatherTool = LLMTool(
name: "getCurrentWeather",
description: "Gets the current weather for a given location. Arguments should be a plain string with the location name.",
execute: { argumentsAsLocationString in
// In a real scenario, you might parse 'arguments' if it's structured (e.g., XML/JSON)
// For this example, assume 'argumentsAsLocationString' is the location directly.
print("Tool 'getCurrentWeather' called with arguments: \(argumentsAsLocationString)")
// Simulate API call or logic
if argumentsAsLocationString.lowercased().contains("paris") {
return "The weather in Paris is sunny, 25°C."
} else if argumentsAsLocationString.lowercased().contains("london") {
return "It's currently cloudy with a chance of rain in London, 18°C."
} else {
return "Sorry, I don't know the weather for \(argumentsAsLocationString)."
}
}
)
let tools = [weatherTool]
```
Then, use `TooledLLMClient`:
```swift
import SwiftLLM
// Initialize the LLMClient first (used by TooledLLMClient)
// The LLMClient itself now uses ChatMessage and supports temperature,
// though TooledLLMClient might manage this internally for its specific flow.
let llmClient = LLMClient(endpoint: endpoint, model: modelName, apiKey: bearerToken)
let tooledClient = TooledLLMClient(llmClient: llmClient) // Pass the LLMClient instance
let endpoint = "your_llm_api_endpoint_supporting_tools" // Ensure this endpoint supports tool use
let modelName = "your_tool_capable_model_name"
let bearerToken = "your_api_key"
let userInput = "What's the weather like in Paris today?"
Task {
do {
print("User Input: \(userInput)")
let stream = try await tooledClient.processWithTools(
userInput: userInput,
tools: tools
)
print("\nFinal LLM Response (after potential tool use):")
var fullResponse = ""
for try await chunk in stream {
print(chunk.message, terminator: "")
fullResponse += chunk.message
}
print("\n--- Full Assembled Response ---")
print(fullResponse)
} catch let error as TooledLLMClientError {
print("TooledLLMClientError: \(error)")
} catch {
print("An unexpected error occurred: \(error.localizedDescription)")
}
}
```
## Core Components
### `LLMClient`
The primary client for all interactions with an LLM, supporting both non-streaming (single request/response) and streaming (continuous updates) communication. It handles the direct network requests to the LLM API. Interactions are based on `ChatMessage` arrays, allowing for conversational history to be passed to the LLM. It also supports a `temperature` parameter to control response randomness.
**`ChatMessage` Structure (Conceptual):**
Your `ChatMessage` objects would typically include a `role` (e.g., `.user`, `.assistant`, `.system`) and `content` (the text of the message).
**Initializer:**
`public init(standard: SSETokenizer.Standard = .openAi, endpoint: String, model: String, apiKey: String, sessionConfiguration: URLSessionConfiguration = .default, requestTransformer: (@Sendable (URLRequest, _ isStream: Bool) -> URLRequest)? = nil)`
- `standard`: (Optional) The SSE parsing standard to use. Defaults to `.openAi`. Can be set to `.vercel` for Vercel AI SDK compatibility.
- `endpoint`: The base URL for the LLM API.
- `model`: The identifier for the LLM model to be used.
- `apiKey`: Your API key for authentication.
- `sessionConfiguration`: (Optional) A `URLSessionConfiguration` for the underlying network session. Defaults to `.default`.
- `requestTransformer`: (Optional) A closure that allows you to modify the `URLRequest` before it's sent.
**Example of `requestTransformer`:**
```swift
let transformer: @Sendable (URLRequest, Bool) -> URLRequest = { request, isStream in
var mutableRequest = request
// Add a custom header
mutableRequest.setValue("my-custom-value", forHTTPHeaderField: "X-Custom-Header")
// Potentially change endpoint based on stream type
if isStream {
// mutableRequest.url = URL(string: "your_streaming_specific_endpoint")
} else {
// mutableRequest.url = URL(string: "your_non_streaming_specific_endpoint")
}
return mutableRequest
}
let llmClient = LLMClient(
standard: .openAi,
endpoint: vercelEndpoint,
model: vercelModelName,
apiKey: vercelBearerToken,
requestTransformer: transformer
)
```
**Key methods:**
- `public func send(messages: [ChatMessage], temperature: Double? = nil) async throws -> ChatOutput` (for non-streaming requests)
- `public func stream(messages: [ChatMessage], temperature: Double? = nil) -> AsyncThrowingStream` (for setting up a streaming connection)
### `TooledLLMClient`
Manages interactions with an LLM that can utilize a predefined set of tools. It orchestrates a multi-pass conversation:
1. Sends user input (often initially as a string, which it converts to `ChatMessage` for the LLM) and tool descriptions to the LLM.
2. Parses the LLM's decision and executes the identified tools.
3. Sends the tool execution results back to the LLM (as `ChatMessage` objects) to generate a final, user-facing response.
**Initializer:**
`public init(llmClient: LLMClient)`
**Key method:**
- `public func processWithTools(userInput: String, tools: [LLMTool]) async throws -> AsyncThrowingStream`
**Note on Tool Argument Formatting:**
The `TooledLLMClient` includes a default prompt that instructs the LLM to provide arguments like `{"toolsToUse": [{"name": "tool_name", "arguments": ""}]}`. The `arguments` field from this JSON is what gets passed to your tool's `execute` closure. You'll need to:
1. Ensure the LLM you use can follow this JSON instruction for its `tool_calls` response.
2. Adapt your tool's `execute` closure to parse the `arguments` string as needed (e.g., if it's plain text, XML, or a JSON string itself). The example above simplifies this for clarity.
## Contributing
Contributions are welcome! Please feel free to submit a Pull Request or open an Issue if you find a bug or have a feature request.
(Optional: Add guidelines for commit messages, code style, running tests, etc.)
## License
`swift-llm` is released under the Apache License 2.0. See [LICENSE](LICENSE) file for details.