{"id":15527110,"url":"https://github.com/stanfordspezi/spezillm","last_synced_at":"2025-12-11T22:55:41.907Z","repository":{"id":167775957,"uuid":"643402647","full_name":"StanfordSpezi/SpeziLLM","owner":"StanfordSpezi","description":"Large Language Model (LLM) module for the Spezi Ecosystem","archived":false,"fork":false,"pushed_at":"2025-03-19T18:50:10.000Z","size":1469,"stargazers_count":218,"open_issues_count":16,"forks_count":25,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-12T21:32:31.680Z","etag":null,"topics":["chatbot","gpt","ios","large-language-models","llm","openai","spezi","swift","swiftui"],"latest_commit_sha":null,"homepage":"https://swiftpackageindex.com/StanfordSpezi/SpeziLLM/documentation","language":"Swift","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/StanfordSpezi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":"CITATION.cff","codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"custom":["spezi.sites.stanford.edu"]}},"created_at":"2023-05-21T03:32:01.000Z","updated_at":"2025-04-06T19:56:04.000Z","dependencies_parsed_at":"2023-10-29T14:33:09.592Z","dependency_job_id":"13e1e163-03cf-495f-b910-3ff6eb2509a0","html_url":"https://github.com/StanfordSpezi/SpeziLLM","commit_stats":{"total_commits":43,"total_committers":9,"mean_commits":4.777777777777778,"dds":0.4883720930232558,"last_synced_commit":"c64983b3b303cd38691aa8cf2df01ab034e3efaf"},"previous_names":["stanfordspezi/speziml","stanfordspezi/spezillm"],"tags_count":35,"template":false,"template_full_name":"StanfordBDHG/SwiftPackageTemplate","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StanfordSpezi%2FSpeziLLM","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StanfordSpezi%2FSpeziLLM/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StanfordSpezi%2FSpeziLLM/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/StanfordSpezi%2FSpeziLLM/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/StanfordSpezi","download_url":"https://codeload.github.com/StanfordSpezi/SpeziLLM/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248640156,"owners_count":21137986,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chatbot","gpt","ios","large-language-models","llm","openai","spezi","swift","swiftui"],"created_at":"2024-10-02T11:04:29.184Z","updated_at":"2025-12-11T22:55:41.900Z","avatar_url":"https://github.com/StanfordSpezi.png","language":"Swift","funding_links":["spezi.sites.stanford.edu"],"categories":[],"sub_categories":[],"readme":"\u003c!--\n                  \nThis source file is part of the Stanford Spezi open source project\n\nSPDX-FileCopyrightText: 2023 Stanford University and the project authors (see CONTRIBUTORS.md)\n\nSPDX-License-Identifier: MIT\n             \n--\u003e\n\n# Spezi LLM\n\n[![Build and Test](https://github.com/StanfordSpezi/SpeziLLM/actions/workflows/build-and-test.yml/badge.svg)](https://github.com/StanfordSpezi/SpeziLLM/actions/workflows/build-and-test.yml)\n[![codecov](https://codecov.io/gh/StanfordSpezi/SpeziLLM/branch/main/graph/badge.svg?token=pptLyqtoNR)](https://codecov.io/gh/StanfordSpezi/SpeziLLM)\n[![DOI](https://zenodo.org/badge/DOI/10.5281/zenodo.7954213.svg)](https://doi.org/10.5281/zenodo.7954213)\n[![](https://img.shields.io/endpoint?url=https%3A%2F%2Fswiftpackageindex.com%2Fapi%2Fpackages%2FStanfordSpezi%2FSpeziLLM%2Fbadge%3Ftype%3Dswift-versions)](https://swiftpackageindex.com/StanfordSpezi/SpeziLLM)\n[![](https://img.shields.io/endpoint?url=https%3A%2F%2Fswiftpackageindex.com%2Fapi%2Fpackages%2FStanfordSpezi%2FSpeziLLM%2Fbadge%3Ftype%3Dplatforms)](https://swiftpackageindex.com/StanfordSpezi/SpeziLLM)\n\n\n## Overview\n\nThe Spezi LLM Swift Package includes modules that are helpful to integrate LLM-related functionality in your application.\nThe package provides all necessary tools for local LLM execution, the usage of remote OpenAI-based LLMs, as well as LLMs running on Fog node resources within the local network.\n\n|\u003cpicture\u003e\u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"Sources/SpeziLLMOpenAI/SpeziLLMOpenAI.docc/Resources/ChatView~dark.png\"\u003e\u003cimg src=\"Sources/SpeziLLMOpenAI/SpeziLLMOpenAI.docc/Resources/ChatView.png\" width=\"250\" alt=\"Screenshot displaying the Chat View utilizing the OpenAI API from SpeziLLMOpenAI.\" /\u003e\u003c/picture\u003e|\u003cpicture\u003e\u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"Sources/SpeziLLMLocalDownload/SpeziLLMLocalDownload.docc/Resources/LLMLocalDownload~dark.png\"\u003e\u003cimg src=\"Sources/SpeziLLMLocalDownload/SpeziLLMLocalDownload.docc/Resources/LLMLocalDownload.png\" width=\"250\" alt=\"Screenshot displaying the Local LLM Download View from SpeziLLMLocalDownload.\" /\u003e\u003c/picture\u003e|\u003cpicture\u003e\u003csource media=\"(prefers-color-scheme: dark)\" srcset=\"Sources/SpeziLLMLocal/SpeziLLMLocal.docc/Resources/ChatView~dark.png\"\u003e\u003cimg src=\"Sources/SpeziLLMLocal/SpeziLLMLocal.docc/Resources/ChatView.png\" width=\"250\" alt=\"Screenshot displaying the Chat View utilizing a locally executed LLM via SpeziLLMLocal.\" /\u003e\u003c/picture\u003e|\n|:--:|:--:|:--:|\n|`OpenAI LLM Chat View`|`Language Model Download`|`Local LLM Chat View`|\n\n## Setup\n\n### 1. Add Spezi LLM as a Dependency\n\nYou need to add the SpeziLLM Swift package to\n[your app in Xcode](https://developer.apple.com/documentation/xcode/adding-package-dependencies-to-your-app#) or\n[Swift package](https://developer.apple.com/documentation/xcode/creating-a-standalone-swift-package-with-xcode#Add-a-dependency-on-another-Swift-package).\n\n\u003e [!IMPORTANT]  \n\u003e If your application is not yet configured to use Spezi, follow the [Spezi setup article](https://swiftpackageindex.com/stanfordspezi/spezi/documentation/spezi/initial-setup) to set up the core Spezi infrastructure.\n\n### 2. Follow the setup steps of the individual targets\n\nAs Spezi LLM contains a variety of different targets for specific LLM functionalities, please follow the additional setup guide in the respective target section of this README.\n\n## Targets\n\nSpezi LLM provides a number of targets to help developers integrate LLMs in their Spezi-based applications:\n- [SpeziLLM](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm): Base infrastructure of LLM execution in the Spezi ecosystem.\n- [SpeziLLMLocal](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmlocal): Local LLM execution capabilities directly on-device. Enables running open-source LLMs from Hugging Face like [Meta's Llama2](https://ai.meta.com/llama/), [Microsoft's Phi](https://azure.microsoft.com/en-us/products/phi), [Google's Gemma](https://ai.google.dev/gemma), or [DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1), among others. See [LLMLocalModel](https://swiftpackageindex.com/stanfordspezi/spezillm/main/documentation/spezillmlocal/llmlocalmodel) for a list of models tested with SpeziLLM.\n- [SpeziLLMLocalDownload](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmlocaldownload): Download and storage manager of local Language Models, including onboarding views. \n- [SpeziLLMOpenAI](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmopenai): Integration with OpenAI's GPT models via using OpenAI's API service.\n- [SpeziLLMFog](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmfog): Discover and dispatch LLM inference jobs to Fog node resources within the local network.\n\nThe section below highlights the setup and basic use of the [SpeziLLMLocal](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmlocal), [SpeziLLMOpenAI](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmopenai), and [SpeziLLMFog](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmfog) targets in order to integrate Language Models in a Spezi-based application. \n\n\u003e [!NOTE]  \n\u003e To learn more about the usage of the individual targets, please refer to the [DocC documentation of the package](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation).\n\n### Spezi LLM Local\n\nThe target enables developers to easily execute medium-size Language Models (LLMs) locally on-device. The module allows you to interact with the locally run LLM via purely Swift-based APIs, no interaction with low-level code is necessary, building on top of the infrastructure of the [SpeziLLM target](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm).\n\n\u003e [!IMPORTANT]  \n\u003e Spezi LLM Local is not compatible with simulators. The underlying [`mlx-swift`](https://github.com/ml-explore/mlx-swift) requires a modern Metal MTLGPUFamily and the simulator does not provide that.\n\n\u003e [!IMPORTANT]\n\u003e To use the LLM local target, some LLMs require adding the *Increase Memory Limit* entitlement to the project.\n\n#### Setup\n\nYou can configure the Spezi Local LLM execution within the typical `SpeziAppDelegate`.\nIn the example below, the `LLMRunner` from the [SpeziLLM](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) target which is responsible for providing LLM functionality within the Spezi ecosystem is configured with the `LLMLocalPlatform` from the [SpeziLLMLocal](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmlocal) target. This prepares the `LLMRunner` to locally execute Language Models.\n\n```swift\nclass TestAppDelegate: SpeziAppDelegate {\n    override var configuration: Configuration {\n        Configuration {\n            LLMRunner {\n                LLMLocalPlatform()\n            }\n        }\n    }\n}\n```\n\n[SpeziLLMLocalDownload](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmlocaldownload) can be used to download an LLM from [HuggingFace](https://huggingface.co/) and save it on the device for execution. The `LLMLocalDownloadView` provides an out-of-the-box onboarding view for downloading models locally.\n\n```swift\nstruct LLMLocalOnboardingDownloadView: View {\n    var body: some View {\n        LLMLocalDownloadView(\n            model: .llama3_8B_4bit,\n            downloadDescription: \"The Llama3 8B model will be downloaded\",\n        ) {\n            // Action to perform after the model is downloaded and the user presses the next button.\n        }\n    }\n}\n```\n\n\u003e [!TIP]\n\u003e The `LLMLocalDownloadView` view can be included in your onboarding process using SpeziOnboarding as [demonstrated in this example](https://swiftpackageindex.com/stanfordspezi/spezillm/main/documentation/spezillmlocaldownload/llmlocaldownloadview#overview).\n\n\n#### Usage\n\nThe code example below showcases the interaction with local LLMs through the the [SpeziLLM](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner), which is injected into the SwiftUI `Environment` via the `Configuration` shown above.\n\nThe `LLMLocalSchema` defines the type and configurations of the to-be-executed `LLMLocalSession`. This transformation is done via the [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner) that uses the `LLMLocalPlatform`. The inference via `LLMLocalSession/generate()` returns an `AsyncThrowingStream` that yields all generated `String` pieces.\n\n```swift\nstruct LLMLocalDemoView: View {\n    @Environment(LLMRunner.self) var runner\n    @State var responseText = \"\"\n\n    var body: some View {\n        Text(responseText)\n            .task {\n                // Instantiate the `LLMLocalSchema` to an `LLMLocalSession` via the `LLMRunner`.\n                let llmSession: LLMLocalSession = runner(\n                    with: LLMLocalSchema(\n                        model: .llama3_8B_4bit,\n                    )\n                )\n\n                do {\n                    for try await token in try await llmSession.generate() {\n                        responseText.append(token)\n                    }\n                } catch {\n                    // Handle errors here. E.g., you can use `ViewState` and `viewStateAlert` from SpeziViews.\n                }\n            }\n    }\n}\n```\n\nThe [`LLMChatViewSchema`](https://swiftpackageindex.com/stanfordspezi/spezillm/main/documentation/spezillm/llmchatviewschema) can be used to easily create a conversational chat interface for your chatbot application with a local LLM.\n\n```swift\nstruct LLMLocalChatView: View {\n    var body: some View {\n        LLMChatViewSchema(\n            with: LLMLocalSchema(\n                model: .llama3_8B_4bit\n            )\n        )\n    }\n}\n```\n\n### Offloading\n\nTo optimize inference performance and minimize resource consumption within the application, use the `LLMLocalSession.offload()` method. This function unloads the model from memory, thereby freeing up system resources when the model is not actively in use.\nWhen further interaction with the model is required, calling either `LLMLocalSession.setup()` or `LLMLocalSession.generate()` will automatically reload the model into memory as needed.\n\n\u003e [!NOTE]  \n\u003e To learn more about the usage of SpeziLLMLocal, please refer to the comprehensive [DocC documentation](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmlocal).\n\n### Spezi LLM Open AI\n\nA module that allows you to interact with GPT-based Large Language Models (LLMs) from OpenAI within your Spezi application.\n`SpeziLLMOpenAI` provides a pure Swift-based API for interacting with the OpenAI GPT API, building on top of the infrastructure of the [SpeziLLM target](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm).\nIn addition, `SpeziLLMOpenAI` provides developers with a declarative Domain Specific Language to utilize OpenAI function calling mechanism. This enables a structured, bidirectional, and reliable communication between the OpenAI LLMs and external tools, such as the Spezi ecosystem.\n\n#### Setup\n\nIn order to use OpenAI LLMs within the Spezi ecosystem, the [SpeziLLM](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner) needs to be initialized in the Spezi `Configuration` with the `LLMOpenAIPlatform`. Only after, the `LLMRunner` can be used for inference of OpenAI LLMs.\nSee the [SpeziLLM documentation](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) for more details.\n\n```swift\nimport Spezi\nimport SpeziLLM\nimport SpeziLLMOpenAI\n\nclass LLMOpenAIAppDelegate: SpeziAppDelegate {\n    override var configuration: Configuration {\n        Configuration {\n            LLMRunner {\n                LLMOpenAIPlatform()\n            }\n        }\n    }\n}\n```\n\n\u003e [!IMPORTANT]\n\u003e If using `SpeziLLMOpenAI` on macOS, ensure to add the *`Keychain Access Groups` entitlement* to the enclosing Xcode project via *PROJECT_NAME \u003e Signing\u0026Capabilities \u003e + Capability*. The array of keychain groups can be left empty, only the base entitlement is required.\n\n#### Usage\n\nThe code example below showcases the interaction with an OpenAI LLM through the the [SpeziLLM](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner), which is injected into the SwiftUI `Environment` via the `Configuration` shown above.\n\nThe `LLMOpenAISchema` defines the type and configurations of the to-be-executed `LLMOpenAISession`. This transformation is done via the [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner) that uses the `LLMOpenAIPlatform`. The inference via `LLMOpenAISession/generate()` returns an `AsyncThrowingStream` that yields all generated `String` pieces.\n\n```swift\nimport SpeziLLM\nimport SpeziLLMOpenAI\nimport SwiftUI\n\nstruct LLMOpenAIDemoView: View {\n    @Environment(LLMRunner.self) var runner\n    @State var responseText = \"\"\n\n    var body: some View {\n        Text(responseText)\n            .task {\n                // Instantiate the `LLMOpenAISchema` to an `LLMOpenAISession` via the `LLMRunner`.\n                let llmSession: LLMOpenAISession = runner(\n                    with: LLMOpenAISchema(\n                        parameters: .init(\n                            modelType: .gpt4o,\n                            systemPrompt: \"You're a helpful assistant that answers questions from users.\",\n                            overwritingToken: \"abc123\"\n                        )\n                    )\n                )\n\n                do {\n                    for try await token in try await llmSession.generate() {\n                        responseText.append(token)\n                    }\n                } catch {\n                    // Handle errors here. E.g., you can use `ViewState` and `viewStateAlert` from SpeziViews.\n                }\n            }\n    }\n}\n```\n\n\u003e [!NOTE]  \n\u003e To learn more about the usage of SpeziLLMOpenAI, please refer to the [DocC documentation](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmopenai).\n\n### Spezi LLM Fog\n\nThe `SpeziLLMFog` target enables you to use LLMs running on [Fog node](https://en.wikipedia.org/wiki/Fog_computing) computing resources within the local network. The fog nodes advertise their services via [mDNS](https://en.wikipedia.org/wiki/Multicast_DNS), enabling clients to discover all fog nodes serving a specific host within the local network.\n`SpeziLLMFog` then dispatches LLM inference jobs dynamically to a random fog node within the local network and streams the response to surface it to the user.\n\n\u003e [!IMPORTANT]\n\u003e `SpeziLLMFog` requires a `SpeziLLMFogNode` within the local network hosted on some computing resource that actually performs the inference requests. `SpeziLLMFog` provides the `SpeziLLMFogNode` Docker-based package that enables an easy setup of these fog nodes. See the `FogNode` directory on the root level of the SPM package as well as the respective `README.md` for more details.\n\n\u003e [!IMPORTANT]\n\u003e `SpeziLLMFog` performs dynamic discovery of available fog node services in the local network using Bonjour. To enable this functionality, the consuming application must configure the following `Info.plist` entries:\n\u003e - `NSLocalNetworkUsageDescription` (`String`): A description explaining why the app requires access to the local network. For example:\n`\"This app uses local network access to discover nearby services.\"`\n\u003e - `NSBonjourServices` (`Array\u003cString\u003e`): Specifies the Bonjour service types the app is allowed to discover.\n\u003e For use with `SpeziLLMFog`, include the following entry:\n\u003e   - `_https._tcp` (for discovering secured services via TLS)\n\u003e   - `_http._tcp` (optional, for testing purposes only; discovers unsecured services)\n\n#### Setup\n\nIn order to use Fog LLMs within the Spezi ecosystem, the [SpeziLLM](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner) needs to be initialized in the Spezi `Configuration` with the `LLMFogPlatform`. Only after, the `LLMRunner` can be used for inference with Fog LLMs. See the [SpeziLLM documentation](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) for more details.\nThe `LLMFogPlatform` needs to be initialized with the custom root CA certificate that was used to sign the fog node web service certificate (see the `FogNode/README.md` documentation for more information). Copy the root CA certificate from the fog node as resource to the application using `SpeziLLMFog` and use it to initialize the `LLMFogPlatform` within the Spezi `Configuration`.\n\n```swift\nclass LLMFogAppDelegate: SpeziAppDelegate {\n    private nonisolated static var caCertificateUrl: URL {\n        // Return local file URL of root CA certificate in the `.crt` format\n    }\n    \n    override var configuration: Configuration {\n         Configuration {\n             LLMRunner {\n                // Set up the Fog platform with the custom CA certificate\n                LLMRunner {\n                    LLMFogPlatform(configuration: .init(connectionType: .http, authToken: .none))\n                    // If required, specify `.https` connection type, including the certificate\n                }\n            }\n        }\n    }\n}\n```\n\nIn addition to set local network discovery entitlements described above, users must grant explicit authorization for local network access.\nThis authorization can be requested during the app’s onboarding process using `LLMFogDiscoveryAuthorizationView`.\nIt informs users about the need for local network access, prompts them to grant it, and attempts to verify the access status (note: the OS does not expose this information).\nFor detailed guidance on integrating the `LLMFogDiscoveryAuthorizationView` in an onboarding flow managed by `[SpeziOnboarding`](https://swiftpackageindex.com/stanfordspezi/spezionboarding), refer to the in-line documentation of the `LLMFogDiscoveryAuthorizationView`.\n\n#### Usage\n\nThe code example below showcases the interaction with a Fog LLM through the the [SpeziLLM](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm) [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner), which is injected into the SwiftUI `Environment` via the `Configuration` shown above.\n\nThe `LLMFogSchema` defines the type and configurations of the to-be-executed `LLMFogSession`. This transformation is done via the [`LLMRunner`](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillm/llmrunner) that uses the `LLMFogPlatform`. The inference via `LLMFogSession/generate()` returns an `AsyncThrowingStream` that yields all generated `String` pieces.\nThe `LLMFogSession` automatically discovers all available LLM fog nodes within the local network upon setup and the dispatches the LLM inference jobs to the fog computing resource, streaming back the response and surfaces it to the user.\n\n\u003e [!IMPORTANT]  \n\u003e The `LLMFogSchema` accepts a closure that returns an authorization token that is passed with every request to the Fog node in the `Bearer` HTTP field via the `LLMFogParameters/init(modelType:overwritingAuthToken:systemPrompt:)`. The token is created via the closure upon every LLM inference request, as the `LLMFogSession` may be long lasting and the token could therefore expire. Ensure that the closure appropriately caches the token in order to prevent unnecessary token refresh roundtrips to external systems.\n\n```swift\nstruct LLMFogDemoView: View {\n    @Environment(LLMRunner.self) var runner\n    @State var responseText = \"\"\n\n    var body: some View {\n        Text(responseText)\n            .task {\n                // Instantiate the `LLMFogSchema` to an `LLMFogSession` via the `LLMRunner`.\n                let llmSession: LLMFogSession = runner(\n                    with: LLMFogSchema(\n                        parameters: .init(\n                            modelType: .llama7B,\n                            overwritingAuthToken: .none,    // potentially overwrite default auth token from `LLMFogPlatform`\n                            systemPrompt: \"You're a helpful assistant that answers questions from users.\"\n                        )\n                    )\n                )\n\n                do {\n                    for try await token in try await llmSession.generate() {\n                        responseText.append(token)\n                    }\n                } catch {\n                    // Handle errors here. E.g., you can use `ViewState` and `viewStateAlert` from SpeziViews.\n                }\n            }\n    }\n}\n```\n\n\u003e [!NOTE]  \n\u003e To learn more about the usage of SpeziLLMFog, please refer to the [DocC documentation](https://swiftpackageindex.com/stanfordspezi/spezillm/documentation/spezillmfog).\n\n## Contributing\n\nContributions to this project are welcome. Please make sure to read the [contribution guidelines](https://github.com/StanfordSpezi/.github/blob/main/CONTRIBUTING.md) and the [contributor covenant code of conduct](https://github.com/StanfordSpezi/.github/blob/main/CODE_OF_CONDUCT.md) first.\n\n\n## License\n\nThis project is licensed under the MIT License. See [Licenses](https://github.com/StanfordSpezi/SpeziLLM/tree/main/LICENSES) for more information.\n\n![Spezi Footer](https://raw.githubusercontent.com/StanfordSpezi/.github/main/assets/FooterLight.png#gh-light-mode-only)\n![Spezi Footer](https://raw.githubusercontent.com/StanfordSpezi/.github/main/assets/FooterDark.png#gh-dark-mode-only)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstanfordspezi%2Fspezillm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstanfordspezi%2Fspezillm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstanfordspezi%2Fspezillm/lists"}