https://github.com/srgtuszy/llama-cpp-swift
Swift bindings for llama-cpp library
https://github.com/srgtuszy/llama-cpp-swift
ios linux llama2 llama3 llm macos swift
Last synced: 10 months ago
JSON representation
Swift bindings for llama-cpp library
- Host: GitHub
- URL: https://github.com/srgtuszy/llama-cpp-swift
- Owner: srgtuszy
- License: mit
- Created: 2024-10-10T15:13:15.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-12-10T12:08:36.000Z (over 1 year ago)
- Last Synced: 2025-06-19T08:05:10.208Z (10 months ago)
- Topics: ios, linux, llama2, llama3, llm, macos, swift
- Language: Swift
- Homepage:
- Size: 44.9 KB
- Stars: 57
- Watchers: 6
- Forks: 20
- Open Issues: 2
-
Metadata Files:
- Readme: README.mdown
- License: LICENSE
Awesome Lists containing this project
README
# llama-cpp-swift
[](https://swiftpackageindex.com/srgtuszy/llama-cpp-swift) [](https://swiftpackageindex.com/srgtuszy/llama-cpp-swift)
Swift bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp) thanks to which you'll be able to run compatible LLM models directly on your device.
## Features
- Lightweight and easy to use
- Works on macOS and Linux
- Supports streaming via structured concurrency
- Swift 6 ready!
## TODO
- [ ] Unit tests
- [ ] Model downloads from URL and HuggingFace
## How to install
Use swift package manager:
```
.package(url: "https://github.com/srgtuszy/llama-cpp-swift", branch: "main")
```
## How to use
Here's a quick example on how to use it. For more, please refer to an example app in `example/` folder.
```swift
// Initialize model
let model = try Model(modelPath: "")
let llama = try LLama(model: model)
// Results are delivered through an `AsyncStream`
let prompt = "what is the meaning of life?"
for try await token in await llama.infer(prompt: prompt, maxTokens: 1024) {
print(token, terminator: "")
}
```