Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/swiftuiux/coreml-stable-diffusion-swift
swift coreml stable diffusion image generation with example in SwiftUI macos ios image to image, image generation, ios, text to image
https://github.com/swiftuiux/coreml-stable-diffusion-swift
coreml swift swiftui
Last synced: about 2 months ago
JSON representation
swift coreml stable diffusion image generation with example in SwiftUI macos ios image to image, image generation, ios, text to image
- Host: GitHub
- URL: https://github.com/swiftuiux/coreml-stable-diffusion-swift
- Owner: swiftuiux
- License: mit
- Created: 2023-03-22T08:29:52.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-11-27T15:28:23.000Z (about 2 months ago)
- Last Synced: 2024-11-28T18:41:35.283Z (about 2 months ago)
- Topics: coreml, swift, swiftui
- Language: Swift
- Homepage:
- Size: 9.73 MB
- Stars: 18
- Watchers: 4
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# CoreML stable diffusion image generation
[](https://swiftpackageindex.com/swiftuiux/coreml-stable-diffusion-swift)
The package is a mediator between Apple's Core ML Stable Diffusion implementation and your app that let you run text-to-image or image-to-image models
## [SwiftUI example](https://github.com/swiftuiux/coreml-stable-diffusion-swift-example)
## [Documentation(API)](https://swiftpackageindex.com/swiftuiux/coreml-stable-diffusion-swift/main/documentation/coreml_stable_diffusion_swift)
## How to use the package
### 1. Create GenerativeManager
```swift
let manager = GenerativeManager()
```
### 2. Run async method **generate**
```swift
let images: [CGImage?] = try await manager.generate(
with: config,
by: pipeline
)
```
### Performance
The speed can be unpredictable. Sometimes a model will suddenly run a lot slower than before. It appears as if Core ML is trying to be smart in how it schedules things, but doesn’t always optimal.
### Typical set of files for a model und the purpose of each file
| File Name | Description |
|--------------------------------------|------------------------------------------------------------------|
| `TextEncoder.mlmodelc` | Encodes input text into a vector space for further processing. |
| `Unet.mlmodelc` | Core model handling the transformation of encoded vectors into intermediate image representations. |
| `UnetChunk1.mlmodelc` | First segment of a segmented U-Net model for optimized processing in environments with memory constraints. |
| `UnetChunk2.mlmodelc` | Second segment of the segmented U-Net model, completing the tasks started by the first chunk. |
| `VAEDecoder.mlmodelc` | Decodes the latent representations into final image outputs. |
| `VAEEncoder.mlmodelc` | Compresses input image data into a latent space for reconstruction or further processing. |
| `SafetyChecker.mlmodelc` | Ensures generated content adheres to safety guidelines by checking against predefined criteria. |
| `vocab.json` | Contains the vocabulary used by the text encoder for tokenization and encoding processes. |
| `merges.txt` | Stores the merging rules for byte-pair encoding used in the text encoder. |
## Documentation(API)
- You need to have Xcode 13 installed in order to have access to Documentation Compiler (DocC)
- Go to Product > Build Documentation or **⌃⇧⌘ D**
## Used packages
- [Files](https://github.com/JohnSundell/Files)
- [Stable Diffusion](https://github.com/apple/ml-stable-diffusion)