https://github.com/julienkay/com.doji.diffusers
A Unity package to run pretrained diffusion models with Unity Sentis
https://github.com/julienkay/com.doji.diffusers
ai diffusers diffusion diffusion-models generative-ai image-generation machine-learning sentis stable-diffusion text2image unity
Last synced: 10 months ago
JSON representation
A Unity package to run pretrained diffusion models with Unity Sentis
- Host: GitHub
- URL: https://github.com/julienkay/com.doji.diffusers
- Owner: julienkay
- License: mit
- Created: 2023-12-24T22:20:28.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2025-04-10T20:00:48.000Z (11 months ago)
- Last Synced: 2025-04-10T21:19:14.785Z (11 months ago)
- Topics: ai, diffusers, diffusion, diffusion-models, generative-ai, image-generation, machine-learning, sentis, stable-diffusion, text2image, unity
- Language: C#
- Homepage:
- Size: 3.08 MB
- Stars: 60
- Watchers: 7
- Forks: 8
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
> 🚧 Experimental! – This package is in an early, exploratory stage and not production-ready. Expect sharp edges, breaking changes, and general chaos.
# Diffusers
A Unity package to run pretrained diffusion models with Unity Sentis
[OpenUPM] · [Documentation (coming soon)] · [Feedback/Questions]
## About
This is essentially a port of Hugging Face’s [diffusers] library.
As of today only a limited number of pipelines and schedulers are supported (see below).
### Installation
via OpenUPM
1. In `Edit -> Project Settings -> Package Manager`, add a new scoped registry:
Name: Doji
URL: https://package.openupm.com
Scope(s): com.doji
4. In the Package Manager install `com.doji.diffusers` either by name or select it in the list under `Package Manager -> My Registries`
5. For the time being, you also have to use a custom fork of Sentis: In the package Manager -> `Package Manager -> Install package from git URL-> https://github.com/julienkay/com.unity.sentis.git`
### Features/Roadmap
Some things that might be worked on next are:
- [x] add/fix classifier-free guidance
- [ ] support more models
- [x] SD 1.5
- [x] SD 2.1
- [x] SDXL
- [x] SD-Turbo
- [x] SDXL-Turbo
- [ ] LoRA models
- [ ] more scheduler implementations
- [x] PNDM
- [x] DDIM
- [x] EulerDiscrete
- [x] EulerAncestralDiscrete
- [ ] LCM
- [ ] DDPM
- [ ] KDPM2Discrete
- [ ] KDPM2AncestralDiscrete
- [ ] more pipeline/model types
- [x] txt2img
- [x] img2img,
- [ ] inpaint,
- [ ] upscale,
- [ ] depth2img
- [ ] marigold
- [x] async pipeline methods (sliced inference)
- [ ] support multiple images per prompt
- [ ] write some documentation
### Dependencies
- [com.doji.transformers]
- [com.doji.pngcs]
[OpenUPM]: https://openupm.com/packages/com.doji.diffusers
[Documentation (coming soon)]: https://github.com/julienkay/com.doji.diffusers
[Feedback/Questions]: https://discussions.unity.com/t/stable-diffusion-diffusers-transformers-package/332701?u=julienkay
[diffusers]: https://github.com/huggingface/diffusers
[com.doji.transformers]: https://github.com/julienkay/com.doji.transformers
[com.doji.pngcs]: https://github.com/julienkay/com.doji.pngcs