Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/hfg-gmuend/zoomaker
Zoomaker - Friendly house keeping for your AI model zoo and related resources
https://github.com/hfg-gmuend/zoomaker
ai ai-art cli-tool generative-art image-generator ml model model-management package-manager stable-diffusion
Last synced: about 1 month ago
JSON representation
Zoomaker - Friendly house keeping for your AI model zoo and related resources
- Host: GitHub
- URL: https://github.com/hfg-gmuend/zoomaker
- Owner: hfg-gmuend
- Created: 2023-04-25T10:27:24.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-09-09T07:04:05.000Z (5 months ago)
- Last Synced: 2024-10-30T21:03:15.749Z (3 months ago)
- Topics: ai, ai-art, cli-tool, generative-art, image-generator, ml, model, model-management, package-manager, stable-diffusion
- Language: Python
- Homepage:
- Size: 84 KB
- Stars: 33
- Watchers: 8
- Forks: 3
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
Zoomaker - Friendly house keeping for your AI model zoo and related resources.
========
Zoomaker is a command-line tool that helps install AI models, git repositories and run scripts.
- **single source of truth**: all resources are neatly defined in the `zoo.yaml` file
- **freeze versions**: know exactly which revision of a resources is installed at any time
- **only download once**: optimize bandwidth and cache your models locally
- **optimize disk usage**: downloaded models are cached
## 😻 TL;DR
1. Install Zoomaker `pip install zoomaker`
2. Define your resources in the `zoo.yaml` file
3. Run `zoomaker install` to install them
## 📦 Installation
```bash
pip install zoomaker
```
## 🦁 zoo.yaml Examples
Example of the `zoo.yaml` of a Stable Diffusion project with the [Automatic1111](https://github.com/AUTOMATIC1111/stable-diffusion-webui) image generator:
```yaml
name: my-automatic1111-model-zoo
version: 1.0
description: Lorem ipsum
author: your name
resources:
image_generator:
- name: automatic1111
src: https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
type: git
revision: 22bcc7be428c94e9408f589966c2040187245d81
install_to: ./
models:
- name: v2-1_768-ema-pruned
src: stabilityai/stable-diffusion-2-1/v2-1_768-ema-pruned.safetensors
type: huggingface
install_to: ./stable-diffusion-webui/models/Stable-diffusion/
```
`zoo.yaml` example long
```yaml
name: my-automatic1111-model-zoo
version: 1.0
description: Lorem ipsum
author: your name
aliases:
image_generator: &image_generator ./
models: &models ./stable-diffusion-webui/models/Stable-diffusion/
controlnet: &controlnet ./stable-diffusion-webui/models/ControlNet/
embeddings: &embeddings ./stable-diffusion-webui/embeddings/
extensions: &extensions ./stable-diffusion-webui/extensions/
resources:
image_generator:
- name: automatic1111
src: https://github.com/AUTOMATIC1111/stable-diffusion-webui.git
type: git
revision: 22bcc7be428c94e9408f589966c2040187245d81
install_to: *image_generator
models:
- name: v1-5-pruned-emaonly
src: runwayml/stable-diffusion-v1-5/v1-5-pruned-emaonly.safetensors
type: huggingface
install_to: *models
controlnet:
- name: control_sd15_canny
src: lllyasviel/ControlNet/models/control_sd15_canny.pth
type: huggingface
install_to: *controlnet
embeddings:
- name: midjourney-style
src: sd-concepts-library/midjourney-style/learned_embeds.bin
type: huggingface
install_to: *embeddings
rename_to: midjourney-style.bin
- name: moebius
src: sd-concepts-library/moebius/learned_embeds.bin
type: huggingface
install_to: *embeddings
rename_to: moebius.bin
extensions:
- name: sd-webui-tunnels
src: https://github.com/Bing-su/sd-webui-tunnels.git
type: git
install_to: *extensions
```
`zoo.yaml` with script snippets
Here are a few examples of how to run scripts snippets from the `zoo.yaml` file. For example for starting the Automatic1111's webui, you could setup snippets like these and then run them with `zoomaker run start_webui`. All scripts are run from the root of the project, please adjust the paths accordingly.
```yaml
scripts:
start_webui: |
cd .\stable-diffusion-webui && call webui.bat
```
```yaml
scripts:
start_webui: |
conda activate automatic1111
cd /home/$(whoami)/stable-diffusion-webui/
./webui.sh --xformers --no-half
```
`zoo.yaml` with web download
```yaml
resources:
models:
- name: analog-diffusion-v1
src: https://civitai.com/api/download/models/1344
type: download
install_to: ./stable-diffusion-webui/models/Stable-diffusion/
rename_to: analog-diffusion-v1.safetensors
```
Please note:
The resource `type: download` can be seen as the last resort. Existing web downloads are skipped, but no other caching. It is recommended to avoid web downloads :)
## 🧮 zoo.yaml Structure
Top level:
- `name` (mandatory)
- `version`, `description`, `author`, `aliases` (optional)
- `resources` (mandatory) : `` : `[]` (array of resources)
- `scripts` (optional) : ``
Resource:
- `name`, `src`, `type`, `install_to` (mandatory)
- `rename_to` (optional)
- `revision` (optional), if none is defined the latest version from the main branch is downloaded
- `type` can either be `git`, `huggingface` or `download`
- `api_key` (optional) for civitai
## 🧞 Zoomaker Commands
All commands are run from the root of the project, where also your `zoo.yaml` file is located.
| Command | Action |
| :--------------------- | :----------------------------------------------- |
| `zoomaker install` | Installs resources as defined in `zoo.yaml` |
| `zoomaker run ` | Run CLI scripts as defined in `zoo.yaml` |
| `zoomaker --help` | Get help using the Zoomaker CLI |
| `zoomaker --version` | Show current Zoomaker version |
## 🤗 Hugging Face Access Token and Custom Cache Location
You might be asked for a [Hugging Face Access Token](https://huggingface.co/docs/hub/security-tokens) during `zoomaker install`. Some resources on Hugging Face require accepting the terms of use of the model. You can set your access token by running this command in a terminal. The command `huggingface-cli` is automatically shipped alongside zoomaker.
```bash
huggingface-cli login
```
You can specify a custom cache location by setting the HF_HOME environment variable. The default cache location is `~/.cache/huggingface/`.
```bash
export HF_HOME=/path/to/your/cache
zoomaker install
```
## 🙏 Acknowledgements
- Most of the internal heavy lifting is done be the [huggingface_hub library](https://huggingface.co/docs/huggingface_hub/guides/download) by Hugging Face. Thanks!
- "Zoomaker Safari Hacker Cat" cover image by Alia Tasler, based on this [OpenMoji](https://openmoji.org/library/emoji-1F431-200D-1F4BB/). Thanks!