https://github.com/martin-rizzo/TinyModelsForLatentConversion
Command-line tool to build VAEs and Transcoders (from Tiny AutoEncoders).
https://github.com/martin-rizzo/TinyModelsForLatentConversion
auto1111 comfyui flux stable-diffusion text2image transcoder txt2img vae
Last synced: about 2 months ago
JSON representation
Command-line tool to build VAEs and Transcoders (from Tiny AutoEncoders).
- Host: GitHub
- URL: https://github.com/martin-rizzo/TinyModelsForLatentConversion
- Owner: martin-rizzo
- License: mit
- Created: 2024-11-26T15:31:10.000Z (11 months ago)
- Default Branch: master
- Last Pushed: 2024-12-13T20:49:32.000Z (10 months ago)
- Last Synced: 2024-12-13T21:27:32.014Z (10 months ago)
- Topics: auto1111, comfyui, flux, stable-diffusion, text2image, transcoder, txt2img, vae
- Language: Python
- Homepage:
- Size: 44.9 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Tiny Models for Latent Conversion
**Tiny Models for Latent Conversion** offers a set of command-line utilities designed to generate three distinct types of models: ultra-fast ComfyUI-compatible Variational Autoencoders (VAEs); latent space Transcoders for seamless conversion between different latent spaces (e.g., SDXL to SD); and a custom `.safetensors` file specifically tailored for a project currently under development.
All generated models are based on the architecture and weights of the "Tiny AutoEncoder" models.
## What are Tiny AutoEncoders?
Tiny AutoEncoders (TAEs) are highly optimized autoencoders that share the same latent space as Stable Diffusion and Flux VAEs. This enables significantly faster and more resource-efficient image encoding and decoding. These models were developed and trained by Ollin Boer Bohan, to whom I extend my sincere gratitude. You can find Ollin's original implementation and pre-trained models in the [Tiny AutoEncoder Repository](https://github.com/madebyollin/taesd).
## Command-Line Tools
This project provides the following command-line conversion tools:
- `build_tiny_vae.py`: Generates a comfyui-compatible VAE model from a Tiny AutoEncoder.
- `build_tiny_transcoder.py`: Creates a transcoder enabling latent space conversion between different models (e.g., SDXL to SD).
- `build_auxiliary_model.py`: Generates a custom `.safetensors` model for an ongoing project.
## Installation and Usage
1. **Clone the Repository:**
First, you need to clone this repository to your local machine.
```bash
git clone https://github.com/martin-rizzo/TinyModelsForLatentConversion.git
cd TinyModelsForLatentConversion
```
2. **Download Original Models:**
Download the necessary TAESD models from Hugging Face to the `original_taesd_models` directory:
```bash
./download_original_models.sh
```
For more information on what TAESD models are needed and how to download them manually, refer to the [original_taesd_models/README.md](original_taesd_models/README.md) documentation.
3. **Create the Virtual Environment and Install Dependencies:**
The `build_tiny_vae.sh` script came with a virtual environment setup argument, so you don't need to create a virtual environment manually.
```bash
./build_tiny_vae.sh --create-venv
```
4. **Run the Bash Script:**
Execute the `makeall.sh` script to generate all VAEs and transcoders automatically:
```bash
./makeall.sh
```
## Output
When running the general `makeall.sh` script, it will generate several files in the `output` directory, including VAE models and transcoders for different latent spaces. The generated files will have names like:
- `tiny_vae_*.safetensors`: VAE models compatible with ComfyUI.
- `transcoder_from_*_to_*.safetensors`: Transcoder model files.
- `auxiliary_model.safetensors`: Custom auxiliary model for a specific project.
## License
**Copyright (c) 2024-2025 Martin Rizzo**
This project is licensed under the MIT license.
Details can be found in the ["LICENSE"](LICENSE) file.
## Disclaimer
This tool is provided "as is" without any warranty. Use at your own risk.