Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/openai/consistencydecoder
Consistency Distilled Diff VAE
https://github.com/openai/consistencydecoder
Last synced: about 1 month ago
JSON representation
Consistency Distilled Diff VAE
- Host: GitHub
- URL: https://github.com/openai/consistencydecoder
- Owner: openai
- License: mit
- Created: 2023-11-02T13:06:36.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2023-11-07T11:21:38.000Z (about 1 year ago)
- Last Synced: 2024-09-27T06:21:14.029Z (about 1 month ago)
- Language: Python
- Size: 772 KB
- Stars: 2,128
- Watchers: 21
- Forks: 75
- Open Issues: 19
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project
- AiTreasureBox - openai/consistencydecoder - 11-02_2134_0](https://img.shields.io/github/stars/openai/consistencydecoder.svg)|Consistency Distilled Diff VAE| (Repos)
README
# Consistency Decoder
[[DALL·E 3]](https://openai.com/dall-e-3)
[[Improving Image Generation with Better Captions]](https://cdn.openai.com/papers/dall-e-3.pdf)
[[Consistency Models]](https://arxiv.org/abs/2303.01469)
Improved decoding for stable diffusion vaes.
## Installation
```
$ pip install git+https://github.com/openai/consistencydecoder.git
```
## Usage
```python
import torch
from diffusers import StableDiffusionPipeline
from consistencydecoder import ConsistencyDecoder, save_image, load_image
# encode with stable diffusion vae
pipe = StableDiffusionPipeline.from_pretrained(
"runwayml/stable-diffusion-v1-5", torch_dtype=torch.float16, device="cuda:0"
)
pipe.vae.cuda()
decoder_consistency = ConsistencyDecoder(device="cuda:0") # Model size: 2.49 GB
image = load_image("assets/gt1.png", size=(256, 256), center_crop=True)
latent = pipe.vae.encode(image.half().cuda()).latent_dist.mean
# decode with gan
sample_gan = pipe.vae.decode(latent).sample.detach()
save_image(sample_gan, "gan.png")
# decode with vae
sample_consistency = decoder_consistency(latent)
save_image(sample_consistency, "con.png")
```
## Examples
Original Image | GAN Decoder | Consistency Decoder |
:---:|:---:|:---:|
 |  |  |
 |  |  |
 |  |  |