https://github.com/olaviinha/neuraltexttoimage
Colabs for text prompt steered image generators
https://github.com/olaviinha/neuraltexttoimage
ai-image big-sleep biggan colab dall-e dalle2 google-colab image-generation laion neural-network neural-networks openai rudalle stable-diffusion text2image txt2img vqgan
Last synced: 3 months ago
JSON representation
Colabs for text prompt steered image generators
- Host: GitHub
- URL: https://github.com/olaviinha/neuraltexttoimage
- Owner: olaviinha
- Created: 2021-07-19T12:03:20.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2024-11-01T20:49:02.000Z (6 months ago)
- Last Synced: 2024-11-01T21:25:26.533Z (6 months ago)
- Topics: ai-image, big-sleep, biggan, colab, dall-e, dalle2, google-colab, image-generation, laion, neural-network, neural-networks, openai, rudalle, stable-diffusion, text2image, txt2img, vqgan
- Language: Jupyter Notebook
- Homepage:
- Size: 261 KB
- Stars: 13
- Watchers: 1
- Forks: 4
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Colab notebooks for text-to-image generators
**❗️ This repository is not maintained since 2022, as closed-source state-of-the-art text-to-image solutions are now widely available for everyone.**
User-friendly Colab notebooks for various text prompt steered synthetic image generators. These notebooks are not actively maintained, though issues and pull requests are welcome.
**Available notebooks (latest to oldest):**
- [Stable Diffusion](#stable-diffusion-by-compvis-stability-ai-and-laion) (v1.4, v1.5, waifu diffusion)
- [DALL·E Mini (craiyon.ai)](#dalle-mini-craiyonai-by-boris-dayma)
- [Latent Diffusion LAION 400M](#latent-diffusion-laion-400m-by-compvis)
- [ruDALLE Malevich](#rudalle-malevich-by-sberbank-ai)
- [JAX CLIP Guided Diffusion](#jax-clip-guided-diffusion-by-nshepperd)
- [CLIP+VQGAN](#clipvqgan-by-katherine-crowson)
- [Latent Vision](#latent-vision-by-ryan-murdock)
- [Big Sleep](#big-sleep-by-phil-wang-more-precisely-this-fork)
---
These techniques are all more or less descendants of Big Sleep (2021), the original concept of turning text into imagery by using [CLIP](https://github.com/openai/CLIP) (OpenAI) to steer [BigGAN](https://arxiv.org/abs/1809.11096) (Andrew Brock et al.), as introduced by [Ryan Murdock](https://github.com/rynmurdock) in his [original notebook](https://colab.research.google.com/drive/1NCceX2mbiKOSlAd_o7IU7nA9UskKN5WR?usp=sharing). The technique has since branched and evolved into what we now know as Midjourney, Dall-E, Stable Diffusion, etc.
The point of this repository is to record the development history of image-generation AIs, and as long as they work, user-friendly and quick-to-run UIs for the different implementations. Some additional basic features have been added to some of the notebooks, such as saving all/selected progression images, saving progression video, batch processing, etc.
I do not use these notebooks on a regular basis, and hence I'm mostly unaware when some library gets updated and a notebook may no longer function; Please do raise an issue should any problems occur.
---
## Stable Diffusion by [CompVis](https://github.com/CompVis), [Stability AI](https://stability.ai) and [LAION](https://laion.ai)
This colab requires registering with [Hugging Face](https://huggingface.co) for login token and accepting model card terms to download required models [here](https://huggingface.co/CompVis/stable-diffusion-v1-4) and [here](https://huggingface.co/runwayml/stable-diffusion-v1-5).
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/Stable_Diffusion.ipynb?force_theme=dark)
_"Satanism in the 1900s"_
---
## DALL·E Mini (craiyon.ai) by [Boris Dayma](https://github.com/borisdayma)
This colab requires registering with [wandb](https://wandb.ai/) for an authentication key to download required models.
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/dalle_mini.ipynb?force_theme=dark)
_"Satanism in the 1900s"_
---
## Latent Diffusion LAION 400M by [CompVis](https://github.com/CompVis)
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/LAION_400M.ipynb?force_theme=dark)
_"Satanism in the 1900s"_ (upscaled)
---
## ruDALLE Malevich by [Sberbank-AI](https://github.com/sberbank-ai)
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/ruDalle.ipynb?force_theme=dark)
_"Satanism in the 1900s"_
---
## JAX CLIP Guided Diffusion by [nshepperd](https://github.com/nshepperd)
Method originally by [Katherine Crowson](https://github.com/crowsonkb)
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/JAX_CLIP_Guided_diffusion_nshepperd.ipynb?force_theme=dark)
300 iterations of _"Satanism in the 1900s"_.
---
## CLIP+VQGAN by [Katherine Crowson](https://github.com/crowsonkb)
**Papers:** 1) [VQGAN by Patrick Esser et al](https://arxiv.org/abs/2012.09841), 2) [CLIP by OpenAI](https://arxiv.org/abs/2103.00020)
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/CLIP%2BVQGAN_crowsonkb.ipynb?force_theme=dark)
400 iterations of _"Satanism in the 1900s"_.
---
## Latent Vision by [Ryan Murdock](https://github.com/rynmurdock)
**Papers:** 1) [VQGAN by Patrick Esser et al](https://arxiv.org/abs/2012.09841), 2) [CLIP by OpenAI](https://arxiv.org/abs/2103.00020)
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/LatentVision_rynmurdock.ipynb?force_theme=dark)
400 iterations of _"Satanism in the 1900s"_.
---
## Big Sleep by [Phil Wang](https://github.com/lucidrains) (more precisely [this fork](https://github.com/olaviinha/big-sleep))
**Papers:** 1) [BigGAN by Andrew Brock et al](https://arxiv.org/abs/1809.11096), 2) [CLIP by OpenAI](https://arxiv.org/abs/2103.00020)
Original Big Sleep method.
[](https://colab.research.google.com/github/olaviinha/NeuralTextToImage/blob/main/BigSleep_lucidrains.ipynb?force_theme=dark)
400 iterations of _"Satanism in the 1900s"_.