https://github.com/tripplyons/clip-upscaler-and-enhancer

Using OpenAI's CLIP to upscale and enhance images
https://github.com/tripplyons/clip-upscaler-and-enhancer

Last synced: 11 months ago
JSON representation

Using OpenAI's CLIP to upscale and enhance images

• Host: GitHub
• URL: https://github.com/tripplyons/clip-upscaler-and-enhancer
• Owner: tripplyons
• License: mit
• Created: 2021-12-17T18:28:22.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2021-12-17T19:19:30.000Z (over 4 years ago)
• Last Synced: 2025-04-13T04:51:39.294Z (about 1 year ago)
• Language: Jupyter Notebook
• Size: 3.22 MB
• Stars: 6
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# CLIP Upscaler and Enhancer

Using OpenAI's CLIP to upscale and enhance images

[![GitHub Repo stars](https://img.shields.io/github/stars/tripplyons/clip-upscaler-and-enhancer?style=social)](https://github.com/tripplyons/clip-upscaler-and-enhancer)

[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tripplyons/clip-upscaler-and-enhancer/blob/main/clip-upscaler-and-enhancer.ipynb)

Based on [nshepperd's JAX CLIP Guided Diffusion v2.4](https://colab.research.google.com/drive/10YWuTxtBI7PS0xBJCLAUjhR5cB0UUXe-)

## Sample Results

Viewport

Original Image (512x512)

Using This Repo (scaled to 1024x1024 in 45 minutes)

Using Waifu2x (scaled to 1024x1024 in a few seconds)

Full Image







Zoomed In







## How it works

1. Start with any image.

2. Find CLIP's embeddings of the image.

3. Resize the image to the desired resolution and add some noise.

4. **Optimize the resized image for the original image's embeddings** (using nshepperd's JAX CLIP Guided Diffusion v2.4).