https://github.com/seanvelasco/memegraph

Meme search engine and recommendation system using CLIP-based neural nets
https://github.com/seanvelasco/memegraph

catlip clip corenet memes mlx openai search-algorithm siglip transformer

Last synced: about 2 months ago
JSON representation

Meme search engine and recommendation system using CLIP-based neural nets

• Host: GitHub
• URL: https://github.com/seanvelasco/memegraph
• Owner: seanvelasco
• Created: 2024-04-21T16:01:22.000Z (about 1 year ago)
• Default Branch: main
• Last Pushed: 2024-05-17T20:28:25.000Z (12 months ago)
• Last Synced: 2024-05-17T21:24:37.565Z (12 months ago)
• Topics: catlip, clip, corenet, memes, mlx, openai, search-algorithm, siglip, transformer
• Language: Python
• Homepage: https://memegraph.sean.app/
• Size: 2.01 MB
• Stars: 1
• Watchers: 2
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

        


# Memegraph

Memegraph is a search engine and recommendation system for memes using OpenAI CLIP and Salesforce BLIP.

Search memes in natural langauge, explore memes with the same template, and discover memes that are visually similar.

### Don't know the name of a meme? Just describe the meme!

![Guy Thinking In Bed meme](https://img.sean.app/memegraph_thinking_in_bed_meme.png)

### Discover memes about your favorite topics

![The Office memes](https://img.sean.app/memegraph_the_office.png)

### Looking for THAT obscure meme? Look it up!

![Memes about Spongebob and Myers-Briggs Type Indicator (MBTI)](https://img.sean.app/memegraph_as_mbti.png)

### Discover similar memes

![Boardroom meme](https://img.sean.app/memegraph_boardroom.png)

### New way to browse memes

![Memegraph home](https://img.sean.app/memegraph_home.png)

## Development

> [!IMPORTANT]  

> This app uses MLX to run CLIP. To generate embeddings, a device with an Apple Silicon processor is needed. For non-Apple Silicon devices, the app can still run, provided there are already images and embeddings in the database, but it will not be able to search for memes by image similarity.

Install dependencies

```bash

pip install -r requirements.txt

```

Modify scripts in `/scripts` to point to your images directory, blob storage, and Postgres database.

Generate image embeddings and generate image captions

```bash

python main.py

```

Run the app in development

```bash

python app.py

```

Run the app in production

```bash

waitress-serve app:app

```

## Data

All memes were retrieved from the top 1000 posts of all time on [Reddit's meme subreddits](https://www.reddit.com/t/memes) using [PRAW](https://praw.readthedocs.io/en/stable/) on the Reddit's API free tier.

### Notes from the author

Python is not my primary language and I'm new to machine learning, so I greatly appreciate and welcome constructive feedback!