Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/cloudmedicio/describe-media-library
Describe images in the WordPress Media Library using a local large language model. Generates titles, captions, descriptions, and alt tags.
https://github.com/cloudmedicio/describe-media-library
accessibility alt-text alt-text-generator apple-silicon computer-vision gpt4 intel-mac linux llama2 llamacpp llava localwp media-library ollama openai plugin search-engine-optimization seo wordpress wp-cli
Last synced: 9 days ago
JSON representation
Describe images in the WordPress Media Library using a local large language model. Generates titles, captions, descriptions, and alt tags.
- Host: GitHub
- URL: https://github.com/cloudmedicio/describe-media-library
- Owner: cloudmedicio
- Created: 2024-01-29T12:24:28.000Z (10 months ago)
- Default Branch: production
- Last Pushed: 2024-01-29T15:16:33.000Z (10 months ago)
- Last Synced: 2024-10-18T22:11:13.126Z (26 days ago)
- Topics: accessibility, alt-text, alt-text-generator, apple-silicon, computer-vision, gpt4, intel-mac, linux, llama2, llamacpp, llava, localwp, media-library, ollama, openai, plugin, search-engine-optimization, seo, wordpress, wp-cli
- Language: PHP
- Homepage:
- Size: 8.79 KB
- Stars: 3
- Watchers: 1
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
## Purpose
Creates image descriptions, captions, titles, and alt tags for image attachments in WordPress using a local large language model.
No data is sent to external services. All data is processed on the local machine.
In a WordPress context, types of text associated with images have these uses:
- **Alt**: Displays on images as an image is loading. Relevant for accessibility, for example, screen readers, and search engines.
- **Title**: Displays as a tooltip on mouse hover. May also be read by screen readers depending on user settings.
- **Description**: Displays on image attachment page in WordPress themes. Also useful for search.
- **Caption**: Displays below the image in the page layout.
## Requirements
This plugin assumes WordPress is running on a [local installation](https://localwp.com), [Ollama.ai](https://ollama.ai) is installed, and you have access to `wp-cli`.
Ollama currently supports macOS and Linux, with a Windows version in progress. Testing was conducted on a 2012 Intel Mac mini and a 2022 M1 Mac mini. Intel processors will process images using the CPU, while Apple Silicon chips will make use of the Neural Engine, running much faster without using the main processor resources.
#### Install Ollama
Graphically: Download the app from [Ollama.ai](https://ollama.ai). Run the app, and click the button at launch to install command line tools.
With [Homebrew](https://brew.sh):
```bash
brew install ollama
```
#### Use Ollama
**Download a vision model**
```bash
ollama pull llava
```
**Run the Language Model server**
Either open the Ollama app, which will appear in the menu bar, or run in Terminal:
```bash
ollama serve
```
#### Install `wp-cli`
`wp-cli` comes bundled with [Local WP](https://localwp.com). It can also be installed with Homebrew:
```bash
brew install wp-cli
```
#### Install this plugin
Option 1: Add this plugin folder to `wp-content/plugins` and activate it.
Option 2: Place the main PHP file in `wp-content/mu-plugins/describe-media-library.php`
## Generate Image Descriptions
Once Ollama is running, image alt tags, descriptions, captions, and titles can be generated by running the plugin's command within a directory located in the WordPress installation:
```bash
wp _medic describe-images
```
...this will output descriptions to the terminal and save a CSV to `wp-content/uploads/image-descriptions.csv`.
Once there are descriptions in the CSV, the file may be reviewed or revised, then written to the WordPress database with:
```bash
wp _medic describe-images --write-to-db
```
Uploading the plugin and CSV to a live site and running the above command provides a way to update images descriptions on a production installation without migrating a database.
## Modify Description Guidelines
When generating, these are the default inputs:
```bash
wp _medic describe-images \
--alt="Please write a thorough description of the image in a format appropriate for an alt tag focused on accessibility." \
--description="Please write a comma-separated list of keywords and relevant synonyms related to this image, focusing on relevancy for search." \
--caption="Please write a descriptive caption for this image appropriate for displaying to a user reading an article where the image is referenced." \
--title="Please write a short title for this image."
```
These can be overriden to anything else:
```bash
wp _medic describe-images \
--alt="Describe the image for an accessible alt tag in French." \
--description="Write a comma-separated list of keywords and topics related to this image." \
--caption="Write a caption for this image in the form of a Haiku." \
--title="Describe this image using only emoji."
```
Or leave the argument blank to skip it.
e.g., to write only alt tags:
```bash
wp _medic describe-images \
--alt="Please write a thorough description of the image in a format appropriate for an alt tag focused on accessibility." \
--description \
--caption \
--title
```
## Image Size
The default image size passed to the model is `large`. Any defined WordPress image size may be used, such as `thumbnail`, `medium`, `large`, or `full`. In testing on a Quad-Core Intel i7 Mac mini using CPU, `large` said it would take around 24 days to generate all 4 description types for `1,600 images`. However, switching to `medium` produced descriptions which seemed fabricated, so the default is `large`.
This will go slower, but is the default due to accuracy:
```bash
wp _medic describe-images --image_size="large"
```
This will go faster, but accuracy may degrade:
```bash
wp _medic describe-images --image_size="medium"
```
If doing this at scale, one would ideally have access to hardware with Apple Silicon, NVIDIA GPUs, or Cloud TPUs. Here, we make use of older hardware by designing systems which distribute tasks across many machines.
## Collaborate with 🌦️️🩺️ Cloud Medic
> Do you have a project involving artificial intelligence, data engineering, or medicine you would like to bring to life?
Connect with us at [[email protected]](mailto:[email protected]?subject=Collaboration&body=Hi%21%20We%20saw%20your%20image%20description%20project%20on%20GitHub%2C%20and%20thought%20it%20might%20be%20related%20to%20something%20we%27re%20working%20on.%20Have%20time%20to%20connect%3F%20Here%20is%20a%20little%20bit%20about%20what%20we%27re%20thinking%3A)
Please use subject `Collaboration` and share short description of how we can help bring your ideas to reality.