Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/matteofasulo/weaviate-search-engine
Vector database (Weaviate) to build an image search engine powered by a deep neural network
https://github.com/matteofasulo/weaviate-search-engine
deep-neural-networks docker docker-compose python streamlit weaviate
Last synced: about 2 hours ago
JSON representation
Vector database (Weaviate) to build an image search engine powered by a deep neural network
- Host: GitHub
- URL: https://github.com/matteofasulo/weaviate-search-engine
- Owner: MatteoFasulo
- License: apache-2.0
- Created: 2023-04-11T09:51:07.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2024-03-18T01:50:41.000Z (8 months ago)
- Last Synced: 2024-03-18T02:50:08.439Z (8 months ago)
- Topics: deep-neural-networks, docker, docker-compose, python, streamlit, weaviate
- Language: Python
- Homepage:
- Size: 686 KB
- Stars: 6
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
# Weaviate Image Search Engine
Vector database (Weaviate) to build an image search engine powered by a deep neural network. It uses Weaviate optimizer for images with **ResNet50 (PyTorch)** as vectorizer and retriever. Currently only PyTorch ResNet50 supports **CUDA-GPU** but in a **single-threaded** manner. You can also use the **keras-based ResNet50 CPU** with **multi-threaded** inference.
## Steps to run it
1. Install Docker (skip this if already installed)
```sh
curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
```
2. Spin up docker compose file:
```sh
docker compose up -d
```
3. Activate the environment:
```sh
pip3 install pipenv && pipenv install && pipenv shell
```
4. Launch `start.sh` script to create Weaviate schema, convert images from `img` folder to base64 and upload them in Weaviate:
```sh
./start.sh
```
5. Navigate to URL that streamlit outputs and upload an image. The program will query Weaviate for a vectorized search of similar images. If you are running this in local environment then just navigate at:
```
http://127.0.0.1:8501/
```
6. (Optional) CORS and XsrfProtection Streamlit. Run this if you have **Socket Errors** while connecting to streamlit app URL:
```
streamlit run streamlit_app.py --server.enableCORS false --server.enableXsrfProtection false
```
## Images
Feel free to insert your images in `img` folder and see if the database suggests you similar images.
## Weaviate Schema
This is the schema implemented in Weaviate. You can modify the class name, change the vectorizer or insert more properties.
```python
schemaConfig = {
'class': 'MyImages', # class name for schema config in Weaviate (change it with a custom name for your images)
'vectorizer': 'img2vec-neural',
'vectorIndexType': 'hnsw',
'moduleConfig': {
'img2vec-neural': {
'imageFields': [
'image'
]
}
},
'properties': [
{
'name': 'image',
'dataType': ['blob']
},
{
'name': 'text',
'dataType': ['string']
}
]
}
```