{"id":24972995,"url":"https://github.com/weaviate-tutorials/demo-image-search-dishes","last_synced_at":"2025-04-17T14:15:31.849Z","repository":{"id":205047084,"uuid":"711950867","full_name":"weaviate-tutorials/DEMO-image-search-dishes","owner":"weaviate-tutorials","description":null,"archived":false,"fork":false,"pushed_at":"2024-03-28T08:40:58.000Z","size":336455,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-29T06:11:24.453Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/weaviate-tutorials.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-30T13:57:05.000Z","updated_at":"2024-03-17T08:39:55.000Z","dependencies_parsed_at":null,"dependency_job_id":"0f2dabfc-d83b-4c99-87ca-ae6c82cbe60b","html_url":"https://github.com/weaviate-tutorials/DEMO-image-search-dishes","commit_stats":null,"previous_names":["weaviate-tutorials/demo-image-search-dishes"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-image-search-dishes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-image-search-dishes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-image-search-dishes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-image-search-dishes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/weaviate-tutorials","download_url":"https://codeload.github.com/weaviate-tutorials/DEMO-image-search-dishes/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249347331,"owners_count":21255148,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2025-02-03T18:11:14.620Z","updated_at":"2025-04-17T14:15:31.829Z","avatar_url":"https://github.com/weaviate-tutorials.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Similar images search\n\n## Overview\n\nThe project aims to highlight the diverse search capabilities of Weaviate, empowered by the CLIP model. \nIt demonstrates the potential to create robust AI applications capable of multilingual understanding \nand visual perception with just a few lines of code.\n\nIn particular, we will index a collection of random pictures featuring various foods from around the world.  \nSubsequently, we'll be able to search through them using three different inputs:\n1. User-provided text\n1. Selected image from the indexed collection\n1. Any uploaded image\n\nThese scenarios in Weaviate terms correspond to the following operators:\n1. [nearText](https://weaviate.io/developers/weaviate/api/graphql/search-operators#neartext)\n2. [nearObject](https://weaviate.io/developers/weaviate/api/graphql/search-operators#nearobject)\n3. [nearImage](https://weaviate.io/developers/weaviate/search/image)\n\n### CLIP model\nCLIP, or Contrastive Language-Image Pre-training, is a multimodal deep learning model \n[by OpenAI](https://openai.com/research/clip) \nthat is designed to understand and generate meaningful representations of images and text, \nallowing it to perform tasks that involve both modalities. \n\nCLIP is trained to learn a joint embedding space \nwhere images and text representations are aligned. \nThis means that similar concepts in images and text are close to each other in the embedding space.\nIn this demo we will use a multilingual CLIP model\n\n## Technology stack\n- Python\n- Weaviate\n- Streamlit\n- Docker\n\n### Used Weaviate modules/models\n\n[multi2vec-clip vectorizer](https://weaviate.io/developers/weaviate/modules/retriever-vectorizer-modules/multi2vec-clip)  \nThe multi2vec-clip module enables Weaviate to obtain vectors locally \nfrom text or images using a Sentence-BERT CLIP model.\n\nTo be able to use it you need to enable it in the [docker compose file](docker-compose.yml)\n\n[sentence-transformers/clip-ViT-B-32-multilingual-v1](https://huggingface.co/sentence-transformers/clip-ViT-B-32-multilingual-v1)\nThe particular model that we'll use is `sentence-transformers/clip-ViT-B-32-multilingual-v1` model. It supports \nencoding of text in 50+ languages. The model is based on Multilingual Knowledge Distillation, which uses the original \nclip-ViT-B-32 model as the teacher and trains a multilingual DistilBERT model as the student. As mentioned above, the \nmodel can map text and images to a common vector space such that the distance between the \ntwo represents their semantic similarity. \n\n## Prerequisites\n1. Python3 interpreter installed\n1. Ability to execute docker compose \n(The most straightforward way to do it on Windows/Mac is to install \n[Docker Desktop](https://www.docker.com/products/docker-desktop/))\n\n## Setup instructions \n\n### Start up\n1. Clone this repository\n1. Download the dataset (you need to be logged in to Kaggle to be able to do it)\n[from this link](https://www.kaggle.com/datasets/abhijeetbhilare/world-cuisines/download?datasetVersionNumber=3) \nand unzip it to the project root\n\n1. Create a virtual environment and activate it\n   \u003e **Note**  \n   \u003e This was tested using python 3.10\n\n    ```shell\n    python3 -m venv venv\n    source venv/bin/activate\n    ```\n1. Install all required dependencies \n    ```shell\n    pip install -r requirements.txt\n    ```\n1. Run containerized instance of Weaviate. It also includes vectorizer module to compute the embeddings.\n\n   \u003e **Note**  \n   \u003e Make sure you don't have anything occupying port 8080   \n   \u003e If you do, you have the option to either stop that process or change the port that Weaviate is using.\n    ```shell\n    docker compose up\n    ```\n1. Index the dataset in Weaviate. By default, 1000 pictures will be ingested\n    ```shell\n    python add_data.py\n    ```\n   If you want to have a bigger dataset you can use `--image-number` parameter to set the number of pictures to ingest:\n   ```shell\n   python add_data.py --image-number 3000\n   ```\n1. Run the Streamlit demo\n   ```shell\n   streamlit run app.py\n   ```\n   Now you can open the app on http://localhost:8501/ and also play with changing [app.py](app.py) on the fly\n\n### Shut down\n1. Both streamlit app and docker compose can be stopped with `Ctrl+C` in the corresponding terminal window\n2. To remove created docker containers and volumes use\n```shell\ndocker compose down -v\n```\n\n## Usage instructions\n[![](https://markdown-videos-api.jorgenkh.no/youtube/lxZegTVduqQ)](https://youtu.be/lxZegTVduqQ)\n\n## Dataset license\n\nThe dataset used for this example is available on Kaggle: \nhttps://www.kaggle.com/datasets/abhijeetbhilare/world-cuisines/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweaviate-tutorials%2Fdemo-image-search-dishes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweaviate-tutorials%2Fdemo-image-search-dishes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweaviate-tutorials%2Fdemo-image-search-dishes/lists"}