{"id":24972996,"url":"https://github.com/weaviate-tutorials/demo-multimodal-search","last_synced_at":"2025-04-11T07:48:57.308Z","repository":{"id":196854311,"uuid":"696435550","full_name":"weaviate-tutorials/DEMO-multimodal-search","owner":"weaviate-tutorials","description":null,"archived":false,"fork":false,"pushed_at":"2023-12-29T20:30:49.000Z","size":5784,"stargazers_count":6,"open_issues_count":1,"forks_count":5,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-25T05:27:26.163Z","etag":null,"topics":["hacktoberfest"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/weaviate-tutorials.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-09-25T18:31:10.000Z","updated_at":"2025-03-05T08:27:06.000Z","dependencies_parsed_at":null,"dependency_job_id":"ba612395-aa7e-42fa-9591-a89a7f3bd516","html_url":"https://github.com/weaviate-tutorials/DEMO-multimodal-search","commit_stats":null,"previous_names":["weaviate-tutorials/demo-multimodal-search"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-multimodal-search","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-multimodal-search/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-multimodal-search/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/weaviate-tutorials%2FDEMO-multimodal-search/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/weaviate-tutorials","download_url":"https://codeload.github.com/weaviate-tutorials/DEMO-multimodal-search/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248359645,"owners_count":21090562,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest"],"created_at":"2025-02-03T18:11:16.627Z","updated_at":"2025-04-11T07:48:57.280Z","avatar_url":"https://github.com/weaviate-tutorials.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Multimodal (and multi-lingual) search demo\n\nThis app demonstrates Weaviate's multi-modal capabilities.\n\nIt uses a CLIP model to encode images and text into the same vector space.\n\n## Prerequisites\n\nTo run this example, you need:\n- Docker (to run Weaviate)\n- Python 3.8 or higher\n\n## Setup instructions\n\n1. Install the Python dependencies.\n    ```bash\n    pip install -r requirements.txt\n    ```\n2. Run docker-compose to spin up an Weaviate instance and the CLIP inference container.\n    ```bash\n    docker compose up -d\n    ```\n3. Create the collection definition and import data, as well as some pre-prepared queries.\n    ```bash\n    python add_data.py\n    ```\n4. Start the Streamlit app.\n    ```bash\n    streamlit run app.py\n    ```\n\n## Usage instructions\n\nInput a search query into the text box, or upload an image.\n\n![mm_demo_screen](./mm_demo_screen.png)\n\nThis will return the top 6 results from the Weaviate instance. The results are sorted by the cosine similarity between the query and the vector representation of the object.\n\nExample search results for an image query:\n![mm_demo_by_img](./mm_demo_by_img.png)\n\nExample search results for a text query:\n![mm_demo_by_text](./mm_demo_by_txt.png)\n\nNote - The model used is multi-lingual! That means it can understand queries in multiple languages. Try a search with an image, and then try inputting a description for that image in different languages!\n\n## Dataset license\n\n[Universe image](data/images/universe1.jpg) from [Unsplash](https://unsplash.com/photos/two-stars-in-the-middle-of-a-black-sky-fsH1KjbdjE8)\n\n[Forest image](data/images/forest2.jpg) from [Unsplash](https://unsplash.com/photos/green-pine-trees-d6kSvT2xZQo)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweaviate-tutorials%2Fdemo-multimodal-search","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fweaviate-tutorials%2Fdemo-multimodal-search","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fweaviate-tutorials%2Fdemo-multimodal-search/lists"}