https://github.com/abdelkareemkobo/dinvo2-similar-image-

An implementation of how to use dinov2 with faiss to search for similar images in your dataset or get similar art like pineterest website :)
https://github.com/abdelkareemkobo/dinvo2-similar-image-

Last synced: 4 months ago
JSON representation

An implementation of how to use dinov2 with faiss to search for similar images in your dataset or get similar art like pineterest website :)

• Host: GitHub
• URL: https://github.com/abdelkareemkobo/dinvo2-similar-image-
• Owner: abdelkareemkobo
• License: apache-2.0
• Created: 2023-07-11T12:11:15.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2023-07-11T12:27:54.000Z (over 2 years ago)
• Last Synced: 2025-03-14T01:47:04.356Z (8 months ago)
• Language: Jupyter Notebook
• Size: 7.57 MB
• Stars: 2
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# dinvo2-similar-image

This project allows you to search for similar images in your dataset or find similar artwork like on the Pinterest website using Dinov2 with Faiss.

![:)](dino_demo.gif)

## Usage

1. Install the required dependencies:

   - `faiss-gpu`

   - `torch`

   - `torchvision`

   - `PIL`

   - `numpy`

   - `matplotlib`

   - `gradio`

2. Load the DINO model:

   - Use the Dinov2 implementation from Facebook Research.

3. Define the image transformations for preprocessing:

   - Resize the images to (224, 224) pixels.

   - Convert the images to tensors.

   - Normalize the image tensors.

4. Implement the `extract_features` function:

   - Extract features from an input image using the DINO model.

5. Use the `nearest_neighbor_search` function:

   - Perform nearest neighbor search using Faiss.

   - Retrieve the indices of the nearest neighbor images.

6. Implement the `return_nearest_neighbor_images` function:

   - Return the paths of the nearest neighbor images based on the query image.

7. Create a Gradio interface:

   - Use `gradio` to create an interface for uploading an image and displaying the nearest neighbor images.

## TODO

- [ ] Add illustrations for the concept (dinov2, instance retrieval, faiss)

- [ ] Explore possible optimizations for search and encoding

- [ ] Implement a pipeline to generate captions based on the input image and its nearest neighbor using NLP

- [ ] Upload the project to HuggingFace space as a demo

- [ ] Save Faiss state for future use

Remember to set the `dataset_path` variable to the path of your dataset before running the Gradio interface.

Enjoy finding similar images and artwork with Dinov2 and Faiss!