{"id":17661827,"url":"https://github.com/18520339/finding-similar-images","last_synced_at":"2025-04-15T02:28:36.526Z","repository":{"id":117071481,"uuid":"336238052","full_name":"18520339/finding-similar-images","owner":"18520339","description":"Finding similar images from image URLs using ImageHash","archived":false,"fork":false,"pushed_at":"2021-11-13T23:41:37.000Z","size":1801,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-12T18:57:06.743Z","etag":null,"topics":["data-preparation","google-sheets-api","gspread","imagehash","similar-images"],"latest_commit_sha":null,"homepage":"https://www.youtube.com/watch?v=G3kVp-01nn8","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/18520339.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-02-05T10:26:54.000Z","updated_at":"2025-04-12T15:03:34.000Z","dependencies_parsed_at":null,"dependency_job_id":"6c2ffcbf-a7bc-473e-a3ac-fb02677ad569","html_url":"https://github.com/18520339/finding-similar-images","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/18520339%2Ffinding-similar-images","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/18520339%2Ffinding-similar-images/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/18520339%2Ffinding-similar-images/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/18520339%2Ffinding-similar-images/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/18520339","download_url":"https://codeload.github.com/18520339/finding-similar-images/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248993389,"owners_count":21195192,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-preparation","google-sheets-api","gspread","imagehash","similar-images"],"created_at":"2024-10-23T17:42:43.381Z","updated_at":"2025-04-15T02:28:36.510Z","avatar_url":"https://github.com/18520339.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Finding similar images\n\n![](https://github.com/18520339/find-similar-images/blob/main/images/image1.png?raw=true)\n\n## Introduction\n\n-   In my [Data Science project](https://github.com/18520339/vietnamese-foods), my team had to collect images through many kinds of **Search Engines** for creating dataset and we chose **Google Sheets** for assigning labeling tasks to each member because of its convenient.\n\n-   There are lots of similar images when crawling from the Internet, this will result in biases in the dataset. Here is my solution to filter similar images for the **Data Preparation** step.\n\n## Implementation\n\n1. Get image urls from **Search Engines**. I have a repo for that [here](https://github.com/18520339/image-search-downloader)\n2. Copy + paste these urls to **Google Sheets**. Here, we can see how similar images arranged next to each other\n3. Connect to **Google Sheets** using **Python**\n4. If just using 1 hash value, some images will be said to be the same even if they are different. Therefore, we decided to caculate 3 hash values for each 2 images:\n\n    - Average hashing ([ahash](http://www.hackerfactor.com/blog/index.php?/archives/432-Looks-Like-It.html))\n    - Perceptual hashing ([phash](http://www.hackerfactor.com/blog/index.php?/archives/432-Looks-Like-It.html))\n    - Difference hashing ([dhash](http://www.hackerfactor.com/blog/index.php?/archives/529-Kind-of-Like-That.html))\n\n![](https://github.com/18520339/find-similar-images/blob/main/images/image2.png?raw=true)\n\n5. If the distances of 2 in these 3 values tell 2 images are similar (\u0026le; **different points**) then arrange these images next to each other\n\n    ```python\n    distances = [ahash0 - ahash1, phash0 - phash1, dhash0 - dhash1]\n    diff_results = sum(dist \u003c args['diff'] for dist in distances)\n\n    if diff_results \u003e= 2:\n        print(f'|--Similar with url {idx1 + 1}: {url1}')\n    ```\n\n6. Decide what images to keep and begin labeling\n\n![](https://github.com/18520339/find-similar-images/blob/main/images/image3.png?raw=true)\n\n## Usage\n\n1. Install libraries: `pip install -r requirements.txt`\n\n2. Sort similar images in **Google Sheets**:\n\n-   Example: `python sort_similar.py -s \"example\" -w \"Sheet1\" -r \"B2:C\" -a credentials.json`\n\n```\nusage: sort_similar.py [-h] -s SPREADSHEET -w WORKSHEET -r RANGE -a AUTH [-d DIFF]\n\noptional arguments:\n-h, --help                                    show this help message and exit\n-s SPREADSHEET, --spreadsheet SPREADSHEET     spreadsheet name\n-w WORKSHEET, --worksheet WORKSHEET           worksheet name\n-r RANGE, --range RANGE                       updated range\n-a AUTH, --auth AUTH                          credentials file\n-d DIFF, --diff DIFF                          different points\n```\n\n3. Download images from urls in **Google Sheets**:\n\n-   Example: `python download_images.py -s \"example\" -w \"Sheet1\" -r \"B2:C\" -a credentials.json -o images/`\n\n```\nusage: download_images.py [-h] -s SPREADSHEET -w WORKSHEET -r RANGE -a AUTH -o OUT\n\noptional arguments:\n-h, --help                                    show this help message and exit\n-s SPREADSHEET, --spreadsheet SPREADSHEET     spreadsheet name\n-w WORKSHEET, --worksheet WORKSHEET           worksheet name\n-r RANGE, --range RANGE                       updated range\n-a AUTH, --auth AUTH                          credentials file\n-o OUT, --out OUT                             path to images directory\n```\n\n## Reference\n\n-   [How to determine whether 2 images are equal or not with the perceptual hash in Python](https://ourcodeworld.com/articles/read/1006/how-to-determine-whether-2-images-are-equal-or-not-with-the-perceptual-hash-in-python)\n-   [A Python Perceptual Image Hashing Module](https://github.com/JohannesBuchner/imagehash)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F18520339%2Ffinding-similar-images","html_url":"https://awesome.ecosyste.ms/projects/github.com%2F18520339%2Ffinding-similar-images","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2F18520339%2Ffinding-similar-images/lists"}