{"id":13410397,"url":"https://github.com/adriacabeza/go-imagecleaner","last_synced_at":"2025-04-30T23:33:24.848Z","repository":{"id":122838098,"uuid":"397360699","full_name":"adriacabeza/go-imagecleaner","owner":"adriacabeza","description":"📸 Clean your image folder using perceptual hashing and BK-trees using Go!","archived":false,"fork":false,"pushed_at":"2021-08-18T08:40:11.000Z","size":996,"stargazers_count":11,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-07-31T20:42:35.155Z","etag":null,"topics":["bk-tree","clustering","golang","image-processing","perceptual-hashing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/adriacabeza.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-08-17T18:51:24.000Z","updated_at":"2024-04-26T13:59:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"323ffb37-1c9d-4b15-aa9f-be9878f11c81","html_url":"https://github.com/adriacabeza/go-imagecleaner","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adriacabeza%2Fgo-imagecleaner","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adriacabeza%2Fgo-imagecleaner/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adriacabeza%2Fgo-imagecleaner/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/adriacabeza%2Fgo-imagecleaner/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/adriacabeza","download_url":"https://codeload.github.com/adriacabeza/go-imagecleaner/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224227229,"owners_count":17276760,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bk-tree","clustering","golang","image-processing","perceptual-hashing"],"created_at":"2024-07-30T20:01:06.636Z","updated_at":"2024-11-12T06:22:40.475Z","avatar_url":"https://github.com/adriacabeza.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"\u003ch1 align=\"center\"\u003e Image Cleaner\n  🏞🏞 ➡ 🏞 \u003c/h1\u003e\n\n[![made-with-Go](https://img.shields.io/badge/Made%20with-Go-1f425f.svg)](http://golang.org) [![Go](https://github.com/adriacabeza/go-imagecleaner/actions/workflows/go.yml/badge.svg)](https://github.com/adriacabeza/go-imagecleaner/actions/workflows/go.yml)\n\nThis tool can take your image gallery and create a new folder with image-alike-cluster folders. It uses a perceptual image hashing algorithm and a custom threshold to cluster them. An improvement could be to add deep-learning to the scene and cluster the images based on features. To cluster the images efficently, it uses a BK-trees since checking duplicates can turn into a O(N^2) problem pretty easily.\n\n\u003cdiv align=\"center\"\u003e\n  \u003ch3\u003e\n      Before\n  \u003c/h3\u003e\n  \u003cbr\u003e\n\u003cimg src=\"./docs/before.png\" width=\"80%\"\u003e\n  \u003cbr\u003e\n  \u003ch3\u003e\n    After\n  \u003c/h3\u003e\n  \u003cbr\u003e\n\u003cimg src=\"./docs/after.png\" width=\"80%\"\u003e\n\u003c/div\u003e\n\n\nImage Cleaner was created upon a friend request. After a friends-trip, he had several pictures that looked alike (from different smartphones) and he wanted to select the best ones. He tried to use [fdupes](https://github.com/adrianlopezroche/fdupes) to start removing the exact duplicates but it didn't even work since some of the images were sent using Google Photos, Whatsapp, etc (different compression algorithms and sizes). After a quick search I found some python examples like: [duplicate images](https://github.com/philipbl/duplicate-images) or [Fast Near Duplicate image search](https://github.com/umbertogriffo/fast-near-duplicate-image-search) but I did not find anything similar written in Go so here it is. \n\n\u003e As a note, this is my first piece of code written in Go so it probably won't be as good as I'd like to. Any comment or improvement will be gladly received :D\n\n## Installation\n\nTo start using Image Cleaner, install Go and run ``go get``:\n\n```shell\ngo get -u github.com/adriacabeza/go-imagecleaner\n```\n\nThis will retrieve the library.\n\nMoreover, if you prefer it, you can use the [binary released version](https://github.com/adriacabeza/go-imagecleaner/releases/tag/v0.1-alpha).\n\n\n## Usage\n\n```shell\nimagecleaner -imagesPath=IMAGE_PATH -threshold=THRESHOLD\n```\nIf you do not specify any value for *threshold* it will use its default value = 10. \n\nThis will create a folder called **clusters** with each image structured into cluster folders. **Note that the code only copies images, it does not remove them.**\n\n**Example**:\n\n```shell\n$ go run main.go ./cluster_utils.go ./image_utils.go -imagesPath=/Users/adria/Downloads/Photos\nStarting to cluster your images from /Users/adria/Downloads/Photos\nSelected 6 images\n 6 / 6 [=================================================================================] 100.00% 2s\nImages hashed and BK-tree created\nCreating clusters\n 6 / 6 [=================================================================================] 100.00% 1s\nFound 3 clusters in 6 images\nClusters created\n 3 / 3 [=================================================================================] 100.00% 0s\nDone\n```\n\n\u003e note that all the clusters that are size 1 (just one image) are merged into a big folder of unique images\n\n\n\n## TODO\n\n- [ ] Try another hash functions\n- [ ] Add some testing\n- [x] Create binary\n\n\n### Credits\n\n- [Duplicate image detection](https://benhoyt.com/writings/duplicate-image-detection/): the idea of this module is mainly based on this very cool blogpost.\n- [Go image Hash Library](https://github.com/corona10/goimagehash): the image hash algorithm was taken from this module.\n- [Go BK-trees Library](https://github.com/agatan/bktree): the BK-tree structure implementation was taken from this library.  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadriacabeza%2Fgo-imagecleaner","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fadriacabeza%2Fgo-imagecleaner","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fadriacabeza%2Fgo-imagecleaner/lists"}