{"id":21625475,"url":"https://github.com/horgh/dupefile","last_synced_at":"2025-03-18T19:56:03.164Z","repository":{"id":137587428,"uuid":"91909168","full_name":"horgh/dupefile","owner":"horgh","description":"Detect and optionally delete duplicate files in a directory tree","archived":false,"fork":false,"pushed_at":"2021-06-06T15:34:39.000Z","size":22,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-01-24T22:34:47.225Z","etag":null,"topics":["dedup","file","file-management","management"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/horgh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-05-20T18:40:48.000Z","updated_at":"2023-09-08T17:25:17.000Z","dependencies_parsed_at":"2024-06-21T19:10:05.709Z","dependency_job_id":"81471163-89eb-4a8c-adfb-571457ade0b2","html_url":"https://github.com/horgh/dupefile","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horgh%2Fdupefile","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horgh%2Fdupefile/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horgh%2Fdupefile/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/horgh%2Fdupefile/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/horgh","download_url":"https://codeload.github.com/horgh/dupefile/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244297909,"owners_count":20430346,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dedup","file","file-management","management"],"created_at":"2024-11-25T01:09:27.350Z","updated_at":"2025-03-18T19:56:03.135Z","avatar_url":"https://github.com/horgh.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"This program helps me deal with duplicate files.\n\nI am trying to organize my collection of images. It is disorganized and\ncontains many duplicates. I want to delete duplicates so I can have less\nto organize.\n\n\n# What this program does\nThis program takes a directory and calculates checksums for each file under\nit. It then checks whether any two files have the same checksum, and if so\nit reports the files as duplicates.\n\nYou can provide rules to define which file to keep in order to delete one\nof them. This only runs in live mode. By default it will only report the\nduplicates.\n\n\n# Defining rules\nYou define rules by writing a JSON file. Each rule specifies which file to\nremove by identifying which directory holds the file to keep and which\nholds the one to delete. This allows fine grained control.\n\nAn example rule file with one rule looks like this:\n\n```\n{\n  \"rules\": [\n    {\n      \"keep\":   \"/directory1\",\n      \"remove\": \"/directory2\"\n    }\n  ]\n}\n```\n\nIn this case, if we detect duplicate files `/directory1/example.png` and\n`/directory2/example-test.png`, the program deletes\n`/directory2/example-test.png` and keeps the other.\n\n\n# Behaviour in more detail\n  - Recursively find all files.\n  - Calculate the checksum of each file.\n  - Check whether any two files have the same checksum.\n  - If they do, check whether the two files are really identical.\n  - If they are, take action. This may be to just report (in non-live mode) or\n    to remove one of them (in live mode).\n  - Report any two files with identical checksums.\n  - Report any two files with identical names.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorgh%2Fdupefile","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fhorgh%2Fdupefile","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fhorgh%2Fdupefile/lists"}