{"id":18828384,"url":"https://github.com/dealfonso/searchdups","last_synced_at":"2025-07-26T02:40:46.646Z","repository":{"id":65651117,"uuid":"596554122","full_name":"dealfonso/searchdups","owner":"dealfonso","description":"Search for duplicate files","archived":false,"fork":false,"pushed_at":"2023-02-02T13:48:23.000Z","size":10,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-01-30T11:36:39.655Z","etag":null,"topics":["command","command-line","command-line-tool","commandline","duplicate-detection","duplicates","files","python","python-script"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dealfonso.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2023-02-02T12:48:45.000Z","updated_at":"2023-09-08T18:40:51.000Z","dependencies_parsed_at":"2023-02-17T20:31:21.113Z","dependency_job_id":null,"html_url":"https://github.com/dealfonso/searchdups","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dealfonso%2Fsearchdups","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dealfonso%2Fsearchdups/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dealfonso%2Fsearchdups/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dealfonso%2Fsearchdups/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dealfonso","download_url":"https://codeload.github.com/dealfonso/searchdups/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239763648,"owners_count":19692812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command","command-line","command-line-tool","commandline","duplicate-detection","duplicates","files","python","python-script"],"created_at":"2024-11-08T01:24:50.901Z","updated_at":"2025-02-20T02:17:03.630Z","avatar_url":"https://github.com/dealfonso.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Search Duplicates (searchdups)\n\nThis is a simple application that searches for duplicate files in a set of folders. To check whether the files are identical or not, it makes use of `md5` or `sha256` algorithms, but the application calculates a _smart hash_ to enhance performance: the idea is to calculate a partial hash and finalize the calculation only if needed.\n\nAdditionally, this application includes a pseudo _hash_ that consists of checking whether the name of the files is the same. If using this _\"hash algorithm\"_, if the name of two files is the same, they are considered to be the same even if the content is not the same.\n\nThe basic usage is\n\n```bash\n$ searchdups -r . \n\u003e 8f8db820d89c39029a0629094e0f18c9*\n/Users/calfonso/Programacion/norepo/searchdups/a1.jpg\n/Users/calfonso/Programacion/norepo/searchdups/a11.jpg\n```\n\nSome other features are:\n\n- Select the hash algorithm (using parameter `-H`).\n- Searching in subfolders (using flag `-r`).\n- Considering hidden folders and files (using flag `-a`).\n- Show a progress bar during the process (using flag `-p`).\n- Selecting which files are processed (using `-f` parameter for _sh-like_ filters, or `-e` parameter for regular expressions).\n- Exclude the files to process (using `-F` parameter for _sh-like_ filters, or `-E` parameter for regular expressions).\n- Summarize the amount of files and folders considered (using flag `-s`).\n- Get the result in a file (using parameter `-o`).\n\nPlease check the CLI help to get updated information about the usage of this tool.\n\n## Installation\n\nTo install the tool you can clone the code and execute the next command inside the cloned folder\n\n```shell\n$ pip install .\n```\n\nor install it from the repositories:\n\n```shell\n$ pip install searchdups\n```","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdealfonso%2Fsearchdups","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdealfonso%2Fsearchdups","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdealfonso%2Fsearchdups/lists"}