{"id":18968878,"url":"https://github.com/tttapa/duplicate-file-finder","last_synced_at":"2025-06-14T23:41:43.763Z","repository":{"id":82576970,"uuid":"292124256","full_name":"tttapa/Duplicate-File-Finder","owner":"tttapa","description":"List all duplicate files in a directory.","archived":false,"fork":false,"pushed_at":"2023-11-25T12:21:36.000Z","size":40,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-11-08T14:56:57.329Z","etag":null,"topics":["cleanup","duplicate-detection","file-manager","utility"],"latest_commit_sha":null,"homepage":"","language":"C++","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tttapa.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2020-09-01T22:49:55.000Z","updated_at":"2024-09-10T05:55:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"bd1ebdd6-6d43-4764-9e6d-595faeb659c8","html_url":"https://github.com/tttapa/Duplicate-File-Finder","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tttapa%2FDuplicate-File-Finder","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tttapa%2FDuplicate-File-Finder/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tttapa%2FDuplicate-File-Finder/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tttapa%2FDuplicate-File-Finder/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tttapa","download_url":"https://codeload.github.com/tttapa/Duplicate-File-Finder/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":232055493,"owners_count":18466161,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cleanup","duplicate-detection","file-manager","utility"],"created_at":"2024-11-08T14:48:43.428Z","updated_at":"2025-01-01T07:09:27.487Z","avatar_url":"https://github.com/tttapa.png","language":"C++","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Duplicate File Finder\n\nCommand line utility that recursively scans a directory for duplicate files \n(i.e. files with the same content).\nThis can be really useful to cleanup backups or to remove unnecessary copies \nof files.\n\n## Usage\n\n```sh\n$ dupfind --help\n\nFind duplicate files.\n\nUsage: dupfind [options] [directory]\n\nIf no directory is provided, the current working directory is used.\n\nAllowed options:\n  -h [ --help ]            Print this help message.\n  -v [ --version ]         Print the version number.\n  -d [ --dir ] \u003cdirectory\u003e The directory to process.\n  -i [ --include ] \u003cregex\u003e Include only files that match these patterns.\n  -e [ --exclude ] \u003cregex\u003e Exclude any files that match these patterns.\n  -s [ --sort-size ]       Sort the output by file size.\n  --include-empty-files    Include all empty files.\n```\n\n### Examples\n\nFind all duplicate files in the current folder:\n\n```sh\ndupfind\n```\n\nFind all duplicate WAV and MP3 files in `~/Music` and `~/Audio`, ignoring files \nnamed `cello.wav`:\n\n```sh\ndupfind ~/Music ~/Audio --include '.*\\.wav' --include '.*\\.mp3' --exclude '.*/cello.wav'\n```\n\nFind all duplicate files in the home folder, sorting the output by file size:\n\n```sh\ndupfind ~ --sort-size\n```\n\n## Installation\n\nIf you're on Debian or Ubuntu, you can download the `.DEB` package from the\n[releases](https://github.com/tttapa/Duplicate-Finder/releases) page. It can\nbe installed through the Software Center or through the command line:\n\n```sh\nsudo apt install ~/Downloads/Duplicate-File-Finder-0.0.1-Linux.deb\n```\n\nYou can also download the `.TAR.GZ` archive. Simply unzip it, and move the \nexecutable to a folder that's in your path:\n\n```sh\ntar xzf Duplicate-File-Finder-0.0.1-Linux.tar.gz\nsudo mv bin/dupfind /usr/local/bin\n```\n\nYou might have to install some dependencies, like `libssl1.1` and \n`libboost-program-options1.71.0`.\n\n## Build from source\n\n```sh\n# Install dependencies\nsudo apt install -y libssl-dev libboost-program-options-dev\n# Download this repository\ngit clone https://github.com/tttapa/Duplicate-Finder\ncd Duplicate-Finder\n# Configure\ncmake . -Bbuild -DCMAKE_INSTALL_PREFIX=/usr/local -DCMAKE_BUILD_TYPE=Release\n# Compile\ncmake --build build\n# Install\nsudo cmake --build build --target install\n```\n\n## How it works\n\nThe program builds two dictionaries: one that keeps track of the file size and\nthe file path of each file, and a second one that keeps track of the SHA-1\nhash of some of the files. If the sizes of two files are different, they\ncannot be duplicates.\nIf the sizes of two files are the same, however, their hashes are computed and\ncompared. Finally, all files with hashes that occur multiple times in the\nsecond dictionary are sorted and printed.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftttapa%2Fduplicate-file-finder","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftttapa%2Fduplicate-file-finder","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftttapa%2Fduplicate-file-finder/lists"}