{"id":47516103,"url":"https://github.com/ftarlao/duplito","last_synced_at":"2026-04-01T07:00:41.818Z","repository":{"id":302590305,"uuid":"1012758924","full_name":"ftarlao/duplito","owner":"ftarlao","description":"It's a command-line tool, a bit like ls, that lists files in folders. But it does more than just list them: it also tells you which files have duplicates elsewhere on your system (and where those duplicates are located), and which files are completely unique.","archived":false,"fork":false,"pushed_at":"2025-07-28T20:34:30.000Z","size":384,"stargazers_count":8,"open_issues_count":3,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-07-28T22:26:27.158Z","etag":null,"topics":["cli-app","duplicate-detection","utility"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ftarlao.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-07-02T20:49:35.000Z","updated_at":"2025-07-21T20:46:24.000Z","dependencies_parsed_at":"2025-07-03T07:37:05.269Z","dependency_job_id":"58da5931-b7d9-42a4-932d-115d5d03d4d2","html_url":"https://github.com/ftarlao/duplito","commit_stats":null,"previous_names":["ftarlao/duplito"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/ftarlao/duplito","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ftarlao%2Fduplito","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ftarlao%2Fduplito/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ftarlao%2Fduplito/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ftarlao%2Fduplito/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ftarlao","download_url":"https://codeload.github.com/ftarlao/duplito/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ftarlao%2Fduplito/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31269176,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T06:57:45.811Z","status":"ssl_error","status_checked_at":"2026-04-01T06:57:42.389Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli-app","duplicate-detection","utility"],"created_at":"2026-03-27T13:00:30.244Z","updated_at":"2026-04-01T07:00:41.807Z","avatar_url":"https://github.com/ftarlao.png","language":"Go","funding_links":[],"categories":["\u003ca name=\"file-dir-cleanup\"\u003e\u003c/a\u003eClean up of files and directories"],"sub_categories":[],"readme":"# Duplito 🔍 - File Lister and Duplicate Finder\n\nDuplito is a lightweight, efficient **command-line tool** designed to help you identify duplicate files on your system. Whether you're cleaning up old \ndownloads, organizing photos, or freeing up disk space, Duplito makes the process simple and straightforward.\nDuplito lists the files in folders (like 'ls' command or like 'find') by highlighting what is duplicate (and where its duplicates are) and what is not.\n\u003cimg width=\"3167\" height=\"1326\" alt=\"exampleoutput\" src=\"https://github.com/user-attachments/assets/24980e9a-161c-4bfc-a9cb-bfe2f290535c\" /\u003e\n\n## Features\n\n* **Fast Scanning:** Utilizes efficient hashing algorithms (quick hash: MD5 of file parts and filesize) to compare file contents, not just names or sizes.\n* **Flexible Paths:** Scan single directories, subdirectories, and even entire drives.\n* **Detailed Output:** Clearly lists all identified duplicate groups, showing their paths and sizes.\n* **Safe Operations:** Only lists files and highlight duplicates, no disk changes are made\n\n**VERY IMPORTANT** duplito looks also at the file content, but with -u, for huge files it only looks at the hash of the first and last portion of the file, and the filesize. -u  **equality measure is an ehuristic**. For the full file hash use -U \n\n```\nUsage: ./duplito [-rUu] [-i] [-t num_threads] [\u003cpath1\u003e ...]\n\n./duplito identifies potential duplicates using a **composite MD5 hash**\nderived from each file's content and size. Hashing info is stored at \n`~/.duplito/filemap.gob`. The program lists all requested files OR files\nin a `folder-path`, highlighting duplicates and their respective locations.\n\nWhen listing files \u003cpath1\u003e defaults to current folder \".\"Options:\n  -r, --recurse         Recurse into subdirectories (auto with -u or -U).\n  -u, --update          Update hash database using quick-partial hash (implies -r).\n                        If no paths, defaults to user home (or / for root).\n  -U, --UPDATE          Update hash database using full file hash (implies -r).\n  -d, --duplicates      Only shows the duplicates in filelist (summary not affected).\n  -m, --min-file-size   Only lists files with size greater or equal, than the provided filesize\n                        in bytes. Directory and overall summaries are not affected.\n  -i, --ignore-errors   Ignore unreadable/inaccessible files.\n  -t, --threads         Number of concurrent hashing threads (default: 3).\n\n  -s, --summary         Display only 'per' directory summaries and the final overall\n                        summary, with statistics.\n  -o, --overall         Display only the final overall summary with statistics.\n\n  -p, --min-dir-perc    Visualizes summary and file list only for folders with a percentage\n                        of duplicates greater than the specified value (default: 0%).\n  -b, --min-dir-bytes   Visualizes summary and file list only for folders with a file size\n                        of duplicates that exceeds the provided value (default: 0 byte).\nBehavior:\n  -u or -U: Recursively computes and saves file hashes. Paths are\n            optional, defaulting to user home or /.\n  No -u/-U: Loads hash database and lists files with duplicate status.\n            Paths or filenames are required for this mode.\n```\n\nDeveloped by Fabiano Tarlao (2025)\n\n## How to Compile from Sources\n\nInstall git and golang  \n\n```\ngit clone https://github.com/ftarlao/duplito.git\ncd duplito\ngo mod tidy         (don't now, perhaps not mandatory)\n```\n\nIn order to create a bin for local usage with all debug symbols:\n\n```go build -o duplito```\n\nIn order to create a release (statically linked bin with debug stuff stripped, and useless path info removed):\n\n```CGO_ENABLED=0 go build -a -trimpath -ldflags '-extldflags \"-static\" -s -w' -o duplito```\n\n\n## Usage Examples\n\n### Updating the File Database\n\nTo **update or create the files database** for all files within the `/home/pippo/` folder and its subfolders, use the `-u` option. This operation is crucial before checking for duplicates, as it builds the necessary index.\nPlease note that **the previous files database is overwritten**.\n\n```bash\nduplito -u -i /home/pippo/\n```\n\nAfter running this, you'll be ready to identify duplicates across all files in '/home/pippo/' and its subfolders.\n\n### Checking for Duplicates in a Specific Directory\n\nTo **identify duplicate and unique files** specifically within the' /home/pippo/testdir/' directory, use the '-r' option.\n```Bash\nduplito -r -i /home/pippo/testdir/\n```\nFiles with zero byte filesize are not checked to be duplicates, are flagged ZERO SIZE.  \n\nYou can also ask to check for duplicates by providing specific filenames or a list of paths:\n```Bash\nduplito -r -i /home/pippo/file1.txt /home/pippo/temp/file2.bin /home/pippo/testdir/\n```\n\nYou can also use the **shell expansion** to check for files with specific name pattern:\n```Bash\nduplito -r -i /home/pippo/*.txt \n```\n\nYou can omit the path information, 'duplito' lists files in the current folder \n```Bash\nduplito   \n```\n\nthe same as ```duplito .```\n\nYou can also choose to visualize only the statistic summaries, for each folder and subfolder \nand the overall summary (of all listed files). With these options the  file list are not\nvisualized, only summaries!\n\n```Bash\nduplito -r -s -i /home/pippo\n```\nIn details, for each folder he visualize the summary only, and recurse into subdirs, also \nignores errors (e.g. i/o errors, and so on) and keeps working on the other files.\n![image](https://github.com/user-attachments/assets/01d39320-f0f6-4bcb-803c-903ea0bf92ce)\n\n```Bash\nduplito -r -o -i /home/pippo\n```\nOnly the final 'statistics' summary for ALL files included in provided paths and folder (-r implies \nsubfolders)\n![image](https://github.com/user-attachments/assets/c4632b5b-e676-4943-909d-89794cac83ff)\n\nTypical file list example:\n\n![image](https://github.com/user-attachments/assets/b551921f-f040-4599-a9b5-80b3fb2811e9)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fftarlao%2Fduplito","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fftarlao%2Fduplito","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fftarlao%2Fduplito/lists"}