https://github.com/joonas-fi/dupfinder
Compares two directories to find duplicate files, and optionally removes them (from either the old or the new directory). Has dry runs for delete.
https://github.com/joonas-fi/dupfinder
Last synced: 2 months ago
JSON representation
Compares two directories to find duplicate files, and optionally removes them (from either the old or the new directory). Has dry runs for delete.
- Host: GitHub
- URL: https://github.com/joonas-fi/dupfinder
- Owner: joonas-fi
- License: apache-2.0
- Created: 2018-11-23T13:18:35.000Z (over 6 years ago)
- Default Branch: main
- Last Pushed: 2024-06-27T09:40:19.000Z (10 months ago)
- Last Synced: 2025-01-03T02:44:25.003Z (4 months ago)
- Language: Go
- Homepage:
- Size: 17.6 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
dupfinder finds duplicate files.
Features
--------
- Compares old and new directory for added/removed/unchanged files
- Emphasis on safety. Your files are precious!
- Understands renames (file content equality is based on sha1)
- Comparison results in a human readable log file, which you can re-read and delete the
unchanged files mentioned in the log, either from the old or the new directory
- Additionally, even though the log-based approach brings safety, we have additional dry run
in the delete process that explains which files it would delete.
- Can remove empty directories (that result from "remove same files from old OR new directory")
Usage
-----
```
$ dupfinder compare demo/old demo/new > comparison.log
2018/11/23 13:36:58 olddir newdir
2018/11/23 13:36:58 Scanning olddir
2018/11/23 13:36:58 starting initializeMissingMap
2018/11/23 13:36:58 Scanning newdir
2018/11/23 13:36:58 Listing missing files
2018/11/23 13:36:58 Done
$ cat comparison.log
INFO olddir newdir
INFO Scanning olddir
INFO starting initializeMissingMap
INFO Scanning newdir
+ demo/new/subfolder/stoned sloth.jpg
= HgAvAA demo/old/subfolder/tcpdump.PNG|demo/new/subfolder/tcpdump renamed.PNG
= NgA2AA demo/old/pcap-screenshot-cloudflare-duplicate-acks.PNG|demo/new/pcap-screenshot-cloudflare-duplicate-acks.PNG
INFO Listing missing files
- demo/old/subfolder/callingcard.jpg
INFO Done
$ dupfinder removeunchangedfilesfromnew < comparison.log
would remove demo/new/subfolder/tcpdump renamed.PNG
would remove demo/new/pcap-screenshot-cloudflare-duplicate-acks.PNG
$ dupfinder removeunchangedfilesfromnew --really < comparison.log
removing demo/new/subfolder/tcpdump renamed.PNG
removing demo/new/pcap-screenshot-cloudflare-duplicate-acks.PNG
$ dupfinder removeemptydirs demo/new
would remove empty dir: demo/new/emptydir
$ dupfinder removeemptydirs --really demo/new
removing empty dir: demo/new/emptydir
```
Alternatives
------------
- https://github.com/jpillora/dedup