https://github.com/metaory/xdedup
find and remove duplicates
https://github.com/metaory/xdedup
cli duplicate-detection
Last synced: 6 months ago
JSON representation
find and remove duplicates
- Host: GitHub
- URL: https://github.com/metaory/xdedup
- Owner: metaory
- License: mit
- Created: 2025-07-06T20:51:16.000Z (6 months ago)
- Default Branch: master
- Last Pushed: 2025-07-18T22:15:28.000Z (6 months ago)
- Last Synced: 2025-07-19T03:10:00.426Z (6 months ago)
- Topics: cli, duplicate-detection
- Language: Shell
- Homepage:
- Size: 7.81 KB
- Stars: 1
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
xdedup
find and remove duplicates
---
designed for large drives, with terabytes of data,
jobs that could take even hours,
with stored hash results on disk for intrupted jobs,
so you can stop a long job mid way and continue later,
the default hashing function is `b3sum` [BLAKE3](http://archlinux.org/packages/extra/x86_64/b3sum/)
you can provide your hashing tool with `--tool` option,
to use other hashing functions like `sha256sum`, `sha512sum`, `md5sum`, ...
by default it only looks for `mkv` video files,
but you can set the extension to look for with `--ext` option,
setting `*` would look for any file
```bash
xdedup
--plan create a delete script instead of deleting
--dry-run perform a trial run with no changes made
--ext EXT file extensions to look for, set * for any (default: mkv)
--tool NAME hashing tool (default: b3sum)
-h, --help show this help
```
## Installation
```bash
# Clone the repo
git clone git@github.com:metaory/xdedup.git
# Navigate to repo
cd xdedup
# Give execution permissions
chmod +x xdedup
# Link it somewhere in your PATH
ln -svf $PWD/xdedup /usr/bin/xdedup
# Use it anywhere
xdedup
```
## License
[MIT](LICENSE)