Projects in Awesome Lists tagged with dedup
A curated list of projects in awesome lists tagged with dedup .
https://laktak.github.io/chkbit/
Check your files for data corruption and run quick file deduplication
backup bitrot-detection btrfs cloud-backup data-degradation data-integrity dedup dedupe deduper deduplication disk-check storage-media
Last synced: 03 Apr 2026
https://github.com/markusressel/py-image-dedup
CLI utility to find near duplicate images and remove all but the best copy.
dedup deduplication duplicate-detection duplicate-images find-duplicates hacktoberfest image-analysis image-comparison python python-3 python3
Last synced: 07 Apr 2025
https://github.com/laktak/chkbit
Check your files for data corruption and run quick file deduplication
backup bitrot-detection btrfs cloud-backup data-degradation data-integrity dedup dedupe deduper deduplication disk-check storage-media
Last synced: 04 Apr 2025
https://github.com/lkarlslund/stringdedup
String deduplication package for Go
dedup deduplication golang string xxhash
Last synced: 22 Apr 2025
https://github.com/veqryn/slog-dedup
Golang structured logging (slog) deduplication and sorting for use with json logging
dedup deduplication golang golang-library json logging slog structured-logging
Last synced: 12 Aug 2025
https://github.com/glehmann/hld
Hard Link Deduplicator
dedup deduplication hardlinks reflinks rust
Last synced: 16 Mar 2025
https://github.com/jumperbot/whitespace-sifter
Sift duplicate whitespaces away!
dedup deduplicator rust rust-crate rust-lang rust-language rust-library sift string string-formatting string-manipulation text whitespace-removal
Last synced: 10 Apr 2025
https://github.com/xyb/chunkdup
Find (partial content) duplicate files.
content-defined-chunking dedup deduplication fastcdc rolling-hash
Last synced: 28 Jul 2025
https://github.com/eminence/deduprs
Hardlink deduplication tool for Linux
dedup deduplication hard-link rust
Last synced: 25 Mar 2025
https://github.com/shahradelahi/undici-extra
🌊 Elegant HTTP client for Node.js based on Undici with extra features
dedup fetch hooks http-client ndjson nodejs pagination proxy retry undici
Last synced: 15 Feb 2026
https://github.com/hekmon/deduper
Analyse 2 paths to found identical files and hard link them to save space
dedup deduplicate deduplication deduplicator duplicate-files filesystem hardlink hardlinking hardlinks linux saving scan scanner space tool utility
Last synced: 14 May 2026
https://github.com/jamjamjon/ilytix
A CLI tool for images analysis: checking image integrity, images deduplication, image retrieval.
dedup image-deduplication rust
Last synced: 24 Jul 2025
https://github.com/horgh/dupefile
Detect and optionally delete duplicate files in a directory tree
dedup file file-management management
Last synced: 18 Mar 2025
https://github.com/xyb/chunksum
Print FastCDC rolling hash chunks and checksums.
checksum chunking content-defined-chunking dedup deduplication fastcdc rolling-hash
Last synced: 25 Mar 2025