Projects in Awesome Lists tagged with file-deduplication

https://github.com/kornelski/dupe-krill

A fast file deduplicator

dedupe dupes file-deduplication hardlinks macos rust-library

Last synced: 12 Apr 2025

https://github.com/jRimbault/yadf

Yet Another Dupes Finder

dedupe deduplication dupes-finder duplicate-detection fdupes file-deduplication

Last synced: 06 Mar 2025

https://github.com/dotfurther/OpenDiscoverSDK

.NET 6 API for document file format identification, text/metadata/attachment/embedded object/sensitive item (PII/PHI)/entity extraction.

archive csharp dotnet email embedded-objects entity-extraction extraction file-deduplication file-format-detection file-identification indexing metadata microsoft-office phi pii pii-detection pst sdk text text-extraction

Last synced: 12 Apr 2025

https://github.com/dotfurther/OpenDiscoverPlatformCaseStudy

Case study using dotfurther's Open Discover Platform with the RavenDB document store to rapidly create a full-text search/eDiscovery/information governance capable demonstration application.

archive-extractor data-breach document-ingestion ediscovery file-deduplication file-format-detection file-identification full-text full-text-extraction full-text-search indexing-engine information-governance information-governance-catalog metadata personally-identifiable-information pii pii-detection ravendb text-extraction

Last synced: 12 Apr 2025

https://github.com/versbinarii/deedoo

File deduplicator

cli-app cli-tool file-deduplication rust rust-lang rust-tools tool

Last synced: 23 Apr 2025

https://github.com/9001/smf

size-match folders

file-deduplication

Last synced: 26 Feb 2025

https://github.com/skippia/twin-scanner-cli

Find duplicate files in multiple folder(s) scanning .txt or/and .torrent files and depending on the selected mode (readonly: true | false) get information about duplicated files /+ extract them into new folders

cli eslint-plugin-functional file-deduplication file-scanner fp-ts functional-programming inquirer inquirer-fuzzy-path nodejs typescript

Last synced: 23 Mar 2025

https://github.com/tobbe84/findduplicates

Этот проект представляет собой мощный инструмент для поиска и анализа дублирующихся файлов в указанной директории. Программа позволяет эффективно выявлять одинаковые файлы на основе их содержимого, используя алгоритм хеширования SHA-256. Она поддерживает настройку параметров, таких как минимальный размер файла для проверки и игнорирование определен

csharp disk-cleanup duplicate-detection duplicate-file-finder external-merge-sort file-comparison file-deduplication file-management file-system linux productivity python sha256 system-utility