Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

Projects in Awesome Lists tagged with duplicate-files

A curated list of projects in awesome lists tagged with duplicate-files .

https://github.com/sreedevk/deduplicator

Filter, Sort & Delete Duplicate Files Recursively

deduplication duplicate-detection duplicate-files duplicatefilefinder filesystem rust

Last synced: 21 Jun 2025

https://github.com/m-manu/go-find-duplicates

Find duplicate files (photos, videos, music, documents) on your computer, portable hard drives etc.

command-line-tool duplicate-files file-scanner go golang golang-application media utilities

Last synced: 16 May 2025

https://github.com/scrubbbbs/cbird

Command-line program for Content-Based Image Retrieval of images and videos. Includes tools for general search and de-duplication.

command-line-interface computer-vision content-based-image-retrieval duplicate-detection duplicate-files duplicates ffmpeg opencv qt6 similarity-search

Last synced: 06 Apr 2025

https://github.com/jvirkki/dupd

CLI utility to find duplicate files

c deduplication duplicate-files duplicatefilefinder duplicates fdupes

Last synced: 21 Mar 2025

https://github.com/elmindreda/duff

Command-line utility for finding duplicate files

c duplicate-files unix

Last synced: 15 Jun 2025

https://github.com/PJDude/dude

Duplicates Detector is a cross-platform GUI utility for finding duplicate files, allowing you to delete or link them to save space. Duplicate files are displayed and processed on two synchronized panels for efficient and convenient operation.

cli deduplication duplicate duplicate-detection duplicate-files duplicates duplicates-removal easy easy-to-use easyui gui gui-application python python3 sha1 threads tkinter utility utility-application

Last synced: 06 Mar 2025

https://github.com/raspi/duplikaatti

Remove duplicate files.

duplicate-files duplicates files go golang

Last synced: 19 Oct 2025

https://github.com/thushan/smash

Smash through to find duplicate files super fast by slicing files intelligently!

cli cli-tool duplicate-files freebsd go linux macos windows

Last synced: 26 Oct 2025

https://github.com/nicolasbizzozzero/dupe_eraser

A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.

cli duplicate-detection duplicate-files duplicates file-management

Last synced: 12 Apr 2025

https://github.com/deadsoul/dugu

Find, remove and avoid duplicates with dugu: The Duplicates Guru

deduplication dugu duplicate-detection duplicate-files duplicatefilefinder duplicates duplicates-guru python

Last synced: 05 Apr 2025

https://github.com/sameera-madushan/findm

Findm is a python script to find duplicate file copies in a given directory.

duplicate-detection duplicate-files duplicatefilefinder file-hashing python

Last synced: 18 Jul 2025

https://github.com/gyaan/bytebrush

This is a command-line tool written in Go to find duplicate files in a specified directory.

cli cross-platform duplicate-files file-management go golang linux macos windows

Last synced: 15 Jan 2026

https://github.com/gechandesu/fdup

Find and remove duplicate files

duplicate-detection duplicate-files vlang vlang-cli

Last synced: 10 Mar 2026

https://github.com/jeff-99/hashdex

A file indexer based on content hashes to quickly find duplicate files on your system.

duplicate-files duplicatefilefinder file hash indexing

Last synced: 16 Jan 2026

https://github.com/s1s1ty/dropbox-duplicate-remover

Remove duplicate files from dropbox

dropbox duplicate-files python

Last synced: 30 Oct 2025

https://github.com/lpcodes/duplicate-files-remover

This Python script removes duplicate files from a given directory. It prompts the user to select a folder and then uses the MD5 hashing algorithm to check if any files have the same hash value. If it finds duplicate files, it deletes all but one of them.

duplicate-files filemanagement md5-hash python python-3 python-script showcase

Last synced: 13 May 2025

https://github.com/noclin/duplicate-scanner

Ultra Fast Duplicate File Scanner Based on Everything (Windows and NTFS only)

cleaner duplicate duplicate-file-scanner duplicate-files everything

Last synced: 29 Jun 2025

https://github.com/magnetikonline/identix

Python utility which will recursively scan one or more given directories for duplicate files.

duplicate-files python sha-1 utility

Last synced: 02 May 2025

https://github.com/LpCodes/Duplicate-Files-Remover

This Python script removes duplicate files from a given directory. It prompts the user to select a folder and then uses the MD5 hashing algorithm to check if any files have the same hash value. If it finds duplicate files, it deletes all but one of them.

duplicate-files filemanagement md5-hash python python-3 python-script showcase

Last synced: 08 Jul 2025

https://github.com/unhammer/wshiml

📚 Word shingling for near duplicate document detection

dup-finder duplicate-files duplicatefilefinder information-retrieval shingles

Last synced: 17 Mar 2026

https://github.com/germabyte/obsidian-deduper

The Obsidian Duplicate Finder is a user-friendly tool designed to help users manage duplicate files within their Obsidian vaults. It identifies similar Markdown files based on their content, making it easier to organize and declutter note-taking environments.

cosine-similarity deduplication duplicate-files markdown notes obsidian python tfidf tkinter vault

Last synced: 02 Mar 2025

https://github.com/joaoviictorti/line

line perform analysis of duplicate information

duplicate-files golang

Last synced: 08 Sep 2025

https://github.com/victorqribeiro/dtf

DTF - Duplicate Thumbnail Files - A method to identify duplicate files.

duplicate-detection duplicate-files systematic-mapping systematic-reviews

Last synced: 26 Jul 2025

https://github.com/idrinth/duplication

A backup of backups solution for among other things s3 storages

backup backup-script duplicate-files s3 s3-storage sftp

Last synced: 17 Jun 2025

https://github.com/mattriley/node-duplicate-file-finder

Finds duplicate files across given directories without hashing.

dedupe duplicate-files hashless javascript nodejs npm-package

Last synced: 03 Sep 2025

https://github.com/sergio0694/clup

A no-nonsense .NET Core 2.1 CLI duplicate files remover

cli cli-app dotnet dotnet-tool dotnetcore duplicate-detection duplicate-files duplicates-removed netcoreapp

Last synced: 03 Aug 2025

https://github.com/mckael/goduf

A simple (but fast) duplicate file finder written in Go [Mirror repository]

cli duplicate-files duplicates go golang

Last synced: 03 Apr 2025

https://github.com/pouyakary/dup

a tiny and fast command line utility to find the duplicate files within a directory

cli cmd duplicate-detection duplicate-files duplicates filesystem gnu-utilities utility

Last synced: 14 May 2025

https://github.com/umstek/dupkiller

Slow, but more reliable duplicate files cleaner.

cleaner duplicate-files storage

Last synced: 03 Jul 2026

https://github.com/theohbrothers/get-duplicatephotos

A script to locate duplicate image files between two sets of folders (e.g. Camera Roll folders vs other folders).

criteria date-taken duplicate-files duplicate-photos duplicates export file-metadata powershell pwsh script search

Last synced: 20 Mar 2025

https://github.com/rnine/scoop-duobolt

Scoop bucket for duobolt-cli – Fast BLAKE3-based duplicate file finder for Windows

blake3 cli duplicate-detection duplicate-files file-scanner files rust scoop-bucket windows

Last synced: 25 Apr 2026

https://github.com/dr8co/doppel

A simple, modern duplicate file detector.

deduplication duplicate-detection duplicate-files duplicatefilefinder duplicates

Last synced: 18 Jan 2026

https://github.com/maknahar/s3copy

An AWS lambda function to duplicate objects from n to m AWS S3 Buckets where n, m >= 1

amazon-web-services aws-lambda aws-s3 copy duplicate-files duplication go golang lambda

Last synced: 17 Jan 2026

https://github.com/junsious/dupfinder

A simple desktop application to search for duplicate files in a specified directory. This application uses SHA-256 hashing to identify duplicates and provides a user-friendly interface with progress tracking.

duplicate-detection duplicate-files files filesfinder rust

Last synced: 11 Jul 2025

https://github.com/pierrre/file-duplicate

Find duplicate files

duplicate-files go golang

Last synced: 21 Jan 2026

https://github.com/rnine/homebrew-duobolt

Homebrew tap for duobolt-cli – Fast BLAKE3-based duplicate file finder for macOS/Linux

blake3 cli duplicate-detection duplicate-files file-scanner files homebrew linux macos rust

Last synced: 14 Apr 2026

https://github.com/px86/dff

Recursively finds and lists duplicate files in a directory. Uses C++17 filesystem library and SHA-256 hashing algorithm.

cpp17 duplicate-files sha256-hash

Last synced: 07 Oct 2025

https://github.com/mathewsbuzetti/powershell-file-server-space-analyzer

PowerShell script para análise de espaço em servidores de arquivos com detecção de duplicados, grandes/antigos e dashboard HTML interativo

cleanup dashboard deduplication duplicate-files file-server html-report ntfs powershell-script read-only sanitization space-analysis storage-optimization windows-server

Last synced: 01 Jul 2025

https://github.com/davdiv/hashfolder

Simple command line tool that can create/update an sqlite database that contains the hash (by default SHA256) of all files inside a specified root folder.

checksum duplicate-detection duplicate-files duplicates sha256

Last synced: 16 May 2025

https://github.com/shivakar/gdupes

A multithreaded tool for identifying duplicate files

duplicate-files go tools

Last synced: 10 Mar 2026

https://github.com/wadjetz/rust_fdf

Find and remove duplicated files

cli-utilities duplicate-files learning rust

Last synced: 04 Jul 2025

https://github.com/fabricesalvaire/filewalker

A Python library to scan a file system, find duplicated file etc.

duplicate-detection duplicate-files python-library python3

Last synced: 16 Jun 2025

https://github.com/tyler-tee/file-deduplicator

Python app built to scan a directory, check for duplicate files, and send them to the trash.

duplicate duplicate-detection duplicate-files pysimplegui python

Last synced: 21 Apr 2026

https://github.com/knotfalse/czkawka-dupes-to-symlinks

A careful CLI for reclaiming disk space by turning duplicate files into symlinks.

czkawka duplicate-detection duplicate-files symlink

Last synced: 08 Apr 2026

https://github.com/dahead/dupefiles

Dupe Files scans your disks for duplicate files.

csharp deduplication dotnet-core duplicate-detection duplicate-files duplicatefilefinder

Last synced: 23 May 2026

https://github.com/Trophonix/RemoveDuplicates

Simple java program to delete all files in the working directory.

duplicate-detection duplicate-files duplicatefilefinder files gpl gplv3 io java

Last synced: 10 Mar 2025

https://github.com/hekmon/deduper

Analyse 2 paths to found identical files and hard link them to save space

dedup deduplicate deduplication deduplicator duplicate-files filesystem hardlink hardlinking hardlinks linux saving scan scanner space tool utility

Last synced: 14 May 2026

https://github.com/giosali/dupeutil

A command-line program written in Python for detecting and removing duplicate files.

command-line-tool duplicate-detection duplicate-files python

Last synced: 17 Jun 2025

https://github.com/firemandesign/similar-images-2026

⬇ similar-images-2026.rar в Releases · #3in1-fix #combo-bypass #net-unblock #discord-not-working #youtube-blocked #telegram-blocked #connecting-fix #goodbyedpi #dpi-bypass #no-vpn

cleanup czkawka disk-cleaner duplicate-files duplicate-finder file-scanner portable storage

Last synced: 02 Jul 2026

https://github.com/abika/scala-find-dups

Yet another tool to find duplicate files

duplicate-files scala scala-find-dups

Last synced: 20 Jul 2025

https://github.com/theohbrothers/get-duplicatefiles

A script to locate duplicate files between two sets of folders (e.g. source folders vs. other folders).

criteria duplicate duplicate-files export file-metadata files json powershell pwsh script search

Last synced: 19 May 2026

https://github.com/darrenwilliamson3-png/folder_audit_v4

Schema-driven Python audit tool for inspecting, validating, duplicate detection using Sha256 and exporting real-world data.

cli-tool csv-export data-cleaning disk-analysis duplicate-files file-hashing filesystem it-support json-export python recursive sha256 sysadmin

Last synced: 19 May 2026

https://github.com/cplieger/docker-fclones-scheduler

Find and deduplicate files on a schedule — reclaim wasted disk space automatically

cron deduplication distroless docker duplicate-files fclones golang hardlink homelab scheduled-tasks

Last synced: 05 Jul 2026

https://github.com/nattyxo/mega-copy

This Windows Forms application allows users to copy, cut, and duplicate files from one location to another.

copy cut duplicate-files filetransfer mega-copy overwrite

Last synced: 02 Nov 2025

https://github.com/akbarisajjad/scanner-file-windows-

This code is a useful tool for checking for duplicate and corrupted files and can be very useful for managing files in a large directory. However, it could be made more robust and reliable by improving error handling and performance optimization.

duplicate-files filesystem large-files managing-files scanner windows

Last synced: 22 May 2026

https://github.com/tbarbette/refdup

Find and delete duplicate files in a folder using regex

duplicate duplicate-detection duplicate-files files regex

Last synced: 04 Oct 2025

https://github.com/nwtgck/same-file-finder-python

Find duplicate files by message digest (hash)

duplicate-files duplicate-finder

Last synced: 16 Aug 2025

https://github.com/lkwg82/fdupes.java

deduplicates files in directory hierarchy with hardlinks

duplicate-files fdupes java tool

Last synced: 12 Mar 2025

https://github.com/youssef-ben/duplicate-files-manager

Utilities library for finding and handling the duplicates files.

copy-files duplicate-files files managing-files

Last synced: 14 Mar 2025

https://github.com/andreid/godupes

Super Fast Go Duplicates Finder

duplicate-files duplicates go golang

Last synced: 01 Jul 2025

https://github.com/greg-kennedy/jpeg_dupe_test

A simple tool to find duplicate JPEG files.

duplicate-files jpeg

Last synced: 30 Mar 2025

https://github.com/kingchenc/diskghost

Fast disk usage & duplicate finder, written in Rust (CLI + JSON for agents; modern GUI coming).

blake3 cli cross-platform deduplication disk-analyzer disk-space disk-usage du duplicate-files duplicate-finder file-management filesystem gui ncdu performance rayon rust storage tauri windows

Last synced: 14 Jul 2026

https://github.com/itsayellow/finddup

Find duplicate files or directories in a list of paths.

duplicate-detection duplicate-files files

Last synced: 20 Jun 2025

https://github.com/onder7/duplicatefinder

Yinelenen Dosya Bulucu

duplicate-files pyhton3

Last synced: 04 Apr 2025

https://github.com/t3hmun/dupereport

Simple exe that reports duplicate files in current directory and all subdirectories recursively.

duplicate-files

Last synced: 22 Mar 2026

https://github.com/fffaraz/dupdig

CLI tool that walks a directory tree, hashes every file with SHA-256, and generates reports on duplicate files

duplicate-detection duplicate-files go golang

Last synced: 04 Apr 2026

https://github.com/thomasmfish/duplicate-file-checker

A simple Python tool to help identify and delete duplicate files that may be distributed within subdirectories.

duplicate-detection duplicate-files python python3

Last synced: 05 Jun 2026

https://github.com/robb-fr/fast-dupes-finder

This repository proposes clean, fast and shell based scripts for identifying finding duplicate files in a folder.

bash duplicate-detection duplicate-files duplicate-images duplicates fdupes fdupes-linux-utility

Last synced: 22 Apr 2026

https://github.com/nir3x/hardlinkdeduplicator

HardLink Deduplicator - Detect and Manage Duplicate Files with Hard Links

command-line-tool deduplication duplicate-files file-management go golang hardlink ntfs storage-tools

Last synced: 10 Jun 2026

https://github.com/kingzcheung/bdzer

A Small Tool for Finding Duplicate Files

duplicate-files

Last synced: 25 Jun 2026