{"id":39533582,"url":"https://github.com/dr8co/doppel","last_synced_at":"2026-01-18T06:31:01.663Z","repository":{"id":302310837,"uuid":"1011985769","full_name":"dr8co/doppel","owner":"dr8co","description":"A simple, modern duplicate file detector.","archived":false,"fork":false,"pushed_at":"2025-12-25T10:17:37.000Z","size":1800,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-12-26T22:58:19.076Z","etag":null,"topics":["deduplication","duplicate-detection","duplicate-files","duplicatefilefinder","duplicates"],"latest_commit_sha":null,"homepage":"https://doppel.draco.codes","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dr8co.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-07-01T16:20:15.000Z","updated_at":"2025-12-25T10:17:34.000Z","dependencies_parsed_at":"2025-07-01T17:54:33.950Z","dependency_job_id":null,"html_url":"https://github.com/dr8co/doppel","commit_stats":null,"previous_names":["dr8co/doppel"],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/dr8co/doppel","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dr8co%2Fdoppel","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dr8co%2Fdoppel/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dr8co%2Fdoppel/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dr8co%2Fdoppel/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dr8co","download_url":"https://codeload.github.com/dr8co/doppel/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dr8co%2Fdoppel/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28531997,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-18T00:39:45.795Z","status":"online","status_checked_at":"2026-01-18T02:00:07.578Z","response_time":98,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["deduplication","duplicate-detection","duplicate-files","duplicatefilefinder","duplicates"],"created_at":"2026-01-18T06:31:00.537Z","updated_at":"2026-01-18T06:31:01.203Z","avatar_url":"https://github.com/dr8co.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003cp align=\"center\"\u003e\n  \u003cimg src=\"./assets/logo480p.png\" alt=\"doppel logo\" height=\"480\"\u003e\n\u003c/p\u003e\n\n\u003c!-- \u003ch1 align=\"center\"\u003e🧿 doppel\u003c/h1\u003e --\u003e\n\u003cp align=\"center\"\u003e\u003cem\u003eYour filesystem has doppelgängers. Let’s hunt.\u003c/em\u003e\u003c/p\u003e\n\n\u003cp align=\"center\"\u003e\n  \u003ca href=\"https://golang.org\"\u003e\u003cimg alt=\"Made with Go\" src=\"https://img.shields.io/badge/Made%20with-Go-00ADD8?logo=go\"\u003e\u003c/a\u003e\n  \u003cimg alt=\"Platform\" src=\"https://img.shields.io/badge/platform-linux%20%7C%20macOS%20%7C%20Windows-blue\"\u003e\n  \u003cimg alt=\"GitHub go.mod Go version\" src=\"https://img.shields.io/github/go-mod/go-version/dr8co/doppel?logo=go\"\u003e\n  \u003cimg alt=\"GitHub Actions CI test\" src=\"https://github.com/dr8co/doppel/actions/workflows/go.yml/badge.svg\"\u003e\n  \u003cimg alt=\"Go report\" src=\"https://goreportcard.com/badge/dr8co/doppel\"\u003e\n  \u003cimg alt=\"License\" src=\"https://img.shields.io/github/license/dr8co/doppel?color=blue\"\u003e\n\u003c/p\u003e\n\n---\n\n**doppel** is a blazing-fast, concurrent CLI tool written in Go for scanning directories\nand finding duplicate files, aka _doppelgängers_! 🕵️‍♂️🗂️\n\nSave disk space and keep your filesystem clean by quickly identifying and managing duplicate files.\nDoppel is designed for speed, flexibility, and reliability.\n\n---\n\n## 📋 Table of Contents\n\n\u003c!-- TOC --\u003e\n* [📋 Table of Contents](#-table-of-contents)\n* [⚡️ Quick Start](#%EF%B8%8F-quick-start)\n* [🔮 Terminal Preview](#-terminal-preview)\n* [✨ Features](#-features)\n* [📦 Installation](#-installation)\n* [🚀 Usage](#-usage)\n  * [🛠️ Command-Line Interface](#%EF%B8%8F-command-line-interface)\n  * [⚙️ Configuration Files](#%EF%B8%8F-configuration-files)\n  * [Environment Variables](#environment-variables)\n    * [Automatic Completion](#automatic-completion)\n  * [🔎 Find Command](#-find-command)\n    * [⚙️ Find Command Options](#%EF%B8%8F-find-command-options)\n  * [🎛️ Preset Command](#%EF%B8%8F-preset-command)\n* [🧬 How It Works](#-how-it-works)\n* [🏗️ Development](#%EF%B8%8F-development)\n* [📜 License](#-license)\n* [🤝 Contributing](#-contributing)\n\u003c!-- TOC --\u003e\n\n## ⚡️ Quick Start\n\nInstall (requires Go 1.25+):\n\n```sh\ngo install github.com/dr8co/doppel@latest\n```\n\nScan your home directory for duplicates:\n\n```sh\ndoppel find ~\n```\n\nOr use a preset for common scenarios:\n\n```sh\ndoppel preset media ~/Pictures\n```\n\n## 🔮 Terminal Preview\n\n![terminal preview](./assets/ghostty_ubuntu.png)\n\n## ✨ Features\n\n* ⚡️ **Fast scanning** with parallel hashing (Blake3, configurable workers)\n* 🔍 **Flexible filtering** by file size, glob patterns, and regular expressions\n* 🔇 **Noise reduction** with path and file exclusions\n* 📊 **Detailed statistics** and verbose output\n* 🛠️ **Dry-run mode** to preview filters\n* 📄 **Structured output** for easy integration with other tools. Supported formats:\n  * JSON\n  * YAML\n  * Text (default)\n* 🧩 **Extensible presets** for common use cases (media, dev, docs, clean)\n* 🧪 **Tested** with unit tests and integration tests\n* 💻 **Cross-platform**: Works on Linux, macOS, and Windows\n* 🛠️ **Automatic completion** for bash, zsh, fish, and PowerShell\n* 📜 **Structured logging** for better automation, debugging, and monitoring. Formats:\n  * JSON\n  * Text\n  * Pretty (default)\n\n## 📦 Installation\n\n**With Go:**\n\n```sh\ngo install github.com/dr8co/doppel@latest\n```\n\n**From source:**\n\n```sh\ngit clone https://github.com/dr8co/doppel.git\ncd doppel\ngo build -o doppel main.go\n```\n\n**Pre-built binaries:**\n\nSee the [🚀 releases page](https://github.com/dr8co/doppel/releases).\n\n## 🚀 Usage\n\n### 🛠️ Command-Line Interface\n\nDoppel provides a simple CLI interface. The main command is `doppel`,\nwith subcommands for different operations.\n\n```sh\ndoppel [global options] [command [command options]]\n```\n\nRun `doppel --help` to see global options and available commands.\n\n\u003e [!NOTE]\n\u003e Running `doppel` with no command defaults to `find`.\n\n### ⚙️ Configuration Files\n\nDoppel supports configuration through TOML (recommended), YAML, or JSON files.\nConfiguration files are automatically loaded from:\n\n* `$CONFIG_DIR/doppel/config.toml`\n* `$CONFIG_DIR/doppel/config.yaml`\n* `$CONFIG_DIR/doppel/config.json`\n* `$CONFIG_DIR/doppel/config` (Assume TOML if no extension)\n\nwhere `$CONFIG_DIR` is your system's user configuration directory:\n\n* Linux: `~/.config`\n* macOS: `~/Library/Application Support`\n* Windows: `%AppData%`\n* Plan 9: `~/lib`\n* Other Unix: `~/.config`\n\nThe configuration files can be used to set default values for any command-line options.\n\nThe key names in the configuration file match the long option names for each command,\nwith dashes replaced with underscores.\nFor example, to set the default minimum file size for the `find` command to 1.5MB,\nyou would add the following to your TOML configuration file:\n\n```toml\n[find]\nmin_size = \"1.5MB\"\n```\n\nFor more details on the TOML format,\nsee the [TOML spec](https://toml.io/en/v1.0.0 \"TOML v1.0.0\").\n\n\u003e [!NOTE]\n\u003e Command-line arguments take precedence over configuration file values.\n\n### Environment Variables\n\nDoppel also supports configuration through environment variables.\nEnvironment variable names are derived from the command and option names,\nwith the following rules:\n\n* The prefix `DOPPEL_` is added to all environment variable names.\n* The command name is added after the prefix (if applicable).\n* The option name is added after the command name.\n* All names are converted to uppercase.\n* Dashes (`-`) in option names are replaced with underscores (`_`).\n\nFor example, to set the default minimum file size for the `find` command to 1.5MB,\nyou would set the following environment variable:\n\n```bash\nDOPPEL_FIND_MIN_SIZE=1.5MB\n```\n\n\u003e [!NOTE]\n\u003e Environment variables take precedence over configuration file values,\n\u003e but are overridden by command-line arguments.\n\n#### Automatic Completion\n\nDoppel supports automatic completion for various shells. To generate completion scripts, run:\n\n```sh\ndoppel completion \u003cshell\u003e\n```\n\nWhere `\u003cshell\u003e` is one of: `bash`, `zsh`, `fish`, or `pwsh`.\n\nThis will print the completion script to stdout.\nYou can redirect it to a file or source it directly in your shell.\n\n### 🔎 Find Command\n\nScan for duplicate files in the current directory:\n\n```sh\ndoppel find\n# or simply\ndoppel\n```\n\nScan specific directories:\n\n```sh\ndoppel find /path/to/dir1 /path/to/dir2\n```\n\n#### ⚙️ Find Command Options\n\n* `-w, --workers \u003cn\u003e`: Number of parallel hashing workers (default: number of CPUs)\n* `-v, --verbose`: Enable verbose output\n* `--min-size \u003csize\u003e`: Minimum file size to consider (default: 0 = no limit)\n* `--max-size \u003csize\u003e`: Maximum file size to consider (default: 0 = no limit)\n* `--exclude-dirs \u003cpatterns\u003e`: Comma-separated glob patterns for directories to exclude\n* `--exclude-files \u003cpatterns\u003e`: Comma-separated glob patterns for files to exclude\n* `--exclude-dirs-regex \u003cregexes\u003e`: Comma-separated regex patterns for directories to exclude\n* `--exclude-files-regex \u003cregexes\u003e`: Comma-separated regex patterns for files to exclude\n* `--show-filters`: Show active filters and exit\n* `--output-format \u003cformat\u003e`: Output format for duplicate groups (default: pretty, options: `pretty`, `json`, `yaml`)\n* `--output-file \u003cfile\u003e`: Write output to a file instead of stdout\n\nFor more details, run:\n\n```sh\ndoppel find --help\n# or\ndoppel find help\n```\n\n**Examples:**\n\nFind duplicates in `~/Downloads` and `~/Documents`, excluding `.git` directories and files smaller than 1MB:\n\n```sh\ndoppel find ~/Downloads ~/Documents --exclude-dirs=.git --min-size=1000000 --verbose\n# or\ndoppel find ~/Downloads ~/Documents --exclude-dirs=.git --min-size=1MB --verbose\n```\n\n`--min-size` and `--max-size` support the following formats:\n\n* Bytes: `100`, `100B`, `100b` are all equivalent\n* Kilobytes: `10KB`, `10kB`, `10Kb`, `10kb`, `10000` are all equivalent\n* Kibibytes: `10KiB`, `10kiB`, `10KIB`, `10240` are all equivalent\n* Megabytes: `1MB`, `1mB`, `1Mb`, `1mb`, `1000000` are all equivalent\n* Mebibytes: `1MiB`, `1miB`, `1MIB`. (same as `1048576`)\n* Gigabytes: `1GB`, `1gB`, `1Gb`, `1gb`. (`1000000000`)\n* Gibibytes: `1GiB`, `1giB`, `1gIB`. (`1073741824`)\n* Terabytes: `1TB`, `1tB`, `1Tb`, `1tb`. (`1000000000000`)\n* Tebibytes: `1TiB`, `1tiB`, `1TIb`. (`1099511627776`)\n* Petabytes: `1PB`, `1pB`, `1Pb`, `1pb`. (`1000000000000000`)\n* Pebibytes: `1PiB`, `1piB`, `1PIB`. (`1125899906842624`)\n* Exabytes: `1EB`, `1eB`, `1Eb`, `1eb`. (`1000000000000000000`)\n* Exbibytes: `1EiB`, `1eiB`, `1EIB`. (`1152921504606846976`)\n\nFind duplicates in `/var/logs`, excluding all `.log` files and directories starting with `temp`,\nand ignoring empty files:\n\n```sh\ndoppel find /var/logs --min-size=1 --exclude-files=\"*.log\" --exclude-dirs=\"temp*\" # Be sure to quote patterns!\n```\n\n\u003e [!NOTE]\n\u003e When using glob patterns and regexes, be sure to quote (and escape, if necessary) them to prevent shell expansion.\n\n### 🎛️ Preset Command\n\nUse presets for common duplicate-hunting scenarios:\n\n* `dev`: Skip development directories and files (e.g., build, temp, version control)\n* `media`: Focus on media files (images/videos), skip small files\n* `docs`: Focus on document files\n* `clean`: Skip temporary and cache files\n\n**Usage:**\n\n```sh\ndoppel preset \u003cpreset\u003e [options]\n```\n\nWhere `\u003cpreset\u003e` is one of: `dev`, `media`, `docs`, or `clean`.\n\nPreset options are the same as for `find`.\n\n**Example:**\n\nFind duplicate media files in your `~/Pictures` folder:\n\n```sh\ndoppel preset media ~/Pictures\n```\n\n## 🧬 How It Works\n\n1. **File Discovery**: Recursively scans specified directories (and their subdirectories), applying filters.\n2. **Grouping**: Groups files by size to quickly eliminate non-duplicates.\n3. **Hashing**: Computes Blake3 hashes for files with matching sizes.\n4. **Reporting**: Displays groups of duplicate files and optional statistics.\n\n## 🏗️ Development\n\n* 📁 Code is organized in `cmd/`, `internal/`, and `assets/` directories.\n* 🧩 Uses [urfave/cli/v3](https://github.com/urfave/cli) for CLI parsing.\n* 🔑 Uses [blake3](https://github.com/lukechampine/blake3) for fast hashing.\n* 🧪 Run tests with:\n\n  ```sh\n  go test -race -v ./...\n  ```\n\n## 📜 License\n\nThis project is licensed under the MIT License. See [LICENSE](LICENSE) for details.\n\n## 🤝 Contributing\n\nContributions, issues, and feature requests are welcome!\nPlease open an issue or pull request on [GitHub](https://github.com/dr8co/doppel).\n\nSee [CONTRIBUTING.md](./CONTRIBUTING.md) for guidelines.\n\n---\n\n**doppel** — Find your duplicate files, fast and reliably. ✨\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdr8co%2Fdoppel","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdr8co%2Fdoppel","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdr8co%2Fdoppel/lists"}