{"id":16891524,"url":"https://github.com/rsalmei/refine","last_synced_at":"2025-07-17T12:40:13.428Z","repository":{"id":62444589,"uuid":"496529788","full_name":"rsalmei/refine","owner":"rsalmei","description":"Refine your file collections using Rust!","archived":false,"fork":false,"pushed_at":"2025-07-05T21:47:25.000Z","size":781,"stargazers_count":14,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-07-09T15:45:17.747Z","etag":null,"topics":["batch","deduplicate","duplicates","files","rename","rust","scan"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/refine","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rsalmei.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2022-05-26T07:52:33.000Z","updated_at":"2025-07-06T21:14:02.000Z","dependencies_parsed_at":"2024-06-24T10:27:38.995Z","dependency_job_id":"5983e246-08df-4b07-b625-277b6ec6885b","html_url":"https://github.com/rsalmei/refine","commit_stats":{"total_commits":16,"total_committers":1,"mean_commits":16.0,"dds":0.0,"last_synced_commit":"970b37c1662527b7c8b3ae4a7b722d64a62e9b9b"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rsalmei/refine","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsalmei%2Frefine","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsalmei%2Frefine/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsalmei%2Frefine/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsalmei%2Frefine/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rsalmei","download_url":"https://codeload.github.com/rsalmei/refine/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rsalmei%2Frefine/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265607250,"owners_count":23797106,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch","deduplicate","duplicates","files","rename","rust","scan"],"created_at":"2024-10-13T17:07:36.743Z","updated_at":"2025-07-17T12:40:13.413Z","avatar_url":"https://github.com/rsalmei.png","language":"Rust","funding_links":["https://www.buymeacoffee.com/rsalmei","https://www.paypal.com/donate?business=6SWSHEB5ZNS5N\u0026no_recurring=0\u0026item_name=I%27m+the+author+of+alive-progress%2C+clearly+and+about-time.+Thank+you+for+appreciating+my+work%21\u0026currency_code=USD"],"categories":[],"sub_categories":[],"readme":"# refine\n\n[![Crates.io](https://img.shields.io/crates/v/refine.svg)](https://crates.io/crates/refine)\n[![dependency status](https://deps.rs/repo/github/rsalmei/refine/status.svg)](https://deps.rs/repo/github/rsalmei/refine)\n![Crates.io](https://img.shields.io/crates/d/refine)\n![GitHub Sponsors](https://img.shields.io/github/sponsors/rsalmei)\n\n### Refine your file collection using Rust!\n\n## What it does\n\nThis tool will revolutionize the way you manage and organize your media collections! It can scan multiple root directories recursively and analyze all the files and directories found as a whole, performing some advanced operations on them.\n\nIt can help you reasonably find duplicated files based on both size and filename, seamlessly join them into a single directory with advanced conflict resolution, quickly list files from multiple directories sorted together by various criteria, effortlessly rename files and directories using advanced regular expressions, intelligently rebuild entire media collection names, and even reliably probe filenames against a remote server! It is a one-stop solution for all your media needs, allowing you to organize them in a way that makes sense to you.\n\n\u003e Use it to _refine_ your photo, music, movie, porn, etc., collection in a simple and efficient way!\n\nI've made this tool to be the fastest and easiest way to organize media collections. I use it a lot, and I hope it can help you too.\n\nAnd yes, it is blazingly fast, like all Rust 🦀 software!\n\nEnjoy!\n\n## How to use it\n\nInstall `refine` with:\n\n```\ncargo install refine\n```\n\nAnd that's it, you're ready to go! You can now call it anywhere.\n\n## What's new\n\n### New in 2.0\n\nYay! This is a major release, with a lot of new features!\n\u003cbr\u003eThe most exciting one is global support for COLORS, making files and directories much easier to read and distinguish!\n\nAlso, the `list` command is greatly improved, with support for listing directory entries, complete with their number of files and full sizes! This was only possible with the new precise recursion feature, allowing you to choose how deep you want to go within directories.\n\u003cbr\u003eYou can also now sort the output by number of files, in addition to size (full recursive size), name, or path.\n\nAnother great new feature is the global `--view` option, which allows you to bypass any command and quickly view the filtered files and directories that will be processed by it! Countless times I wanted to preview my filter results, forcing me to replace the command with `list`, remove all other arguments, execute it, study the output, and painstakingly reconstruct the original command—a hugely frustrating process. And now we can do it in any command without changing anything, just by adding `--view`!\n\nEverything is again more polished and optimized. Even the usage and help are much more user-friendly!\n\nAnd last but not least, the input paths can now be relative, which will make all output also be relative and thus easier to read.\n\nBehold the new `refine` command!\n\n![refine 2.0 list](https://raw.githubusercontent.com/rsalmei/refine/main/img/list-2.0.png)\n\n\u003cdetails\u003e\u003csummary\u003e(older versions)\u003c/summary\u003e\n\n### New in 1.4\n\nThis version introduces the `probe` command, which allows you to probe filenames against a remote server! This can be used to validate the filenames of your media collections by checking whether a URL points to a valid file or page on a remote server.\n\nAlso, the `rebuild` command has a new `--case` option, which allows you to keep the original case of the filenames, and the `rename` command has improved support for handling clashes, allowing you to insert sequence numbers in the filenames when you really want to let them be the same.\n\n### New in 1.3\n\nThis version is mostly about polishing, with some improvements and bug fixes.\n\nWe have a smarter list command, which hides full paths by default and uses descending order for size and ascending for name and path; join: change no_remove flag to parents (n -\u003e p) and some clash options; rebuild: change simple_match flag to simple and fix full mode, which was not resetting sequences; general polishing.\n\n### New in 1.2\n\nHere is a much improved partial mode in Rebuild command, which can alter groups of filenames while preserving sequences, and even detect and fix gaps in sequences caused by deleted files.\n\n### New in 1.1\n\nRevamped join command!\nIt now supports non-empty target folders, and will resolve clashes accordingly.\n\nAlso, several enum CLI arguments now support aliases, and I've fixed join command still moving files even when copy was requested.\n\n### New in 1.0\n\nYes, it is time. After a complete overhaul of the code, it's time to release 1.0!\n\u003cbr\u003eIt's an accomplishment I'm proud of, which took over 70 commits and a month's work, resulting in most of the code being rewritten.\nIt is more mature, stable, and well-structured now.\n\nThe major motivation for this version is the rebuild Partial mode! We can now rebuild collections even when some directories are not available! This means that files not affected by the specified naming rules will stay the same, keeping their sequence numbers, while new files are appended after the highest sequence found. It is handy for collections on external drives or cloud storage which are not always connected, allowing you to, even on the go, rebuild new files without messing up previous ones.\n\nAnd this also includes:\n\n- rebuild: new `--replace` option to replace all occurrences of some string or regex in the filenames with another one.\n- new internal CLI options handling, which enables commands to modify them prior to their execution.\n    - the new rebuild partial mode is auto-enabled in case not all directories are currently available.\n\n### New in 0.18\n\n- rebuild: new force implementation that is easier to use\n    - it conflicts with any other options so must be used alone\n    - now it just overwrites filenames without exceptions → best used with `-i` or on already organized collections\n    - improved memory usage\n\n### New in 0.17\n\n- join: new clash resolve option\n    - by default, no changes are allowed in directories where clashes are detected\n    - all directories with clashes are listed, showing exactly which files are in them\n\n### New in 0.16\n\n- complete overhaul of the scan system, allowing directories to be extracted alongside files\n- new `join` command, already with directory support\n- new magic `-i` and `-x` options that filter both files and directories\n- new filter options for files, directories, and extensions\n- rename: include full directory support\n\n### New in 0.15\n\n- nicer rename command output by parent directory\n- new threaded yes/no prompt that can be aborted with CTRL-C\n\n### New in 0.14\n\n- rename: disallow by default changes in directories where clashes are detected\n    - new `--clashes` option to allow them\n\n### New in 0.13\n\n- rename: new replace feature, finally!\n- global: make strip rules also remove `.` and `_`, in addition to `-` and spaces\n- global: include and exclude options do not check extensions\n- dupes: remove case option, so everything is case-insensitive now\n\n### New in 0.12\n\n- global: new `--dir-in` and `--dir-out` options.\n\n### New in 0.11\n\n- new `rename` command\n- rebuild, rename: improve strip exact, not removing more spaces than needed\n\n### New in 0.10\n\n- global: new `--exclude` option to exclude files\n\n### New in 0.9\n\n- new support for Ctrl-C, to abort all operations and gracefully exit the program at any time.\n    - all commands will stop collecting files when Ctrl-C is pressed\n    - both `dupes` and `list` command will show partial results\n    - the `rebuild` command will just exit, as it needs all the files to run\n\n### New in 0.8\n\n- new \"list\" command\n\n### New in 0.7\n\n- global: new `--include` option to filter input files\n- rebuild: new `--force` option to easily rename new files\n- rebuild: new interactive mode by default, making `--dry_run` obsolete (removed), with new `--yes` option to bypass it (good for automation)\n- rebuild: auto fix renaming errors\n- dupes: faster performance by ignoring groups with one file (thus avoiding loading samples)\n- rebuild: smaller memory consumption by caching file extensions\n\n\u003c/details\u003e\n\n## Commands\n\nAll commands will:\n\n1. scan all the given directories recursively (excluding hidden `.folders`)\n    - can optionally filter files and directories based on several options\n2. load the metadata for each file like size and creation date, as required by some commands\n3. execute the command and show the results\n4. ask the user to perform the changes, if applicable\n\n\u003e Everything is always interactive, so don't worry experimenting!\n\u003e \u003cbr\u003eIt's like you're always in dry-run mode, so nothing will be persisted until you review the results and agree to apply them.\n\n### The global refine command (help)\n\n\u003e `$ refine --help`\n\u003e\n\u003e ```\n\u003e Refine your file collections using Rust!\n\u003e \n\u003e Usage: refine \u003cCOMMAND\u003e [DIRS]... [FETCH] [OPTIONS]\n\u003e \n\u003e Commands:\n\u003e   dupes    Find possibly duplicated files by both size and filename\n\u003e   join     Join files into a single directory with advanced conflict resolution\n\u003e   list     List files from multiple disjoint directories sorted together\n\u003e   rebuild  Rebuild entire media collections' filenames intelligently\n\u003e   rename   Rename files and directories in batch using advanced regex rules\n\u003e   probe    Probe collections' filenames against a remote server\n\u003e   help     Print this message or the help of the given subcommand(s)\n\u003e \n\u003e Arguments:\n\u003e   [DIRS]...  Directories to scan\n\u003e \n\u003e Options:\n\u003e   -h, --help     Print help\n\u003e   -V, --version  Print version\n\u003e \n\u003e Fetch:\n\u003e   -R, --recurse \u003cINT\u003e    The maximum recursion depth; use 0 for unlimited [default: 0]\n\u003e   -F, --only-files       Include only files\n\u003e   -D, --only-dirs        Include only directories\n\u003e   -i, --include \u003cREGEX\u003e  Include only these files and directories\n\u003e   -x, --exclude \u003cREGEX\u003e  Exclude these files and directories\n\u003e   -I, --dir-in \u003cREGEX\u003e   Include only these directories\n\u003e   -X, --dir-ex \u003cREGEX\u003e   Exclude these directories\n\u003e       --file-in \u003cREGEX\u003e  Include only these files\n\u003e       --file-ex \u003cREGEX\u003e  Exclude these files\n\u003e       --ext-in \u003cREGEX\u003e   Include only these extensions\n\u003e       --ext-ex \u003cREGEX\u003e   Exclude these extensions\n\u003e       --view             Bypass the command execution and preview the filter results to be processed\n\u003e \n\u003e For more information, see https://github.com/rsalmei/refine\n\u003e ```\n\n### The `dupes` command\n\nThe `dupes` command will analyze and report the possibly duplicated files, either by size or name. It will even load a sample from each file, to guarantee they are indeed duplicated. It is a small sample by default but can help reduce false positives a lot, and you can increase it if you want.\n\n1. group all the files by size\n2. for each group with the exact same value, load a sample of its files\n3. compare the samples with each other and find possible duplicates\n4. group all the files by words in their names\n    - the word extractor ignores sequence numbers like file-1, file copy, file-3 copy 2, etc.\n5. run 2. and 3. again, and print the results\n\n\u003e `$ refine dupes --help`\n\u003e\n\u003e ```\n\u003e Find reasonably duplicated files by both size and filename\n\u003e \n\u003e Usage: refine dupes [DIRS]... [FETCH] [OPTIONS]\n\u003e \n\u003e Arguments:\n\u003e   [DIRS]...  Directories to scan\n\u003e \n\u003e Options:\n\u003e   -s, --sample \u003cINT\u003e  Sample size in bytes (0 to disable) [default: 2048]\n\u003e   -h, --help          Print help\n\u003e ```\n\n\u003e There's also the \"Fetch\" options, which are the same as for the global refine command.\n\nExample:\n\n```\n$ refine dupes ~/Downloads /Volumes/External --sample 20480\n```\n\n### The `join` command\n\nThe `join` command will let you grab all files and directories in the given directories and join them into a single one. You can filter files however you like, and choose whether they will be joined by moving or copying. It will even remove the empty parent directories after a move joining!\n\n\u003e Note: any deletions are only performed after files and directories have been successfully moved/copied. So, in case any errors occur, the files and directories partially moved/copied will be found in the target directory, so you should manually delete them before trying again.\n\n1. detect clashes, i.e. files with the same name in different directories, and apply the given clash strategy\n2. detect already in-place files\n3. print the resulting changes to the filenames and directories, and ask for confirmation\n4. if the user confirms, apply the changes\n5. remove any empty parent directories when moving files\n\n\u003e `$ refine join --help`\n\u003e\n\u003e ```\n\u003e Join files into a single directory with advanced conflict resolution\n\u003e \n\u003e Usage: refine join [OPTIONS] [DIRS]...\n\u003e \n\u003e Options:\n\u003e   -t, --target \u003cPATH\u003e  The target directory; will be created if it doesn't exist [default: .]\n\u003e   -b, --by \u003cSTR\u003e       The type of join to perform [default: move] [possible values: move, copy]\n\u003e   -c, --clashes \u003cSTR\u003e  How to resolve clashes [default: name-sequence] [possible values: name-sequence, parent-name, name-parent, ignore]\n\u003e   -f, --force          Force joining already in place files and directories, i.e. in subdirectories of the target\n\u003e   -p, --parents        Do not remove empty parent directories after joining files\n\u003e   -y, --yes            Skip the confirmation prompt, useful for automation\n\u003e   -h, --help           Print help\n\u003e ```\n\n\u003e There's also the \"Fetch\" options, which are the same as for the global refine command.\n\nExample:\n\n```\n$ refine join ~/media/ /Volumes/External/ -i 'proj-01' -X 'ongoing' -t /Volumes/External/proj-01\n```\n\n### The `list` command\n\nThe `list` command will gather all the files in the given directories, sort them by name, size, or path, and display them in a friendly format.\n\n1. sort all files by either name, size, or path\n    - ascending by default for name and path, descending for size, or optionally reverse\n2. print the results\n\n\u003e `$ refine list --help`\n\u003e\n\u003e ```\n\u003e List files from multiple directories sorted together\n\u003e \n\u003e Usage: refine list [DIRS]... [FETCH] [OPTIONS]\n\u003e \n\u003e Arguments:\n\u003e   [DIRS]...  Directories to scan\n\u003e \n\u003e Options:\n\u003e   -b, --by \u003cSTR\u003e      Sort by [default: size] [possible values: size, count, name, path]\n\u003e   -r, --rev           Reverse the default order (size/count:desc, name/path:asc)\n\u003e   -p, --paths         Show full file paths\n\u003e   -c, --no-calc-dirs  Do not calculate directory sizes\n\u003e   -h, --help          Print help\n\u003e ```\n\n\u003e There's also the \"Fetch\" options, which are the same as for the global refine command.\n\nExample:\n\n```\n$ refine list ~/Downloads /Volumes/External --by size --desc\n```\n\n### The `rebuild` command\n\nI’m really proud of the `rebuild` command. It smartly rebuilds all the filenames of entire media collections, e.g., musics by album/singer and videos by streamers and even photos from your camera. Sequence numbers are removed, filenames are stripped according to your needs, similar names are intelligently matched, groups are sorted deterministically by creation date, sequence numbers are regenerated, and files are finally renamed!\n\nIt's awesome to quickly find your collections neatly sorted automatically; it's like magic, getting all files cleaned up, sorted, and sequenced with a single command. And upon running it again, the tool will seem to recognize the new files that have been added, as it will regenerate everything but only display entries that need to be changed, as the rest are already correct! And in case you delete files, all the subsequent ones will be renamed accordingly! Quite impressive, don't you think?\n\nAnd don't worry as this tool is interactive, so you can review all changes before applying them.\n\n1. apply naming rules to strip or replace parts of the filenames\n2. extract and strip sequence numbers from names\n3. if force mode is enabled, set all names to the forced value\n4. if smart match is enabled, remove spaces and underscores from names\n5. group the files by their resulting names\n6. sort the groups according to the files' created dates\n7. regenerate sequence numbers for each group; if partial mode is enabled, continue from the highest sequence found in the group\n   \u003e Note that these groups can contain files from different directories, and it will just work\n8. print the resulting changes to the filenames, and ask for confirmation\n9. if the user confirms, apply the changes\n\n\u003e `$ refine rebuild --help`\n\u003e\n\u003e ```\n\u003e Rebuild entire media collections intelligently\n\u003e \n\u003e Usage: refine rebuild [DIRS]... [FETCH] [OPTIONS]\n\u003e \n\u003e Arguments:\n\u003e   [DIRS]...  Directories to scan\n\u003e \n\u003e Options:\n\u003e   -b, --strip-before \u003cSTR|REGEX\u003e    Strip from the start of the filename; separators nearby are automatically removed\n\u003e   -a, --strip-after \u003cSTR|REGEX\u003e     Strip to the end of the filename; separators nearby are automatically removed\n\u003e   -e, --strip-exact \u003cSTR|REGEX\u003e     Strip all occurrences in the filename; separators nearby are automatically removed\n\u003e   -r, --replace \u003cSTR|REGEX=STR|$N\u003e  Replace all occurrences in the filename with another; separators are not touched\n\u003e   -s, --simple                      Disable smart matching, so \"foo bar.mp4\", \"FooBar.mp4\" and \"foo__bar.mp4\" are different\n\u003e   -f, --force \u003cSTR\u003e                 Force to overwrite filenames (use the Global options to filter files)\n\u003e   -p, --partial                     Assume not all directories are available, which retains current sequences (but fixes gaps)\n\u003e   -c, --case                        Keep the original case of filenames, otherwise they are lowercased\n\u003e   -y, --yes                         Skip the confirmation prompt, useful for automation\n\u003e   -h, --help                        Print help\n\u003e ```\n\n\u003e There's also the \"Fetch\" options, which are the same as for the global refine command.\n\nExample:\n\n```\n$ refine rebuild ~/media /Volumes/External -a 720p -a Bluray -b xpto -e old\n```\n\n### The `rename` command\n\nThe `rename` command will let you batch rename files like no other tool, seriously! You can quickly strip common prefixes, suffixes, and exact parts of the filenames, as well as apply any regex replacements you want. By default, in case a filename ends up clashing with other files in the same directory, that whole directory will be disallowed to make any changes. The list of clashes will be nicely formatted and printed, so you can manually check them. And you can optionally allow changes to other files in the same directory, removing only the clashes if you find it safe.\n\n1. apply naming rules to strip or replace parts of the filenames\n2. handle the clashes according to the given strategy, which can:\n    - forbid any changes in the directory where clashes are detected\n    - ignore the clashes, allowing other changes in the same directory\n    - apply sequence numbers to the clashes, allowing all changes\n3. print the resulting changes to the filenames and directories, and ask for confirmation\n4. if the user confirms, apply the changes\n\n\u003e `$ refine rename --help`\n\u003e\n\u003e ```\n\u003e Rename files and directories using advanced regular expression rules\n\u003e \n\u003e Usage: refine rename [DIRS]... [FETCH] [OPTIONS]\n\u003e \n\u003e Arguments:\n\u003e   [DIRS]...  Directories to scan\n\u003e \n\u003e Options:\n\u003e   -b, --strip-before \u003cSTR|REGEX\u003e    Strip from the start of the filename; separators nearby are automatically removed\n\u003e   -a, --strip-after \u003cSTR|REGEX\u003e     Strip to the end of the filename; separators nearby are automatically removed\n\u003e   -e, --strip-exact \u003cSTR|REGEX\u003e     Strip all occurrences in the filename; separators nearby are automatically removed\n\u003e   -r, --replace \u003cSTR|REGEX=STR|$N\u003e  Replace all occurrences in the filename with another; separators are not touched\n\u003e   -c, --clashes \u003cSTR\u003e               How to resolve clashes [default: forbid] [possible values: forbid, ignore, name-sequence]\n\u003e   -y, --yes                         Skip the confirmation prompt, useful for automation\n\u003e   -h, --help                        Print help\n\u003e ```\n\n\u003e There's also the \"Fetch\" options, which are the same as for the global refine command.\n\nExample:\n\n```\n$ refine rename ~/media /Volumes/External -b \"^\\d+_\" -r '([^\\.]*?)\\.=$1 '\n```\n\n### The `probe` command\n\nThe `probe` command allows you to probe filenames against a remote server, which can be very useful to validate the filenames of your media collections. It works by checking whether a URL points to a valid file or page on a remote server.\n\nThe URL can be any valid HTTP(S) URL, and must have a placeholder for the filename. The command generates URLs by replacing the placeholder with the names of the files, and sends a HEAD request to each one, allowing you to use some advanced options to control the behavior, such as the timeout, number of retries, wait times, exponential backoff, and when to display errors. The request is expected to return:\n- a 200 OK or 403 Forbidden response to be considered valid;\n- a 404 Not Found to be considered invalid;\n- any other response is retried, with exponential backoff, until the maximum number of retries is reached, then it is considered failed.\n\nIt does not support any kind of parallel connections or API rate limiting by design in order to not disturb the server too much. It thus only works in a sequential manner and may take a while to complete. It also does not support any kind of authentication, redirects, or custom headers, so it may not work for some servers.\n\n1. extract the names from files (without sequence numbers and extension), and deduplicate them\n2. pick the desired subset of them (by a regex)\n3. prepare the URL for each name and probe it with a HEAD request\n4. split the results into Valid, Invalid, Failed, and Pending (in case you press Ctrl+C)\n5. print the invalid ones, along with a summary of the results\n\n\u003e `$ refine probe --help`\n\u003e\n\u003e ```\n\u003e Probe filenames against a remote server\n\u003e \n\u003e Usage: refine probe [DIRS]... [FETCH] [OPTIONS]\n\u003e \n\u003e Arguments:\n\u003e   [DIRS]...  Directories to scan\n\u003e \n\u003e Options:\n\u003e   -p, --pick \u003cREGEX\u003e     Pick a subset of the files to probe\n\u003e   -u, --url \u003cURL\u003e        The URL to probe filenames against (use `$` as placeholder, e.g. https://example.com/$/)\n\u003e   -t, --timeout \u003cINT\u003e    The HTTP connection and read timeouts in milliseconds [default: 2000]\n\u003e   -n, --min-wait \u003cINT\u003e   The initial time to wait between retries in milliseconds [default: 1000]\n\u003e   -b, --backoff \u003cFLOAT\u003e  The factor by which to increase the time to wait between retries [default: 1.5]\n\u003e   -a, --max-wait \u003cINT\u003e   The maximum time to wait between retries in milliseconds [default: 5000]\n\u003e   -r, --retries \u003cINT\u003e    The maximum number of retries; use 0 to disable and -1 to retry indefinitely [default: -1]\n\u003e   -e, --errors \u003cSTR\u003e     Specify when to display errors [default: each10] [possible values: never, last, always, each10]\n\u003e   -h, --help             Print help\n\u003e ```\n\n\u003e There's also the \"Fetch\" options, which are the same as for the global refine command.\n\nExample:\n\n```\n$ refine probe ~/media /Volumes/External --url 'https://example.com/$/' -r3 -el\n```\n\n## Changelog\n\n\u003cdetails\u003e\u003csummary\u003e(click to expand)\u003c/summary\u003e\n\n- 2.0.0 Mar 24, 2025: global support for COLORS; `list` command is greatly improved with support for listing directory entries, number of files and full sizes; new precise recursion feature; new global `--view` option; input paths can now be relative.\n  Behold the new `refine` command!\n- 1.4.0 Feb 28, 2025: new `probe` command; rebuild: new `--case` option to keep original case, rename: included support for handling clashes by inserting sequences in the filenames.\n- 1.3.1 Feb 04, 2025: rebuild: fix the last full mode change, to actually reset sequences.\n- 1.3.0 Jan 31, 2025: list: smarter list command, which hides full paths by default (with a flag for showing them if needed) and uses by default descending order for size and ascending for name and path (with a flag to reverse it if needed); join: change no_remove flag to parents (n -\u003e p) and some clash options; rebuild: change simple_match flag to simple and fix full mode, which was not resetting sequences; general polishing.\n- 1.2.1 Nov 19, 2024: just require newer regex, so deps badge won't show \"maybe insecure\".\n- 1.2.0 Nov 19, 2024: rebuild: much improved partial mode which can alter groups of filenames while preserving sequences, and even detect and fix gaps in sequences caused by deleted files.\n- 1.1.0 Oct 10, 2024: join: support not empty target folders and resolve clashes accordingly; include support for aliases in several enum CLI arguments; fix join by copy still moving files.\n- 1.0.0 Oct 09, 2024: major overhaul; rebuild: new partial mode, new replace feature, auto-enable partial mode in case not all directories are available.\n- 0.18.0 Aug 27, 2024: rebuild: new force implementation that is easier to use with improved memory usage.\n- 0.17.1 Aug 15, 2024: global: fix `--shallow` option.\n- 0.17.0 Aug 05, 2024: global: dedup input directories, enables to select only files by filtering extensions; join: new clash resolve option.\n- 0.16.0 Ago 01, 2024: global: scan with directory support, new `join` command, new magic filter options, new filter options; rename: include full directory support.\n- 0.15.0 Jul 18, 2024: rename: nicer command output by parent directory; new threaded yes/no prompt that can be aborted with CTRL-C.\n- 0.14.0 Jul 11, 2024: rename: disallow by default changes in directories where clashes are detected, including new `--clashes` option to allow them.\n- 0.13.0 Jul 10, 2024: rename: new replace feature; global: make strip rules also remove `.` and `_`, `--include` and `--exclude` options do not check file extensions; dupes: remove case sensitivity option.\n- 0.12.0 Jul 09, 2024: global: new `--dir-in` and `--dir-out` options.\n- 0.11.0 Jul 08, 2024: global: new `rename` command; rebuild, rename: improve strip exact.\n- 0.10.0 Jul 02, 2024: global: new `--exclude`.\n- 0.9.0 Jul 01, 2024: global: support for CTRL-C.\n- 0.8.0 Jun 30, 2024: new `list` command.\n- 0.7.1 Jun 28, 2024: global: `--include` is now case-insensitive; rebuild: fix smart detect not grouping some files, strip rules remove hyphens too.\n- 0.7.0 Jun 27, 2024: global: new `--include`; rebuild: new `--force`, new interactive mode, new `--yes`, auto fix rename errors, smaller memory consumption; dupes: improved performance.\n- 0.6.0 Jun 24, 2024: global: new `rebuild` command, general polishing overall.\n- 0.5.0 Jun 20, 2024: support for shallow scan, verbose mode; dupes: ignores repetition systems.\n- 0.4.0 Jun 17, 2024: include `dupes` command, support match case and changing sample size.\n- 0.3.0 Nov 07, 2023: include dedup by both size and name.\n- 0.2.2 Jun 04, 2022: use 2KB sample size.\n- 0.2.1 Jun 04, 2022: improve error handling.\n- 0.2.0 Jun 01, 2022: publish as `refine`, use split crate `human-repr`.\n- 0.1.1 May 27, 2022: samples the center of the files, which seems to fix false positives.\n- 0.1.0 May 25, 2022: first release, detects duplicated files, simple sampling strategy (1KB from the start of the files).\n\n\u003c/details\u003e\n\n## License\n\nThis software is licensed under the MIT License. See the LICENSE file in the top distribution\ndirectory for the full license text.\n\n\n---\nMaintaining an open source project is hard and time-consuming, and I've put much ❤️ and effort into\nthis.\n\nIf you've appreciated my work, you can back me up with a donation! Thank you. 😊\n\n[\u003cimg align=\"right\" src=\"https://cdn.buymeacoffee.com/buttons/default-orange.png\" width=\"217px\" height=\"51x\"\u003e](https://www.buymeacoffee.com/rsalmei)\n[\u003cimg align=\"right\" alt=\"Donate with PayPal button\" src=\"https://www.paypalobjects.com/en_US/i/btn/btn_donate_LG.gif\"\u003e](https://www.paypal.com/donate?business=6SWSHEB5ZNS5N\u0026no_recurring=0\u0026item_name=I%27m+the+author+of+alive-progress%2C+clearly+and+about-time.+Thank+you+for+appreciating+my+work%21\u0026currency_code=USD)\n\n---\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsalmei%2Frefine","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frsalmei%2Frefine","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frsalmei%2Frefine/lists"}