{"id":31040512,"url":"https://github.com/moold/fastmd5","last_synced_at":"2025-09-14T08:53:22.178Z","repository":{"id":311851818,"uuid":"749716617","full_name":"moold/fastMD5","owner":"moold","description":"Print or check MD5 checksums.","archived":false,"fork":false,"pushed_at":"2025-08-27T01:40:26.000Z","size":1062,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-08-27T10:15:18.111Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/moold.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-01-29T08:55:47.000Z","updated_at":"2025-08-27T01:40:30.000Z","dependencies_parsed_at":"2025-08-27T10:15:23.182Z","dependency_job_id":"50152a9a-ef51-4947-9d59-812cd7ae33c3","html_url":"https://github.com/moold/fastMD5","commit_stats":null,"previous_names":["moold/fastmd5"],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/moold/fastMD5","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FfastMD5","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FfastMD5/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FfastMD5/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FfastMD5/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/moold","download_url":"https://codeload.github.com/moold/fastMD5/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/moold%2FfastMD5/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":275082360,"owners_count":25402339,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-14T02:00:10.474Z","response_time":75,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-09-14T08:53:19.925Z","updated_at":"2025-09-14T08:53:22.168Z","avatar_url":"https://github.com/moold.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![install with bioconda](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat)](http://bioconda.github.io/recipes/fastmd5/README.html)\n[![Conda Version](https://img.shields.io/conda/v/bioconda/fastMD5)](https://anaconda.org/bioconda/fastmd5)\n# fastMD5 - print or check MD5 checksums\n\n**fastMD5** is a high-performance MD5 checksum tool, designed as a faster and parallelized alternative to the standard `md5sum`. It offers flexible speed modes, multi-threading support, and efficient directory handling—making it especially well-suited for verifying very large files or entire directories.\n\n## Key Features\n\n- [x] 🚀 **Adjustable speed levels (0–9)**\n  * **0** – Full sequential computation, \\~20% faster than GNU `md5sum`, fully compatible with it.\n  * **1** – Full-file computation with optimized performance, up to **\u003e 4× faster** than GNU `md5sum`.\n  * **2–9** – Block-based sampling for approximate checksums. For example, a file \u003e100 GB can be processed in **under 1 second**.\n- [x] 🧵 **Multi-threaded execution** – Leverages multi-core CPUs for even faster performance.\n- [x] 📂 **Recursive directory support** – Optionally include hidden files and symbolic links.\n- [x] ✅ **Checksum verification mode (`--check`)** – Validate files against precomputed MD5 sums.\n\n\n## Getting Started\n\n```bash\n# Compute MD5 checksums at the default speed level (5), typically used to verify file integrity after copying.\nfastmd5 -t 10 file1.fastq.gz file2.fastq.gz data_directory/\n# Run in the most accurate mode (level 0, equivalent to `md5sum`).\nfastmd5 -t 3 -s 0 file1.fastq.gz file2.fastq.gz data_directory/\n# Run full-file computation with optimized performance (level 1).\nfastmd5 -t 5 -s 1 file1.fastq.gz file2.fastq.gz data_directory/\n# Verify checksums from a checksum file (equivalent to `md5sum -c`).\nfastmd5 --check checksums.md5\n```\n\n## Table of Contents\n\n- [Installation](#install)\n- [General usage](#usage)\n- [Getting help](#help)\n- [Benchmarking](#benchmark)\n\n### \u003ca name=\"install\"\u003e\u003c/a\u003eInstallation\n#### Installing from bioconda\n```sh\nconda install bioconda::fastmd5\n```\n\n#### Installing from source\n##### Dependencies\n\n`fastMD5` is written in rust, try below commands (no root required) or refer [here](https://www.rust-lang.org/tools/install) to install `Rust` first.\n```sh\ncurl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh\n```\n\n##### Download and install\n国内用户请参考[这里](https://mirrors.tuna.tsinghua.edu.cn/help/crates.io-index/)设置清华源加速\n```sh\ngit clone https://github.com/moold/fastMD5.git\ncd fastMD5 \u0026\u0026 cargo build --release\n```\n\n##### Test\n\n```sh\ncd test \u0026\u0026 bash hh.sh\n```\n\n### \u003ca name=\"usage\"\u003e\u003c/a\u003eGeneral usage\n#### 1. Quickly verify the integrity of copied files\n```bash\n# Generate checksum file (default speed level 5)\nfastmd5 -t 10 file1.fastq.gz file2.fastq.gz data_directory/ \u003e checksums.md5  \n\n# Verify against checksum file\nfastmd5 --check checksums.md5\n```\n\n#### 2. Verify integrity of modified files (full-file mode)\n```bash\n# Generate checksum file (speed level 1, full-file computation)\nfastmd5 -t 10 -s 1 file1.fastq.gz file2.fastq.gz data_directory/ \u003e checksums.md5  \n\n# Verify against checksum file\nfastmd5 --check checksums.md5\n```\n\n#### 3. Generate results fully compatible with GNU `md5sum`\n```bash\n# Generate checksum file (speed level 0, GNU-compatible mode)\nfastmd5 -t 10 -s 0 file1.fastq.gz file2.fastq.gz data_directory/ \u003e checksums.md5  \n# Equivalent GNU command\nls file1.fastq.gz file2.fastq.gz data_directory/ | while read line; do md5sum $line \u003e\u003e checksums.md5; done  \n\n# Verify against checksum file\nfastmd5 --check checksums.md5  \n# Equivalent GNU command\nmd5sum -c checksums.md5\n```\nℹ️ Run `./target/release/fastMD5 -h` to view all available options.\n\n### \u003ca name=\"help\"\u003e\u003c/a\u003eGetting help\n#### Help\n\n   Feel free to raise an issue at the [issue page](https://github.com/moold/fastMD5/issues/new).\n\n   ***Note:*** Please ask questions on the issue page first. They are also helpful to other users.\n#### Contact\n   \n   For additional help, please send an email to huj1203\\_at\\_foxmail\\_dot\\_cn.\n\n\u003c!-- ### \u003ca name=\"cite\"\u003e\u003c/a\u003eCitation --\u003e\n\n### \u003ca name=\"benchmark\"\u003e\u003c/a\u003eBenchmarking\n\n| Program     | File Size (GB) | Speed Level (-s) | Wall Clock (s) | Speedup (×) |\n|:------------|---------------:|:----------------:|---------------:|------------:|\n| **GNU `md5sum`**| 1          | –                | 1.945          | 1.0×        |\n| `fastMD5`   | 1              | 0                | 1.594          | 1.22×       |\n| `fastMD5`   | 1              | 1                | 0.454          | 4.29×       |\n| `fastMD5`   | 1              | 2                | 0.164          | 11.85×      |\n| **GNU `md5sum`**| **10**     | –                | **19.347**     | **1.0×**    |\n| `fastMD5`   | 10             | 0                | 15.826         | 1.22×       |\n| `fastMD5`   | 10             | 1                | 4.387          | 4.41×       |\n| `fastMD5`   | 10             | 2                | 0.165          | 117.26×     |\n| **GNU `md5sum`**| **40**     | –                | **79.040**     | **1.0×**    |\n| `fastMD5`   | 40             | 0                | 63.150         | 1.25×       |\n| `fastMD5`   | 40             | 1                | 18.738         | 4.22×       |\n| `fastMD5`   | 40             | 2                | 0.173          | 456.65×     |\n| **GNU `md5sum`**| **90**     | –                | **179.552**    | **1.0×**    |\n| `fastMD5`   | 90             | 0                | 141.837        | 1.27×       |\n| `fastMD5`   | 90             | 1                | 43.649         | 4.11×       |\n| `fastMD5`   | 90             | 2                | 0.176          | 1020.27×    |\n\n***Note:*** Wall-clock times were measured using `hyperfine` with three warm-up runs on a single CPU core. \n\n### Star\nYou can track updates by tab the **Star** button on the upper-right corner at the [github page](https://github.com/moold/fastMD5).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoold%2Ffastmd5","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmoold%2Ffastmd5","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmoold%2Ffastmd5/lists"}