{"id":15907949,"url":"https://github.com/thequantumphysicist/thash","last_synced_at":"2025-03-21T21:31:48.239Z","repository":{"id":257801630,"uuid":"862022611","full_name":"TheQuantumPhysicist/thash","owner":"TheQuantumPhysicist","description":"A terminal hashing program with many hashing algorithms and many output formats, and recursive hashing support","archived":false,"fork":false,"pushed_at":"2024-10-04T18:10:09.000Z","size":130,"stargazers_count":6,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-18T05:34:30.337Z","etag":null,"topics":["blake2b","blake2s","blake3","checksum","hash","hashing","md5","performance","rust","rust-lang","sha1","sha256","sha3-256","sha3-384","sha3-512","sha384","sha512"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/TheQuantumPhysicist.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-23T22:38:46.000Z","updated_at":"2024-10-04T18:10:12.000Z","dependencies_parsed_at":null,"dependency_job_id":"e05db7e4-385e-4acd-b1ce-1a281f82e124","html_url":"https://github.com/TheQuantumPhysicist/thash","commit_stats":{"total_commits":48,"total_committers":1,"mean_commits":48.0,"dds":0.0,"last_synced_commit":"043832b8969102f47f394f8fd55356546d1bc544"},"previous_names":["thequantumphysicist/thash"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheQuantumPhysicist%2Fthash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheQuantumPhysicist%2Fthash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheQuantumPhysicist%2Fthash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/TheQuantumPhysicist%2Fthash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/TheQuantumPhysicist","download_url":"https://codeload.github.com/TheQuantumPhysicist/thash/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244874373,"owners_count":20524577,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blake2b","blake2s","blake3","checksum","hash","hashing","md5","performance","rust","rust-lang","sha1","sha256","sha3-256","sha3-384","sha3-512","sha384","sha512"],"created_at":"2024-10-06T14:07:06.742Z","updated_at":"2025-03-21T21:31:48.204Z","avatar_url":"https://github.com/TheQuantumPhysicist.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# thash\n\nA fast and versatile terminal hashing program that prints the hash digest to stdout, with many hashing algorithms, many output formats, and support to request the number of iterations for hashing.\n\nThis program can be seen as a drop-in replacement for all `XXXsum` programs on Linux, usage-wise, assuming the defaults are desired.\n\n## Why?\n\nBasically I wanted something more flexible than `sha256sum`, `md5sum` and the other similar terminal tools. It always helps to have a tool that can do multiple things with hashing, so I spent some time writing a good, reliable program.\n\n## Installation\n\nYou can install this with [cargo](https://www.rust-lang.org/tools/install).\n\n```bash\ncargo install thash\n```\n\nCurrently it's only available on crates.io with cargo. If it gets popular enough, I'll happily upload it elsewhere.\n\n## Usage\n\nRun `thash --help`, to see all available options, algorithms, etc.\n\nThe data input is passed through stdin. This program can be seen as a drop-in replacement for all `XXXsum` programs on Linux, usage-wise, assuming the defaults are desired.\n\nFor platforms that don't have commands like `cat` to pass a file through stdin, you can use the `-f` command line argument to specify the file, instead of using stdin. If no `-f` is specified, stdin will be used.\n\n### Examples\n\nAnyway, here are some examples on how to use this program, with the expected outcome:\n\n- Hash the string \"abc\" using blake2b (the default hashing algorithm), and output the hash digest as hex (default output format).\n\n```bash\n$ echo -n \"abc\" | thash\nba80a53f981c4d0d6a2797b69f12f6e94c212f14685ac4b74b12bb6fdbffa2d17d87c5392aab792dc252d5de4533cc9518d38aa8dbf1925ab92386edd4009923\n```\n\nNotice that we use `-n` with `echo` to avoid adding a new line at the end.\n\n- Hash the string \"abc\" using sha256, and output the hash digest as hex. Remember: The list of all available algorithms can be viewed with `--help`.\n\n```bash\n$ echo -n \"abc\" | thash -a sha256\nba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad\n```\n\n- Same, but now as different types of outputs\n\n```bash\n$ echo -n \"abc\" | thash -a sha256 -F hex-lower\nba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad\n\n$ echo -n \"abc\" | thash -a sha256 -F hex-upper\nBA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD\n\n$ echo -n \"abc\" | thash -a sha256 -F base64\nungWv48Bz+pBQUDeXa4iI7ADYaOWF3qctBD/YfIAFa0=\n\n$ echo -n \"abc\" | thash -a sha256 -F base64-no-pad\nungWv48Bz+pBQUDeXa4iI7ADYaOWF3qctBD/YfIAFa0\n\n$ echo -n \"abc\" | thash -a sha256 -F base64-url-safe\nungWv48Bz-pBQUDeXa4iI7ADYaOWF3qctBD_YfIAFa0=\n\n$ echo -n \"abc\" | thash -a sha256 -F base64-url-safe-no-pad\nungWv48Bz-pBQUDeXa4iI7ADYaOWF3qctBD_YfIAFa0\n```\n\nYou can even write the output as binary with `-F binary`! Obviously, this isn't properly visible in a text file. But you can pipe it to other programs.\n\n- How about hashing \"abc\" twice using sha256, instead of once?\n\n```bash\n$ echo -n \"abc\" | thash -a sha256 -i2\n4f8b42c22dd3729b519ba6f68d2da7cc5b2d606d05daed5ad5128cc03e6c6358\n```\n\nThis will take the hash digest output, and hash it again with sha256.\n\n- In fact, this is equivalent to this:\n\n```bash\n$ echo -n \"abc\" | thash -a sha256 -F binary | thash -a sha256\n4f8b42c22dd3729b519ba6f68d2da7cc5b2d606d05daed5ad5128cc03e6c6358\n```\n\nNotice how outputting binary is now useful.\n\n- You can pipe your data through multiple hash functions.\n\n```bash\n$ echo -n \"abc\" | thash -a sha256 -F binary | thash -a blake2s -F binary | thash -a sha3-224\nc287a7a4abc221b50aa406b6b6e47017f0bb5bc354870912fc00588d\n```\n\nThis will hash \"abc\" with sha256, hash the binary result with blake2s, then hash the binary result with sha3-224, and print the final result.\n\n- Some algorithms provide options. For example, K12 hashing algorithm provides an option to specify the size of the output.\n\n```bash\n$ echo -n \"abc\" | thash -a k12 -o output-size=64\nab174f328c55a5510b0b209791bf8b60e801a7cfc2aa42042dcb8f547fbe3a7d3f5b54d116a705d36aac2a7eac7a19e3f0f058cb3c238ac7f034178ae34f212e\n```\n\nTo see the available options, input some random value, and the list will be specified\n\n```bash\n$ echo -n \"abc\" | thash -a k12 -o abc=xyz\nError: Option abc is not a valid option for algorithm K12. Available options:\n\n+-------------+----------------------------------------------+\n| Option name | Description                                  |\n+-------------+----------------------------------------------+\n| output-size | The size of the output as a positive integer |\n+-------------+----------------------------------------------+\n```\n\n## Performance\n\n`thash` beats all regular tools, like `sha256`, `b2sum`, etc, in performance. On Mac, the performance of `thash` is even better than shown below, and is faster for everything. However, `md5` is the only exception we found, where `thash` is slower on Linux. Note that this is not gospel, and your machine/OS may behave differently. Feel free to re-run these benchmarks on your computer to evaluate it.\n\n### Benchmarking methodology\n\nWe use `hyperfine` for benchmarking. You can install it with `cargo install hyperfine`, or `apt install hyperfine` on Debian derivatives, or `brew install hyperfine` for MacOS, assuming you have homebrew.\n\nTo benchmark, first, create a file with random data (below is a 1 GB file), followed by some hashing program to ensure any disk-caching operation in the OS for the file is done:\n\n```bash\ndd if=/dev/random of=random-file.bin bs=1M count=1024\ncat random-file.bin | sha256sum\n```\n\n#### SHA256 performance\n\n```bash\n$ hyperfine \"cat random-file.bin | thash -a sha256\"\nBenchmark 1: cat random-file.bin | thash -a sha256\n  Time (mean ± σ):     670.7 ms ±   9.0 ms    [User: 499.6 ms, System: 589.4 ms]\n  Range (min … max):   654.1 ms … 684.2 ms    10 runs\n\n$ hyperfine \"cat random-file.bin | sha256sum\"\nBenchmark 1: cat random-file.bin | sha256sum\n  Time (mean ± σ):      2.928 s ±  0.050 s    [User: 2.795 s, System: 0.497 s]\n  Range (min … max):    2.841 s …  3.016 s    10 runs\n```\n\n#### SHA512 performance\n\n```bash\n$ hyperfine \"cat random-file.bin | thash -a sha512\"\nBenchmark 1: cat random-file.bin | thash -a sha512\n  Time (mean ± σ):      1.653 s ±  0.053 s    [User: 1.491 s, System: 0.629 s]\n  Range (min … max):    1.608 s …  1.785 s    10 runs\n\n$ hyperfine \"cat random-file.bin | sha512sum\"\nBenchmark 1: cat random-file.bin | sha512sum\n  Time (mean ± σ):      2.114 s ±  0.018 s    [User: 1.982 s, System: 0.488 s]\n  Range (min … max):    2.088 s …  2.143 s    10 runs\n```\n\n#### Blake2b performance\n\n```bash\n$ hyperfine \"cat random-file.bin | thash -a blake2b\"\nBenchmark 1: cat random-file.bin | thash -a blake2b\n  Time (mean ± σ):     970.0 ms ±  18.7 ms    [User: 796.6 ms, System: 610.0 ms]\n  Range (min … max):   951.9 ms … 1011.0 ms    10 runs\n\n$ hyperfine \"cat random-file.bin | b2sum\"\nBenchmark 1: cat random-file.bin | b2sum\n  Time (mean ± σ):      1.060 s ±  0.020 s    [User: 0.927 s, System: 0.480 s]\n  Range (min … max):    1.036 s …  1.093 s    10 runs\n```\n\n#### SHA-1 sum\n\n```bash\n$ hyperfine \"cat random-file.bin | thash -a sha1\"\nBenchmark 1: cat random-file.bin | thash -a sha1\n  Time (mean ± σ):     652.3 ms ±   7.3 ms    [User: 476.8 ms, System: 600.5 ms]\n  Range (min … max):   639.8 ms … 662.3 ms    10 runs\n\n$ hyperfine \"cat random-file.bin | sha1sum\"\nBenchmark 1: cat random-file.bin | sha1sum\n  Time (mean ± σ):      1.247 s ±  0.029 s    [User: 1.120 s, System: 0.472 s]\n  Range (min … max):    1.209 s …  1.288 s    10 runs\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthequantumphysicist%2Fthash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthequantumphysicist%2Fthash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthequantumphysicist%2Fthash/lists"}