https://github.com/thequantumphysicist/thash
A terminal hashing program with many hashing algorithms and many output formats, and recursive hashing support
https://github.com/thequantumphysicist/thash
blake2b blake2s blake3 checksum hash hashing md5 performance rust rust-lang sha1 sha256 sha3-256 sha3-384 sha3-512 sha384 sha512
Last synced: about 1 month ago
JSON representation
A terminal hashing program with many hashing algorithms and many output formats, and recursive hashing support
- Host: GitHub
- URL: https://github.com/thequantumphysicist/thash
- Owner: TheQuantumPhysicist
- License: mit
- Created: 2024-09-23T22:38:46.000Z (7 months ago)
- Default Branch: master
- Last Pushed: 2024-10-04T18:10:09.000Z (7 months ago)
- Last Synced: 2025-03-18T05:34:30.337Z (about 1 month ago)
- Topics: blake2b, blake2s, blake3, checksum, hash, hashing, md5, performance, rust, rust-lang, sha1, sha256, sha3-256, sha3-384, sha3-512, sha384, sha512
- Language: Shell
- Homepage:
- Size: 127 KB
- Stars: 6
- Watchers: 1
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# thash
A fast and versatile terminal hashing program that prints the hash digest to stdout, with many hashing algorithms, many output formats, and support to request the number of iterations for hashing.
This program can be seen as a drop-in replacement for all `XXXsum` programs on Linux, usage-wise, assuming the defaults are desired.
## Why?
Basically I wanted something more flexible than `sha256sum`, `md5sum` and the other similar terminal tools. It always helps to have a tool that can do multiple things with hashing, so I spent some time writing a good, reliable program.
## Installation
You can install this with [cargo](https://www.rust-lang.org/tools/install).
```bash
cargo install thash
```
Currently it's only available on crates.io with cargo. If it gets popular enough, I'll happily upload it elsewhere.
## Usage
Run `thash --help`, to see all available options, algorithms, etc.
The data input is passed through stdin. This program can be seen as a drop-in replacement for all `XXXsum` programs on Linux, usage-wise, assuming the defaults are desired.
For platforms that don't have commands like `cat` to pass a file through stdin, you can use the `-f` command line argument to specify the file, instead of using stdin. If no `-f` is specified, stdin will be used.
### Examples
Anyway, here are some examples on how to use this program, with the expected outcome:
- Hash the string "abc" using blake2b (the default hashing algorithm), and output the hash digest as hex (default output format).
```bash
$ echo -n "abc" | thash
ba80a53f981c4d0d6a2797b69f12f6e94c212f14685ac4b74b12bb6fdbffa2d17d87c5392aab792dc252d5de4533cc9518d38aa8dbf1925ab92386edd4009923
```
Notice that we use `-n` with `echo` to avoid adding a new line at the end.
- Hash the string "abc" using sha256, and output the hash digest as hex. Remember: The list of all available algorithms can be viewed with `--help`.
```bash
$ echo -n "abc" | thash -a sha256
ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad
```
- Same, but now as different types of outputs
```bash
$ echo -n "abc" | thash -a sha256 -F hex-lower
ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad
$ echo -n "abc" | thash -a sha256 -F hex-upper
BA7816BF8F01CFEA414140DE5DAE2223B00361A396177A9CB410FF61F20015AD
$ echo -n "abc" | thash -a sha256 -F base64
ungWv48Bz+pBQUDeXa4iI7ADYaOWF3qctBD/YfIAFa0=
$ echo -n "abc" | thash -a sha256 -F base64-no-pad
ungWv48Bz+pBQUDeXa4iI7ADYaOWF3qctBD/YfIAFa0
$ echo -n "abc" | thash -a sha256 -F base64-url-safe
ungWv48Bz-pBQUDeXa4iI7ADYaOWF3qctBD_YfIAFa0=
$ echo -n "abc" | thash -a sha256 -F base64-url-safe-no-pad
ungWv48Bz-pBQUDeXa4iI7ADYaOWF3qctBD_YfIAFa0
```
You can even write the output as binary with `-F binary`! Obviously, this isn't properly visible in a text file. But you can pipe it to other programs.
- How about hashing "abc" twice using sha256, instead of once?
```bash
$ echo -n "abc" | thash -a sha256 -i2
4f8b42c22dd3729b519ba6f68d2da7cc5b2d606d05daed5ad5128cc03e6c6358
```
This will take the hash digest output, and hash it again with sha256.
- In fact, this is equivalent to this:
```bash
$ echo -n "abc" | thash -a sha256 -F binary | thash -a sha256
4f8b42c22dd3729b519ba6f68d2da7cc5b2d606d05daed5ad5128cc03e6c6358
```
Notice how outputting binary is now useful.
- You can pipe your data through multiple hash functions.
```bash
$ echo -n "abc" | thash -a sha256 -F binary | thash -a blake2s -F binary | thash -a sha3-224
c287a7a4abc221b50aa406b6b6e47017f0bb5bc354870912fc00588d
```
This will hash "abc" with sha256, hash the binary result with blake2s, then hash the binary result with sha3-224, and print the final result.
- Some algorithms provide options. For example, K12 hashing algorithm provides an option to specify the size of the output.
```bash
$ echo -n "abc" | thash -a k12 -o output-size=64
ab174f328c55a5510b0b209791bf8b60e801a7cfc2aa42042dcb8f547fbe3a7d3f5b54d116a705d36aac2a7eac7a19e3f0f058cb3c238ac7f034178ae34f212e
```
To see the available options, input some random value, and the list will be specified
```bash
$ echo -n "abc" | thash -a k12 -o abc=xyz
Error: Option abc is not a valid option for algorithm K12. Available options:
+-------------+----------------------------------------------+
| Option name | Description |
+-------------+----------------------------------------------+
| output-size | The size of the output as a positive integer |
+-------------+----------------------------------------------+
```
## Performance
`thash` beats all regular tools, like `sha256`, `b2sum`, etc, in performance. On Mac, the performance of `thash` is even better than shown below, and is faster for everything. However, `md5` is the only exception we found, where `thash` is slower on Linux. Note that this is not gospel, and your machine/OS may behave differently. Feel free to re-run these benchmarks on your computer to evaluate it.
### Benchmarking methodology
We use `hyperfine` for benchmarking. You can install it with `cargo install hyperfine`, or `apt install hyperfine` on Debian derivatives, or `brew install hyperfine` for MacOS, assuming you have homebrew.
To benchmark, first, create a file with random data (below is a 1 GB file), followed by some hashing program to ensure any disk-caching operation in the OS for the file is done:
```bash
dd if=/dev/random of=random-file.bin bs=1M count=1024
cat random-file.bin | sha256sum
```
#### SHA256 performance
```bash
$ hyperfine "cat random-file.bin | thash -a sha256"
Benchmark 1: cat random-file.bin | thash -a sha256
Time (mean ± σ): 670.7 ms ± 9.0 ms [User: 499.6 ms, System: 589.4 ms]
Range (min … max): 654.1 ms … 684.2 ms 10 runs
$ hyperfine "cat random-file.bin | sha256sum"
Benchmark 1: cat random-file.bin | sha256sum
Time (mean ± σ): 2.928 s ± 0.050 s [User: 2.795 s, System: 0.497 s]
Range (min … max): 2.841 s … 3.016 s 10 runs
```
#### SHA512 performance
```bash
$ hyperfine "cat random-file.bin | thash -a sha512"
Benchmark 1: cat random-file.bin | thash -a sha512
Time (mean ± σ): 1.653 s ± 0.053 s [User: 1.491 s, System: 0.629 s]
Range (min … max): 1.608 s … 1.785 s 10 runs
$ hyperfine "cat random-file.bin | sha512sum"
Benchmark 1: cat random-file.bin | sha512sum
Time (mean ± σ): 2.114 s ± 0.018 s [User: 1.982 s, System: 0.488 s]
Range (min … max): 2.088 s … 2.143 s 10 runs
```
#### Blake2b performance
```bash
$ hyperfine "cat random-file.bin | thash -a blake2b"
Benchmark 1: cat random-file.bin | thash -a blake2b
Time (mean ± σ): 970.0 ms ± 18.7 ms [User: 796.6 ms, System: 610.0 ms]
Range (min … max): 951.9 ms … 1011.0 ms 10 runs
$ hyperfine "cat random-file.bin | b2sum"
Benchmark 1: cat random-file.bin | b2sum
Time (mean ± σ): 1.060 s ± 0.020 s [User: 0.927 s, System: 0.480 s]
Range (min … max): 1.036 s … 1.093 s 10 runs
```
#### SHA-1 sum
```bash
$ hyperfine "cat random-file.bin | thash -a sha1"
Benchmark 1: cat random-file.bin | thash -a sha1
Time (mean ± σ): 652.3 ms ± 7.3 ms [User: 476.8 ms, System: 600.5 ms]
Range (min … max): 639.8 ms … 662.3 ms 10 runs
$ hyperfine "cat random-file.bin | sha1sum"
Benchmark 1: cat random-file.bin | sha1sum
Time (mean ± σ): 1.247 s ± 0.029 s [User: 1.120 s, System: 0.472 s]
Range (min … max): 1.209 s … 1.288 s 10 runs
```