https://github.com/psychedelicious/hash_bench
https://github.com/psychedelicious/hash_bench
Last synced: over 1 year ago
JSON representation
- Host: GitHub
- URL: https://github.com/psychedelicious/hash_bench
- Owner: psychedelicious
- Created: 2024-02-26T06:04:11.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-03-04T12:20:37.000Z (over 2 years ago)
- Last Synced: 2025-01-25T13:22:22.085Z (over 1 year ago)
- Language: Python
- Size: 6.84 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# hash bench
Simple script to benchmark file hashing algorithms in python. The files are stable diffusion models.
## Setup
Requires `blake3` and `tqdm`. The rest is python builtins.
```sh
pip install -r requirements.txt
```
## Algorithms
You can easily add any `hashlib` algorithm in a naive (read the whole file into memory at once) or optimized (via memory view) implementations.
Update `hash_functions` to test others:
```py
hash_functions = {
# Any `hashlib` algorithm works here
"SHA1_naive": get_hashlib_naive("sha1"),
"SHA1_mv": get_hashlib_mv("sha1"),
"MD5_naive": get_hashlib_naive("md5"),
"MD5_mv": get_hashlib_mv("md5"),
# BLAKE3 is not in `hashlib`, must use the provided functions
"BLAKE3_mmap": blake3_mmap,
"BLAKE3_mv": blake3_mv,
}
```