https://github.com/rtmigo/litehash_py
Fast coarse hashes from files
https://github.com/rtmigo/litehash_py
crc32 fast fibonacci file hashing md5 python
Last synced: about 1 year ago
JSON representation
Fast coarse hashes from files
- Host: GitHub
- URL: https://github.com/rtmigo/litehash_py
- Owner: rtmigo
- License: mit
- Created: 2021-07-13T22:40:16.000Z (almost 5 years ago)
- Default Branch: staging
- Last Pushed: 2022-06-20T14:34:05.000Z (almost 4 years ago)
- Last Synced: 2025-02-21T22:37:47.360Z (over 1 year ago)
- Topics: crc32, fast, fibonacci, file, hashing, md5, python
- Language: Python
- Homepage: https://pypi.org/project/litehash/
- Size: 248 KB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# [litehash](https://github.com/rtmigo/litehash_py)
Calculates fast coarse hashes from files. Instead of hashing entire files, we take individual bytes, add the size of the file, and calculate the hash from that.
This allows you to get hashes of large files very quickly. However, it
may not display small differences in file content.
## Install
``` bash
$ pip3 install litehash
```
## Use
### Fibonacci
Calculate the checksum based on the bytes located at an increasing distance from
each other, and the file size. We will read more bytes from the
file header than from the body.
``` python3
from pathlib import Path
from litehash import file_to_hash_fibonacci
print(file_to_hash_fibonacci(Path('/path/to/file.dat')))
```
### Equidistant
Calculate the checksum based on ten equidistant bytes, and the file size.
The very first and the very last byte of the file will be among the ten read.
``` python3
from pathlib import Path
from litehash import file_to_hash_equidistant
print(file_to_hash_equidistant(Path('/path/to/file.dat'), n=10))
```
### HashAlgo
The optional `HashAlgo` argument allows you to select a hashing algorithm.
By default, MD5 is used as a compromise between speed and size.
| Algorithm | Digest Size (bytes) |
|-------------------|---------------------|
| `HashAlgo.crc32` | 4 |
| `HashAlgo.md5` | 24 |
| `HashAlgo.sha256` | 32 |
```python3
from pathlib import Path
from litehash import file_to_hash_fibonacci, HashAlgo
print(file_to_hash_fibonacci(Path('/path/to/file.dat'), algo=HashAlgo.crc32))
```