Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/leonidessaguisagjr/filehash

Python module that wraps around hashlib and zlib to facilitate generating checksums / hashes of files and directories.
https://github.com/leonidessaguisagjr/filehash

adler-32 checksum checksum-digests checksum-validator crc-32 crc32 md5 python python-library sfv sha1 sha1-hash sha1sum sha256 sha256-hash sha256sum sha512 sha512-hash sha512sum simple-file-verification

Last synced: 9 days ago
JSON representation

Python module that wraps around hashlib and zlib to facilitate generating checksums / hashes of files and directories.

Awesome Lists containing this project

README 

        
``filehash``

============



.. image:: https://img.shields.io/pypi/v/filehash.svg

        :target: https://pypi.python.org/pypi/filehash



.. image:: https://img.shields.io/github/workflow/status/leonidessaguisagjr/filehash/Python%20filehash

        :target: https://github.com/leonidessaguisagjr/filehash/actions/workflows/python-app.yml



Python module to facilitate calculating the checksum or hash of a file.  Tested against Python 2.7.x, Python 3.6.x, Python 3.7.x, Python 3.8.x, Python 3.9.x, Python 3.10.x, PyPy 2.7.x and PyPy3 3.7.x.  Currently supports `Adler-32 `_, `BLAKE2b, BLAKE2s `_, `CRC32 `_, `MD5 `_, `SHA-1 `_, `SHA-224, SHA-256, SHA-384 and SHA-512 `_.



(Note: BLAKE2b and BLAKE2s are only supported on Python 3.6.x and later.)



``FileHash`` class

------------------



The ``FileHash`` class wraps around the ``hashlib`` (provides hashing for MD5, SHA-1, SHA-224, SHA-256, SHA-384 and SHA-512) and ``zlib`` (provides checksums for Adler-32 and CRC32) modules and contains the following methods:



- ``hash_file(filename)`` - Calculate the file hash for a single file.  Returns a string with the hex digest.

- ``hash_files(filename)`` - Calculate the file hash for multiple files.  Returns a list of tuples where each tuple contains the filename and the calculated hash.

- ``hash_dir(path, pattern='*')`` - Calculate the file hashes for an entire directory.  Returns a list of tuples where each tuple contains the filename and the calculated hash.

- ``cathash_files(filenames)`` - Calculate a single hash for multiple files.  Files are sorted by their individual hash values and then traversed in that order to generate a combined hash value.  Returns a string with the hex digest.

- ``cathash_dir(path, pattern='*')`` - Calculate a single hash for an entire directory of files. Files are sorted by their individual hash values and then traversed in that order to generate a combined hash value. Returns a string with the hex digest.

- ``verify_sfv(sfv_filename)`` - Reads the specified SFV (Simple File Verification) file and calculates the CRC32 checksum for the files listed, comparing the calculated CRC32 checksums against the specified expected checksums.  Returns a list of tuples where each tuple contains the filename and a boolean value indicating if the calculated CRC32 checksum matches the expected CRC32 checksum.  To find out more about SFV files, see the `Simple file verification entry in Wikipedia `_.

- ``verify_checksums(checksum_filename)`` - Reads the specified file and calculates the hashes for the files listed, comparing the calculated hashes against the specified expected hashes.  Returns a list of tuples where each tuple contains the filename and a boolean value indicating if the calculated hash matches the expected hash.



For the checksum file, the file is expected to be a plain text file where each line has an entry formatted as follows::



   {hash}[SPACE][ASTERISK]{filename}



This format is the format used by programs such as the `sha1sum `_ family of tools for generating checksum files.  Here is an example generated by ``sha1sum``::



   f7ef3b7afaf1518032da1b832436ef3bbfd4e6f0 *lorem_ipsum.txt

   03da86258449317e8834a54cf8c4d5b41e7c7128 *lorem_ipsum.zip



The ``FileHash`` constructor has two optional arguments:



- ``hash_algorithm='sha256'`` - Specifies the hashing algorithm to use.  See ``filehash.SUPPORTED_ALGORITHMS`` for the list of supported hash / checksum algorithms.  Defaults to SHA256.

- ``chunk_size=4096`` - Integer specifying the chunk size to use (in bytes) when reading the file.  This comes in useful when processing very large files to avoid having to read the entire file into memory all at once.  Default chunk size is 4096 bytes.



Example usage

-------------



The library can be used as follows::



   >>> import os

   >>> from filehash import FileHash

   >>> md5hasher = FileHash('md5')

   >>> md5hasher.hash_file("./testdata/lorem_ipsum.txt")

   '72f5d9e3a5fa2f2e591487ae02489388'

   >>> sha1hasher = FileHash('sha1')

   >>> sha1hasher.hash_dir("./testdata", "*.zip")

   [FileHashResult(filename='lorem_ipsum.zip', hash='03da86258449317e8834a54cf8c4d5b41e7c7128')]

   >>> sha512hasher = FileHash('sha512')

   >>> os.chdir("./testdata")

   >>> sha512hasher.verify_checksums("./hashes.sha512")

   [VerifyHashResult(filename='lorem_ipsum.txt', hashes_match=True), VerifyHashResult(filename='lorem_ipsum.zip', hashes_match=True)]

   >>> crc32hasher = FileHash('crc32')

   >>> crc32hasher.verify_sfv("./lorem_ipsum.sfv")

   [VerifyHashResult(filename='lorem_ipsum.txt', hashes_match=True), VerifyHashResult(filename='lorem_ipsum.zip', hashes_match=True)]



``chkfilehash`` command line tool

---------------------------------



A command-line tool called ``chkfilehash`` is also included with the ``filehash`` package.  Here is an example of how the tool can be used::



   $ chkfilehash -a sha512 -c hashes.sha512

   lorem_ipsum.txt: OK

   lorem_ipsum.zip: OK

   $ chkfilehash -a crc32 lorem_ipsum.zip

   7425D3BE *lorem_ipsum.zip

   $



Run the tool without any parameters or with the ``-h`` / ``--help`` switch to get a usage screen.



License

-------



This is released under an MIT license.  See the ``LICENSE`` file in this repository for more information.