{"id":13813487,"url":"https://github.com/boyter/hashit","last_synced_at":"2025-04-10T05:10:06.172Z","repository":{"id":46537559,"uuid":"187121661","full_name":"boyter/hashit","owner":"boyter","description":"A cross platform tool to compute hashes of files quickly. Similar to hashdeep.","archived":false,"fork":false,"pushed_at":"2025-03-21T09:32:13.000Z","size":40681,"stargazers_count":74,"open_issues_count":3,"forks_count":8,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-04-10T05:09:37.410Z","etag":null,"topics":["blake2","blake2b","blake3","golang","hash","hashdeep","hashing","md5","md5sum","sha1sum","sha256","sha256sum","sha512sum"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/boyter.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-05-17T00:51:16.000Z","updated_at":"2025-04-08T07:20:36.000Z","dependencies_parsed_at":"2024-06-19T01:34:57.704Z","dependency_job_id":"4309aece-7999-4b38-94fc-b86862588700","html_url":"https://github.com/boyter/hashit","commit_stats":{"total_commits":127,"total_committers":2,"mean_commits":63.5,"dds":"0.023622047244094446","last_synced_commit":"aa2ad3c830e840948bed11f1bb32056d6cc1c583"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fhashit","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fhashit/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fhashit/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/boyter%2Fhashit/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/boyter","download_url":"https://codeload.github.com/boyter/hashit/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248161274,"owners_count":21057555,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["blake2","blake2b","blake3","golang","hash","hashdeep","hashing","md5","md5sum","sha1sum","sha256","sha256sum","sha512sum"],"created_at":"2024-08-04T04:01:19.254Z","updated_at":"2025-04-10T05:10:06.124Z","avatar_url":"https://github.com/boyter.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"Hash It!\n--------\n\nA hash tool which can work like hashdeep or md5sum, sha1sum, etc... When you want to find the hash or hashes of a file quickly, cross platform using a single command.\n\nYes the name is *very intentional* and similar to what say when I realise I need to get multiple hashes of a large file. This tool attempts to solve that pain.\n\n\n[![Go](https://github.com/boyter/hashit/actions/workflows/go.yml/badge.svg)](https://github.com/boyter/hashit/actions/workflows/go.yml)\n[![Scc Count Badge](https://sloc.xyz/github/boyter/hashit/)](https://github.com/boyter/hashit/)\n[![Go Report Card](https://goreportcard.com/badge/github.com/boyter/hashit)](https://goreportcard.com/report/github.com/boyter/hashit)\n\nLicensed under the MIT license.\n\nOther similar projects,\n\n - [hashdeep](https://github.com/jessek/hashdeep) originally called md5deep\n\n### Support\n\nUsing `hashit` commercially? If you want priority support for `hashit` you can purchase a years worth https://app.gumroad.com/products/mcpni which entitles you to priority direct email support from the developer.\n \n### Install\n\nIf you are comfortable using Go and have \u003e= 1.16 installed the usual `go get -u github.com/boyter/hashit/` will install for you.\n\nBinaries will be on [Releases](https://github.com/boyter/hashit/releases) for Windows, GNU/Linux and macOS for both i386 and x86_64 bit machines once it hits version 1.0.0.\n\nIf you would like to assist with getting `hashit` added into apt/homebrew/chocolatey/etc... please submit a PR or at least raise an issue with instructions.\n\n### Development\n\nYou need to have Go installed. Minimum version is Go 1.24 https://go.dev/\n\nInstall the following tools, either via the indicated command or what is suggested on site\n\n- sqlc brew install sqlc https://sqlc.dev/\n\nsqlc is used for the audit functionality as well as the SQLite output format. If you never change this functionality\nit may not be required, however _never_ edit the `./processor/db/` files directly.\n\n### Pitch\n\nWhy use `hashit`?\n\n - It is very fast\n - You can get multiple hashes \"for free\" on any CPU with multiple cores\n - Works very well across multiple platforms without slowdown (Windows, Linux, macOS)\n - Supports many hashes `hashit --hashes` CRC32, xxHash64, MD4, MD5, SHA1, SHA256, SHA512, Blake2b-256, Blake2b-512, Blake3, SHA3-224, SHA3-256, SHA3-384, SHA3-512\n - Output is compatible with `hashdeep`\n\n### Usage\n\nCommand line usage of `hashit` is designed to be as simple as possible.\nFull details can be found in `hashit --help` or `hashit -h`.\n\n```\n$ hashit -h\nHash It!\nVersion 1.2.0\nBen Boyter \u003cben@boyter.org\u003e\n\nUsage:\n  hashit [flags]\n\nFlags:\n      --debug             enable debug output\n  -f, --format string     set output format [text, json, sum, hashdeep, hashonly] (default \"text\")\n  -c, --hash strings      hashes to be run for each file (set to 'all' for all possible hashes) (default [md5,sha1,sha256,sha512])\n      --hashes            list all supported hashes\n  -h, --help              help for hashit\n      --no-stream         do not stream out results as processed\n  -o, --output string     output filename (default stdout)\n  -p, --progress          display progress of files as they are processed\n  -r, --recursive         recursive subdirectories are traversed\n      --stream-size int   min size of file in bytes where stream processing starts (default 1000000)\n      --threads int       number of threads processing files, by default the number of CPU cores (default 8)\n      --trace             enable trace output\n  -v, --verbose           verbose output\n      --version           version for hashit\n```\n\nOutput should look something like the below for operations on this repository\n\n```\n$ hashit README.md\nREADME.md (6616 bytes)\n        MD5 17cad37b7b873eed74e15cafd7855f6d\n       SHA1 8e4256ef60302acf72e9c25bea5a41e195cf14ec\n     SHA256 5ff9cd136f827570acc98be42d5a76a9409c83e3b9300ff2466de48c71760ca0\n     SHA512 e4d108219def5c36089f7cd45d4531548d52a41cc5992004c78e816fbe64d63a21bc7ea1303d7a31bd693bf4f5435c916fbbe4e9d3e1fd0b1982a9734b4ec739\n\n$ hashit --hash all README.md\nREADME.md (8964 bytes)\n      CRC32 a6f59c84\n   xxHash64 d3b87287c02996de\n        MD4 f106887bd63a7b7c5269a039c501b6b6\n        MD5 22ed507c2a5fc7b822c8080340d799ab\n       SHA1 2e74fa58da1f19b66b5826cf73d0c28a6c48aed6\n     SHA256 c84475732426797adfc3b1f14717d1aeeee976ca19ddd596f2ea5e59c932a062\n     SHA512 4f77e1d43cfcdb04c1ad99b5d53087b960ea7301da21f76de42cfefe77eb6f0bb3d2781db6c99f3f99d161e76c68d5a74a990b81e4e908a7104aef7d512efd31\nBlake2b-256 59627f6afdb0bcd783a9fe1f77763ff87c5768135e839fd9f16b440828f2a8e6\nBlake2b-512 9d00cd557c20b15ee85fdcc292d53604f4783d79dde6d3aada67391423633f28a1e5279519260b85171558bafc4b3785adc1854b88b004e402e49962426d86b4\n     Blake3 dcef77d73bc12c8d931b253835eae13854d18b8b16c784c089929815592a4506\n   SHA3-224 3912468ee2c0dfe3a4bf5f2bf4c2c36b684a465a73445d9651cf5e4d\n   SHA3-256 ca7bcbd1ad389727a2502cfd5b785968ebff7453551eb44ffd1e879f47fc00dc\n   SHA3-384 b97839411747e2174ed9d4dbf415e7945160746962f3208a4fd05ded8013c569da66db2fa8e8001c438896e160a153c7\n   SHA3-512 be6ac69cbb38fd2dc2646e7ec3e04878c0bc8824fe196da63ac577fae35a026251ef59618e9584bc2fbb274424a6a5251aea0ab64148469eb1b704a6c836627e\n\n\n$ hashit scripts\nscripts/include.go (1835 bytes)\n        MD5 73d7180f48af0b44e4ca7ae650335ac1\n       SHA1 dcd8b23288c1604c7b541c7626ae569bec9001b6\n     SHA256 929750fcace21c4a261d19824f691eae8691958989e8c1870b0934cfa9493462\n     SHA512 b37ac5a309f9006b740fb0933fe5c4569923cab0fe822c1e2fbf0fbd2a15e9787681ec509ca9f7ea13d921a82257ecc3a32e2dfa18cc6892ea82978befe2629c\n```\n\nhashit can produce `hashdeep` compatible audit files if you want to verify its output,\n\n```\n$ hashit --format hashdeep processor\n%%%% HASHDEEP-1.0\n%%%% size,md5,sha256,filename\n## Invoked from: /home/bboyter/Go/src/github.com/boyter/hashit\n## $ hashit --format hashdeep processor\n##\n12093,4bf92524edf098a6f5ada7b9ce6ae933,54be4e99f2635c00c6eb769a0342c2c040eac9b4f10627233e6dea8b9b20981b,processor/constants.go\n18786,5b0971442f17ae00b7ad6087855d5089,b1074780eee33b1c7d548b1b94c6743691dcbc5c7d475d685c9ca77a8b7905ba,processor/workers.go\n5856,58043d636928a4c0e7e6a04e69d385de,12f0e925a67d10da9327f11976c9156ba158458874d5d6fde632c27e27dead67,processor/processor.go\n758,76697adfb4c818d816d3092c04fdeb46,af61af65db73a2aec2d2bea66468d9e7c44bc92bade2561754b426484a7f235b,processor/file.go\n8883,8865704b023f417ffa2d6d347f5d2164,56a04cd56fb30b4ccb7b1344fbee119607b514eac57c99222dbe1319020adb5a,processor/formatters.go\n444,22158f610b8a262ca8ae68424ad96ad9,47f026a06d7ced7ecbbdaf199926678cbc003b7a387eb9bbee78a2a0340297bf,processor/structs.go\n2840,ce29ce9a95713628e1d8e43a51027ac1,7dcc785a34ce95c4e741e92177f221e6d05d9c1663481f35c54286fc6645934f,processor/workers_test.go\n```\n\nThe output of the above can be run through hashdeep for verification,\n\n```\n$ hashit --format hashdeep processor \u003e audit.txt \u0026\u0026 hashdeep -l -r -a -v -k audit.txt processor\nhashdeep: Audit passed\n          Files matched: 7\nFiles partially matched: 0\n            Files moved: 0\n        New files found: 0\n  Known files not found: 0\n```\n\nNote that you don't have to specify the directory you want to run against. Running `hashit` will assume you want to run against the current directory.\n\nIf you supply a single argument to `hashit` and its a file it will process it. If you supply a single argument and it is a directory it will recurse that directory.\n\nIf you supply multiple arguments which consist of files and directories then directories will be skipped. This allows wild card's to be used and just process the files in a directory.\n\n```\n$ hashit *\nGopkg.toml (736 bytes)\n        MD5 3e88135ebf43e8199ce0c19c8bebd925\n       SHA1 c3324f86a84c7a3ee941fcf94d221877aa25e799\n     SHA256 712667bfe27e245e0f34b2d64ad6b0d21580c058a01ce8374e5b6fa408a97f66\n     SHA512 d897a428487e31143b1b1ba9b3fa237bc9ed3aac7ad15ecd9507b7f58eaf5f1059385ebfcbeed4505e2cbf63113f70a0252acbf9f738f25fc79e563e7c8b32b6\n\nGopkg.lock (1514 bytes)\n        MD5 d19eb48da41406376a361a1c4a97e496\n       SHA1 977b14e5ed765137c0e146789ed37c52a8cd51ee\n     SHA256 5b76da627bd28a7bc7459b3281e3582616b21c4694302ae00b8888689effe35c\n     SHA512 95b9450cf19a3ef5c287841b3606ac7a1ad3066308a638fb8be908d649360a656ae41291e657e6f4250c87b0e18e05d73fca111fede9ed496eb600afdb245a0d\n\nREADME.md (7094 bytes)\n        MD5 df97688a1cb29d4b06d630df034b6e16\n       SHA1 d1c94c30c97f8fac75fe485e8352da813c4d4bb7\n     SHA256 78ce6718977135593fbd9867d6c2a68f91aa6d2846fe7bf08cbe9924790517b9\n     SHA512 759a3ef175466639fdc5bfbd1a8a47087410148405b15a3ac4bc0c88e677207f7c50f62cee947dabbbcf739c1031f3ff1ecdfd1688d83f566c75068b11a1f680\n\n... OUTPUT SNIPPED ...\n```\n\nIf you are running hashit on a slower mechanical HDD you may want to limit the number of threads which read files from\ndisk using `--threads 1`\n\n```shell\n$ hashit --threads 1 /mnt/slowdisk/\n```\n\nFor large files you can use `-p` to see the progress of the file to get an idea of how long it might take to process.\n\n```shell\n$ hashit -p --threads 1 large.file\n[==================================\u003e---------------------------------] file: large.file\n```\n\n\n#### Misc stuff below\n\nUsage of hashdeep\n\nhttps://linhost.info/2010/05/using-hashdeep-to-ensure-data-integrity/\n\nIssues to address\n\nhttps://github.com/jessek/hashdeep/issues/4\nhttps://github.com/jessek/hashdeep/issues/358\n\nExample usage\n\nhashdeep -r vendor \u003e audit.txt\nhashdeep -r -a -k audit.txt vendor\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboyter%2Fhashit","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fboyter%2Fhashit","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fboyter%2Fhashit/lists"}