{"id":13672247,"url":"https://github.com/nickbabcock/highway-rs","last_synced_at":"2025-05-15T10:03:52.792Z","repository":{"id":41368139,"uuid":"144920459","full_name":"nickbabcock/highway-rs","owner":"nickbabcock","description":"Native Rust port of Google's HighwayHash, which makes use of SIMD instructions for a fast and strong hash function","archived":false,"fork":false,"pushed_at":"2025-04-04T12:15:29.000Z","size":3883,"stargazers_count":163,"open_issues_count":2,"forks_count":17,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-14T16:56:53.804Z","etag":null,"topics":["hash","highwayhash","rust","simd"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/highway","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nickbabcock.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-08-16T01:24:02.000Z","updated_at":"2025-04-14T08:56:48.000Z","dependencies_parsed_at":"2023-01-30T22:32:06.014Z","dependency_job_id":"0f8777c2-ab95-47a3-83d7-4786b112b404","html_url":"https://github.com/nickbabcock/highway-rs","commit_stats":{"total_commits":269,"total_committers":8,"mean_commits":33.625,"dds":"0.052044609665427455","last_synced_commit":"b884bfb795864dab82988a9a5cd61c9d65cf69e1"},"previous_names":[],"tags_count":21,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fhighway-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fhighway-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fhighway-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nickbabcock%2Fhighway-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nickbabcock","download_url":"https://codeload.github.com/nickbabcock/highway-rs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254319716,"owners_count":22051072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hash","highwayhash","rust","simd"],"created_at":"2024-08-02T09:01:30.295Z","updated_at":"2025-05-15T10:03:51.193Z","avatar_url":"https://github.com/nickbabcock.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"![ci](https://github.com/nickbabcock/highway-rs/workflows/ci/badge.svg) [![](https://docs.rs/highway/badge.svg)](https://docs.rs/highway) [![Rust](https://img.shields.io/badge/rust-1.59%2B-blue.svg?maxAge=3600)](https://github.com/nickbabcock/highway-rs) [![Version](https://img.shields.io/crates/v/highway.svg?style=flat-square)](https://crates.io/crates/highway)\n\n# Highway-rs\n\nThis crate is a native Rust port of [Google's HighwayHash](https://github.com/google/highwayhash), which is a fast, keyed, and strong hash function, whose output is hardware independent.\n\n## Features\n\n - ✔ pure / stable rust\n - ✔ zero dependencies\n - ✔ generate consistent 64, 128, and 256bit hashes across all hardware\n - ✔ \u003e 10 GB/s with SIMD (SSE 4.1 AVX 2, NEON) aware instructions on x86 and aarch64 architectures\n - ✔ \u003e 3 GB/s on Wasm with the Wasm SIMD extension\n - ✔ \u003e 1 GB/s hardware agnostic implementation with zero unsafe code\n - ✔ incremental / streaming hashes that can be checkpointed and restored\n - ✔ zero heap allocations\n - ✔ `no_std` compatible\n - ✔ fuzzed against reference implementation to ensure stability and compatibility\n\n## Caution\n\n`HighwayHash` (the algorithm) has not undergone extensive cryptanalysis like SipHash (the default hashing algorithm in Rust), but according to the authors, HighwayHash output bits are uniformly distributed and should withstand differential and rotational attacks. Hence HighwayHash is referred to as a strong hash function, not a cryptographic hash function. I encourage anyone interested to [peruse the paper](https://arxiv.org/abs/1612.06257) to understand the risks.\n\n## Examples\n\nThe quickest way to get started:\n\n```rust\nuse highway::{HighwayHasher, HighwayHash};\nlet res: u64 = HighwayHasher::default().hash64(\u0026[]);\nlet res2: [u64; 2] = HighwayHasher::default().hash128(\u0026[]);\nlet res3: [u64; 4] = HighwayHasher::default().hash256(\u0026[]);\n```\n\nA more complete tour of the API follows:\n\n```rust\nuse highway::{HighwayHasher, HighwayHash, Key};\n\n// HighwayHash requires a key that should be hidden from attackers\n// to ensure outputs are unpredictable, so attackers can't mount\n// DoS attacks.\nlet key = Key([1, 2, 3, 4]);\n\n// A HighwayHasher is the recommended approach to hashing,\n// as it will select the fastest algorithm available\nlet mut hasher = HighwayHasher::new(key);\n\n// Append some data\nhasher.append(\u0026[255]);\n\n// After all data has been appended, you ask for\n// 64, 128, or 256bit output. The hasher is consumed\n// after finalization.\nlet res: u64 = hasher.finalize64();\n\nassert_eq!(0x07858f24d_2d79b2b2, res);\n```\n\nCreating a 128bit and 256bit hash is just as simple.\n\n```rust\nuse highway::{HighwayHasher, HighwayHash, Key};\n\n// Generate 128bit hash\nlet key = Key([1, 2, 3, 4]);\nlet mut hasher128 = HighwayHasher::new(key);\nhasher128.append(\u0026[255]);\nlet res128: [u64; 2] = hasher128.finalize128();\nassert_eq!([0xbb007d2462e77f3c, 0x224508f916b3991f], res128);\n\n// Generate 256bit hash\nlet key = Key([1, 2, 3, 4]);\nlet mut hasher256 = HighwayHasher::new(key);\nhasher256.append(\u0026[255]);\nlet res256: [u64; 4] = hasher256.finalize256();\nlet expected: [u64; 4] = [\n    0x7161cadbf7cd70e1,\n    0xaac4905de62b2f5e,\n    0x7b02b936933faa7,\n    0xc8efcfc45b239f8d,\n];\nassert_eq!(expected, res256);\n```\n\nUse highway hash in standard rust collections\n\n```rust\nuse std::collections::HashMap;\nuse highway::{HighwayBuildHasher, Key};\nlet mut map =\n  HashMap::with_hasher(HighwayBuildHasher::new(Key([\n    0xcbf29ce484222325,\n    0xc3a5c85c97cb3127,\n    0xb492b66fbe98f273,\n    0x9ae16a3b2f90404f,\n  ])));\n\nmap.insert(1, 2);\nassert_eq!(map.get(\u00261), Some(\u00262));\n```\n\nOr if utilizing a key is not important, one can use the default\n\n```rust\nuse std::collections::HashMap;\nuse std::hash::BuildHasherDefault;\nuse highway::HighwayHasher;\nlet mut map =\n  HashMap::with_hasher(BuildHasherDefault::\u003cHighwayHasher\u003e::default());\n\nmap.insert(1, 2);\nassert_eq!(map.get(\u00261), Some(\u00262));\n```\n\nHashing a file, or anything implementing `Read`\n\n```rust\nuse std::hash::Hasher;\nuse highway::{PortableHash, HighwayHash};\n\nlet mut file = \u0026b\"hello world\"[..];\n\n// We're using the `PortableHash` to show importing a specific hashing\n// implementation (all hash outputs are already portable / hardware agnostic).\n// The main reason for directly using `PortableHash` would be if avoiding\n// `unsafe` code blocks is a top priority.\nlet mut hasher = PortableHash::default();\nstd::io::copy(\u0026mut file, \u0026mut hasher)?;\nlet hash64 = hasher.finish(); // core Hasher API\nlet hash256 = hasher.finalize256(); // HighwayHash API\n```\n\n## Use Cases\n\n`HighwayHash` can be used against untrusted user input where weak hashes can't be used due to exploitation, verified cryptographic hashes are too slow, and a strong hash function meets requirements. Some specific scenarios given by the authors of HighwayHash:\n\n- Use 64bit hashes to for authenticating short lived messages\n- Use 256bit hashes for checksums. Think file storage (S3) or any longer lived data where there is a need for strong guarantees against collisions.\n\n`HighwayHash` may not be a good fit if the payloads trend small (\u003c 100 bytes) and speed is up of the utmost importance, as HighwayHash hits its stride at larger payloads.\n\n## Wasm SIMD\n\nWhen deploying HighwayHash to a Wasm environment, one can opt into using the Wasm SIMD instructions by adding a Rust flag:\n\n```bash\nRUSTFLAGS=\"-C target-feature=+simd128\" wasm-pack build\n```\n\nThen `HighwayHasher` will automatically defer to the Wasm SIMD implementation via `WasmHash`.\n\nOnce opted in, the execution environment must support Wasm SIMD instructions, which Chrome, Firefox, and Node LTS have stabilized since mid-2021. The opt in is required as there is not a way for Wasm to detect SIMD capabilities at runtime. The mere presence of Wasm SIMD instructions will cause incompatible environments to fail to compile, so it is recommended to provide two Wasm payloads to downstream users: one with SIMD enabled and one without.\n\n### `no_std` crates\n\nThis crate has a feature, `std`, that is enabled by default. To use this crate\nin a `no_std` context, add the following to your `Cargo.toml`:\n\n```toml\n[dependencies]\nhighway = { version = \"x\", default-features = false }\n```\n\nBe aware that the `no_std` version is unable to detect CPU features and so will always default to the portable implementation. If building for a known SSE 4.1 or AVX 2 machine (and the majority of machines in the last decade will support SSE 4.1), then explicitly enable the target feature:\n\n```bash\nRUSTFLAGS=\"-C target-feature=+sse4.1\" cargo test\nRUSTFLAGS=\"-C target-feature=+avx2\" cargo test\n```\n\n## Benchmarks\n\nBenchmarks are ran with the following command:\n\n```bash\n(cd compare \u0026\u0026 cargo clean \u0026\u0026 RUSTFLAGS=\"-C target-cpu=native\" cargo bench)\nfind ./compare/target -wholename \"*/new/raw.csv\" -print0 | xargs -0 xsv cat rows \u003e assets/highway.csv\n```\n\nAnd can be analyzed with the [R script](assets/analysis.R) found in the assets directory\n\nKeep in mind, benchmarks will vary by machine. Newer machines typically handle AVX payloads better than older.\n\nWe'll first take a look at the throughput when calculating the 64bit hash of a varying payload with various implementations\n\n![64bit-highwayhash.png](assets/64bit-highwayhash.png)\n\nTakeaways:\n\n- The lower left corner of the graph illustrates HighwayHash's weakness: small payloads, as with a bit of squinting, one can see that HighwayHash ranks amongst the bottom.\n- At larger payloads, HighwayHash can be competitive in performance as the CPU has room to stretch its proverbial SIMD legs on the input.\n- AHash and t1ha perform fantastically and should be in one's toolkit for in memory data structures.\n\nNow taking a look at calculating a 256bit hash value, we see a similar story.\n\n![256bit-highwayhash.png](assets/256bit-highwayhash.png)\n\nTakeaways:\n\n- HighwayHash is by far the fastest compared to the other functions, but if one needs a cryptographic hash, then BLAKE3 should be chosen\n\nEven with the best eyesight, the differences are indistinguishable at smaller payloads, so let's look at the hash rate: \n\n![256bit-highwayhash-rate.png](assets/256bit-highwayhash-rate.png)\n\nTakeaways:\n\n- At smaller payloads HighwayHash maintains its performance lead\n\nHighwayHash uses more rounds of permutation when finalizing the 256bit output compared to the 64bit and this is reflected in the following graphic:\n\n![64bit-vs-256bit-highwayhash.png](assets/64bit-vs-256bit-highwayhash.png)\n\nTakeaways:\n\n- At max, the 64bit hash can be computed 33% faster than the 256bit output\n- After 64KiB there is no performance difference between 64bit and 256bit outputs \n\nFor those more into numbers and are curious about specifics or want more details about the hash functions at small payloads size, here is a table that breaks down throughput (in GB/s) at all payload sizes\n\n![highwayhash-table.png](assets/highwayhash-table.png)\n\n### Builder Benchmarks\n\nHave fun running the builder benchmarks to see how performance differs with flags:\n\n*Default compilation*\n\n```bash\ncargo bench -- highway-builder\n```\n\n*Explicitly disable avx2*\n\n```bash\nRUSTFLAGS=\"-C target-feature=-avx2\" cargo bench -- highway-builder\n```\n\n*Explicitly disable avx2 when targeting native cpu*\n\n```bash\nRUSTFLAGS=\"-C target-cpu=native -C target-feature=+sse4.1,-avx2\" \\\n  cargo bench -- highway-builder\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickbabcock%2Fhighway-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnickbabcock%2Fhighway-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnickbabcock%2Fhighway-rs/lists"}