{"id":18144860,"url":"https://github.com/cloudflare/entropy-map","last_synced_at":"2026-03-17T12:13:44.505Z","repository":{"id":258227345,"uuid":"874483311","full_name":"cloudflare/entropy-map","owner":"cloudflare","description":"Low-latency hash map using minimal perfect hash functions and compact encoding.","archived":false,"fork":false,"pushed_at":"2024-11-19T04:55:47.000Z","size":75,"stargazers_count":19,"open_issues_count":0,"forks_count":3,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-29T23:12:27.446Z","etag":null,"topics":["data-structures","hash-map","mphf"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudflare.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-17T23:04:03.000Z","updated_at":"2025-03-15T22:35:27.000Z","dependencies_parsed_at":"2024-10-18T08:04:15.054Z","dependency_job_id":null,"html_url":"https://github.com/cloudflare/entropy-map","commit_stats":null,"previous_names":["cloudflare/entropy-map"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fentropy-map","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fentropy-map/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fentropy-map/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fentropy-map/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudflare","download_url":"https://codeload.github.com/cloudflare/entropy-map/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250403833,"owners_count":21424795,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["data-structures","hash-map","mphf"],"created_at":"2024-11-01T20:06:31.262Z","updated_at":"2025-10-28T07:31:49.772Z","avatar_url":"https://github.com/cloudflare.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# entropy-map\n![build](https://img.shields.io/github/actions/workflow/status/cloudflare/entropy-map/ci.yml?branch=main)\n[![docs.rs](https://docs.rs/entropy-map/badge.svg)](https://docs.rs/entropy-map)\n[![crates.io](https://img.shields.io/crates/v/entropy-map.svg)](https://crates.io/crates/entropy-map)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)\n\n`entropy-map` is an ultra-low latency hash map Rust crate using minimal perfect hash functions ([MPHF](https://en.wikipedia.org/wiki/Perfect_hash_function)) and compact encoding of values, designed for scenarios where both memory efficiency and fast data retrieval are critical. Ideal for applications in high-performance computing, `entropy-map` offers a unique blend of speed and compactness.\n\n## Getting Started\nSimple example to quickly get started:\n```rust\nuse entropy_map::Mphf;\n\nlet keys = [1, 2, 3, 4, 5];\nlet mphf = Mphf::\u003c32, 8\u003e::from_slice(\u0026keys, 2.0).unwrap();\nassert!(mphf.get(\u00261).is_some());\n```\n\nCheck out the provided examples for detailed usage:\n* [mphf](examples/mphf.rs)\n* [map_with_dict](examples/map_with_dict.rs)\n* [map_with_dict_bitpacked](examples/map_with_dict_bitpacked.rs)\n* [set](examples/set.rs)\n\n## Overview\nThis crate provides advanced data structures leveraging MPHF, optimized for scenarios requiring high-speed data access and minimal memory usage.\nIt includes the following key components:\n\n### Minimal Perfect Hash Function (MPHF)\n- Implements MPHF based on fingerprinting techniques as detailed in [Fingerprinting-based minimal perfect hashing revisited](https://doi.org/10.1145/3596453)\n- Inspired by [ph](https://github.com/beling/bsuccinct-rs/tree/main/ph) crate but with improved rank storage and reduced construction and query times.\n- Optimized rank storage mechanism based on [Engineering Compact Data Structures for Rank and Select Queries on Bit Vectors](https://arxiv.org/pdf/2206.01149.pdf)\n- Memory usage ranging from `2.10 bits` to `2.71 bits` per key depending on parameters.\n- Query time ranging from `5 ns` to `20 ns` depending on the parameters, number of keys and L1-L3 cache sizes.\n- Configurable template parameters for flexibility.\n  - `B`: group size in bits in [1..64] range, default 32 bits.\n  - `S`: defines maximum seed value to try (2^S) in [0..16] range, default 8.\n  - `ST`: seed type (unsigned integer), default `u8`.\n  - `H`: hasher used to hash keys, default `WyHash`.\n- Configurable `gamma` parameter to tune construction time vs query time trade-off.\n- Optional [rkyv](https://rkyv.org/) support to enable zero-copy serialization/deserialization of MPHF.\n\n### MapWithDict\n- Immutable hash map leveraging MPHF for indexing.\n- Stores keys to ensure presence/absence of the key in the map.\n- Optimized for space, using a dictionary to pack unique values.\n- Efficient storage and retrieval, reducing overall memory footprint.\n- Optional [rkyv](https://rkyv.org/) support to enable zero-copy serialization/deserialization and superior memory footprint and performance when compared with `rkyv::ArchivedHashMap`.\n\n### MapWithDictBitpacked\n- Specialized version of `MapWithDict`, further optimized for memory usage when values are `Vec\u003cu32\u003e`.\n- Bit-packs `Vec\u003cu32\u003e` values for minimal space usage using SIMD instructions.\n- Excels in scenarios where values are within a limited range and can be efficiently encoded.\n\n### Set\nSpecial case of `MapWithDict`, optimized for set membership operations.\n- Immutable set using MPHF for indexing.\n- Stores keys to ensure presence/absence of the key in the set.\n- Optional rkyv support to enable zero-copy serialization/deserialization.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudflare%2Fentropy-map","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudflare%2Fentropy-map","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudflare%2Fentropy-map/lists"}