{"id":13672602,"url":"https://github.com/ayazhafiz/xorf","last_synced_at":"2025-04-12T18:42:17.244Z","repository":{"id":38282726,"uuid":"229877920","full_name":"ayazhafiz/xorf","owner":"ayazhafiz","description":"Xor filters - efficient probabilistic hashsets. Faster and smaller than bloom and cuckoo filters.","archived":false,"fork":false,"pushed_at":"2024-04-15T06:46:06.000Z","size":422,"stargazers_count":126,"open_issues_count":11,"forks_count":26,"subscribers_count":5,"default_branch":"master","last_synced_at":"2024-10-13T21:06:57.973Z","etag":null,"topics":["bloom-filter","data-structures","hashset","probability","rust","xor-filter"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ayazhafiz.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-12-24T05:41:34.000Z","updated_at":"2024-10-10T15:22:50.000Z","dependencies_parsed_at":"2024-06-19T16:03:52.767Z","dependency_job_id":"ac7634ae-eef6-4e34-8cdd-27c89a680c3f","html_url":"https://github.com/ayazhafiz/xorf","commit_stats":{"total_commits":73,"total_committers":12,"mean_commits":6.083333333333333,"dds":0.3013698630136986,"last_synced_commit":"1ea394238f6cbd5e00424b89b70e685e937b39da"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayazhafiz%2Fxorf","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayazhafiz%2Fxorf/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayazhafiz%2Fxorf/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ayazhafiz%2Fxorf/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ayazhafiz","download_url":"https://codeload.github.com/ayazhafiz/xorf/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247070927,"owners_count":20878586,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bloom-filter","data-structures","hashset","probability","rust","xor-filter"],"created_at":"2024-08-02T09:01:41.199Z","updated_at":"2025-04-03T20:12:22.037Z","avatar_url":"https://github.com/ayazhafiz.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# xorf\n\n[![Xorf docs](https://docs.rs/xorf/badge.svg)](https://docs.rs/xorf)\n[![Crates.io](https://img.shields.io/crates/v/xorf)](https://crates.io/crates/xorf)\n[![Build Status](https://github.com/ayazhafiz/xorf/actions/workflows/ci.yml/badge.svg)](https://github.com/ayazhafiz/xorf/actions/workflows/ci.yml)\n\n- [Documentation](https://docs.rs/xorf)\n- [Crates.io Registry](https://crates.io/crates/xorf)\n\nThis repository hosts a Rust library implementing\n[xor filters](https://arxiv.org/abs/1912.08258) and their derivates:\n\n- Binary Fuse filters (most recommended)\n  - [`BinaryFuse8`](./src/bfuse8.rs), [`BinaryFuse16`](./src/bfuse16.rs), [`BinaryFuse32`](./src/bfuse32.rs)\n- Xor filters\n  - [`Xor8`](./src/xor8.rs), [`Xor16`](./src/xor16.rs), [`Xor32`](./src/xor32.rs)\n- Fuse filters (deprecated, use Binary Fuse filters instead)\n  - [`Fuse8`](./src/fuse8.rs), [`Fuse16`](./src/fuse16.rs), [`Fuse32`](./src/fuse32.rs)\n\nXor filters are data structures for fast approximation of set membership using\nlittle memory. Probabilistic filters like\nxor filters are useful when it's okay to have false positives sometimes, but\nit's important to be space and time efficient. In other words, they trade off\naccuracy for efficiency as compared to general-purpose hashsets. Filters like\nxor filter are often used in conjunction with larger hash-based data structures,\nwith the filter doing a \"first pass\" of the work to avoid using a more expensive\nresource unnecessarily. For example, filters like xor filters can be used to\n[reduce disk writes](https://en.wikipedia.org/wiki/Bloom_filter#Cache_filtering)\nin a cache or\n[identify malicious URLs](https://en.wikipedia.org/wiki/Bloom_filter#Examples)\nin a browser.\n\nXor filters are faster and smaller than Bloom and Cuckoo filters. Xor filters\nincur a relative time penalty in construction, but are very fast in lookups; the\nexpectation is that construction of a filter is amortized after many queries.\nDaniel Lemire's [go implementation](https://github.com/FastFilter/xorfilter)\nprovides a useful summary of xor filters' benefits and listing of other xor\nfilter libraries.\n\nThis library is `no_std` and\n[`needs_allocator`](https://doc.rust-lang.org/1.9.0/book/custom-allocators.html).\n\n`xorf` also provides a [`HashProxy`](./src/hash_proxy.rs) for using Xor filters\nwith arbitrary key types.\n\n## Installation\n\nAdd a dependency to `xorf` in `Cargo.toml`:\n\n```toml\n[dependencies]\nxorf = \"M.m.p\" # use a desired version\n```\n\nAvailable versions are listed on [crates](https://crates.io/crates/xorf) and the in [repository's\nreleases](https://github.com/ayazhafiz/xorf/releases).\n\n## Usage\n\nPlease see the [library documentation](https://docs.rs/xorf) for usage\ninformation.\n\n### Features\n\n#### Custom allocator\n\nTo use a [custom global allocator](https://doc.rust-lang.org/1.9.0/book/custom-allocators.html),\nyou must be using a nightly release of rustc and have enabled the `nightly`\nfeature for `xorf`.\n\n```toml\n[dependencies]\nxorf = { version = \"M.m.p\", features = [\"nightly\"] }\n```\n\nThis will tag the crate as `needs_allocator`, which you will then have to\nprovide. At this time, a custom allocator is used globally.\n\n#### Serialization/Deserialization\n\nSerialization and deserialization with [serde](https://serde.rs/) cab be enabled\nwith the `serde` feature.\n\n```toml\n[dependencies]\nxorf = { version = \"M.m.p\", features = [\"serde\"] }\n```\n\n#### Default features\n\n##### Uniform Random\n\nBy default, `xorf` uses the `uniform-random` feature, which uses random values for unused\nfingerprint entries rather than initializing them to zero. This provides a slightly lower false-positive\nrate, but incurs a higher initialization cost. The cost of lookups is not affected.\n\nThe distribution of zero-valued fingerprints for a 1,000,000-element\n`BinaryFuse8` filter, both without and with the `uniform-random` feature, is\nvisualized below. Notice that the scales of the y-axes differ.\n\n| `BinaryFuse8` without `uniform-random` | `BinaryFuse8` with `uniform-random` |\n|--|--|\n| ![Ouch](./analysis/uniformity/BinaryFuse8-no-uniform-random.png) | ![Nice](./analysis/uniformity/BinaryFuse8-uniform-random.png) |\n\nTo disable the `uniform-random` feature, specify that default features should be disabled:\n\n```toml\n[dependencies]\nxorf = { version = \"M.m.p\", default-features = false }\n```\n\n##### Binary Fuse\n\nBy default, `xorf` uses the `binary-fuse` feature, which adds support for and\nexposes Binary Fuse filter implementations. This feature pulls in a dependency\nof `libm`, but has no runtime cost. This feature is highly recommended, as\nBinary Fuse filters are the most powerful in the Xor filter family.\n\n## Development\n\nDevelopment of `xorf` targets the master branch of this repository.\n\nChanges can be tested by running the [`check`](./scripts/check) script:\n\n```bash\nscripts/check lf     # validates lint and format\nscripts/check test   # tests source code\n```\n\n## Contribution\n\nContributions are warmly welcomed. No contribution is too small, and all are\nappreciated.\n\n## License\n\n[MIT](./LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fayazhafiz%2Fxorf","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fayazhafiz%2Fxorf","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fayazhafiz%2Fxorf/lists"}