{"id":13672481,"url":"https://github.com/prataprc/xorfilter","last_synced_at":"2025-12-12T14:37:43.134Z","repository":{"id":48797372,"uuid":"229431424","full_name":"prataprc/xorfilter","owner":"prataprc","description":"Rust library implementing xor-filters","archived":false,"fork":false,"pushed_at":"2023-06-06T09:21:16.000Z","size":183,"stargazers_count":139,"open_issues_count":8,"forks_count":18,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-04-04T04:11:41.853Z","etag":null,"topics":["bitmap-index","bloom-filter","fuse-filter","probablistic-data-structures","rust","rust-lang","xor-filter"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prataprc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2019-12-21T13:30:05.000Z","updated_at":"2025-02-04T22:05:58.000Z","dependencies_parsed_at":"2023-07-14T08:57:36.154Z","dependency_job_id":null,"html_url":"https://github.com/prataprc/xorfilter","commit_stats":null,"previous_names":["bnclabs/xorfilter"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fxorfilter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fxorfilter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fxorfilter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fxorfilter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prataprc","download_url":"https://codeload.github.com/prataprc/xorfilter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254576272,"owners_count":22094328,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitmap-index","bloom-filter","fuse-filter","probablistic-data-structures","rust","rust-lang","xor-filter"],"created_at":"2024-08-02T09:01:36.865Z","updated_at":"2025-12-12T14:37:43.043Z","avatar_url":"https://github.com/prataprc.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"[![Rustdoc](https://img.shields.io/badge/rustdoc-hosted-blue.svg)](https://docs.rs/xorfilter-rs)\n[![simple-build-test](https://github.com/bnclabs/xorfilter/actions/workflows/simple-build-test.yml/badge.svg)](https://github.com/bnclabs/xorfilter/actions/workflows/simple-build-test.yml)\n\nRust library implementing xor filters\n-------------------------------------\n\nImplementation of [Xor Filters: Faster and Smaller Than Bloom and Cuckoo Filters](https://arxiv.org/abs/1912.08258)\nin [rust-lang](https://www.rust-lang.org/), Journal of Experimental Algorithmics (to appear).\n\nThis package is a port from its [golang implementation](https://github.com/FastFilter/xorfilter).\n\n### How to use _xorfilter_ in my rust project ?\n\nAdd the following under project's `Cargo.toml`:\n\n```toml\n[dependencies]\nxorfilter-rs = \"0.2.0\"\n```\n\nor\n\n```toml\n[dependencies]\nxorfilter-rs = { git = \"https://github.com/bnclabs/xorfilter\" }\n```\n\n```rust\nuse xorfilter::Xor8;\n\nlet mut keys: Vec\u003cu64\u003e = vec![];\nfor _ in 0..num_keys {\n    keys.push(rng.gen());\n}\n\nlet mut filter = Xor8::new(); // new filter.\nfilter.populate_keys(\u0026keys); // populate keys.\nfilter.build(); // build bitmap.\n\nfor key in 0..lookup {\n    // there can be false positives, but no false negatives.\n    filter.contains_key(key);\n}\n```\n\nOpen issues\n-----------\n\n* [ ] Serialize / Deserialize Xor8 type.\n* [ ] Incrementally adding keys to a pre-built Xor8 instance.\n* [ ] Gather benchmark results for other implementations - Go, C, C++, Erlang, Java, Python.\n\nBenchmarks\n----------\n\nFollowing are the results for a set of 10-million `u64` keys:\n\n|             |  build 10M keys |  membership |   FPP   |  Bits/Entry |\n|-------------|-----------------|-------------|---------|-------------|\n| Xor8-C      |   1.206 secs    |    NA       | 0.389 % |  9.84 bits  |\n| Xor8-rust   |   1.809 secs    | 61.716 ns   | 0.392 % |  9.84 bits  |\n| Fuse8-C     |   0.508 secs    |    NA       | 0.390 % |  9.02 bits  |\n| Fuse8-rust  |   0.577 secs    | 42.657 ns   | 0.392 % |  9.02 bits  |\n| Fuse16-C    |   0.515 secs    |    NA       | 0.001 % | 18.04 bits  |\n| Fuse16-rust |   0.621 secs    | 54.657 ns   | 0.001 % | 18.03 bits  |\n\n* **Build time** is measured in `Seconds`, for 10 million entries.\n* **Membership** is measured in `Nanosec`, for single lookup in a set of 10 million entries.\n* **FPP** = False Positive Probability measured in percentage\n\nUseful links\n------------\n\n* [Xor Filters: Faster and Smaller Than Bloom and Cuckoo Filters](https://arxiv.org/abs/1912.08258)\n* [Blog post by Daniel Lemire](https://lemire.me/blog/2019/12/19/xor-filters-faster-and-smaller-than-bloom-filters/)\n\n\nContribution\n------------\n\n* Simple workflow. Fork - Modify - Pull request.\n* Before creating a PR,\n  * Run `make build` to confirm all versions of build is passing with\n    0 warnings and 0 errors.\n  * Run `check.sh` with 0 warnings, 0 errors and all test-cases passing.\n  * Run `perf.sh` with 0 warnings, 0 errors and all test-cases passing.\n  * [Install][spellcheck] and run `cargo spellcheck` to remove common spelling mistakes.\n* [Developer certificate of origin][dco] is preferred.\n\n[dco]: https://developercertificate.org/\n[spellcheck]: https://github.com/drahnr/cargo-spellcheck\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprataprc%2Fxorfilter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprataprc%2Fxorfilter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprataprc%2Fxorfilter/lists"}