{"id":29510894,"url":"https://github.com/vigna/card-est-array-rs","last_synced_at":"2026-04-01T21:26:30.724Z","repository":{"id":285463752,"uuid":"958229813","full_name":"vigna/card-est-array-rs","owner":"vigna","description":"Infrastructure for managing large arrays of counters","archived":false,"fork":false,"pushed_at":"2026-03-14T23:02:53.000Z","size":153,"stargazers_count":3,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2026-03-15T08:56:28.417Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vigna.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-Apache-2.0","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-03-31T21:21:52.000Z","updated_at":"2026-03-14T23:02:57.000Z","dependencies_parsed_at":"2025-04-07T10:39:03.617Z","dependency_job_id":null,"html_url":"https://github.com/vigna/card-est-array-rs","commit_stats":null,"previous_names":["vigna/counter-array-rs","vigna/card-est-array-rs"],"tags_count":8,"template":false,"template_full_name":null,"purl":"pkg:github/vigna/card-est-array-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fcard-est-array-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fcard-est-array-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fcard-est-array-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fcard-est-array-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vigna","download_url":"https://codeload.github.com/vigna/card-est-array-rs/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fcard-est-array-rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31292246,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-01T21:15:39.731Z","status":"ssl_error","status_checked_at":"2026-04-01T21:15:34.046Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-07-16T09:43:28.076Z","updated_at":"2026-04-01T21:26:30.717Z","avatar_url":"https://github.com/vigna.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Arrays of Cardinality Estimators\n\n[![crates.io](https://img.shields.io/crates/v/card-est-array.svg)](https://crates.io/crates/card-est-array)\n[![docs.rs](https://docs.rs/card-est-array/badge.svg)](https://docs.rs/card-est-array)\n[![rustc](https://img.shields.io/badge/rustc-1.85+-red.svg)](https://rust-lang.github.io/rfcs/2495-min-rust-version.html)\n[![CI](https://github.com/vigna/card-est-array-rs/actions/workflows/rust.yml/badge.svg)](https://github.com/vigna/card-est-array-rs/actions)\n![license](https://img.shields.io/crates/l/card-est-array)\n[![downloads](https://img.shields.io/crates/d/card-est-array)](https://crates.io/crates/card-est-array)\n[![coveralls](https://coveralls.io/repos/github/vigna/card-est-array-rs/badge.svg?branch=main)](https://coveralls.io/github/vigna/card-est-array-rs?branch=main)\n\nInfrastructure for managing large arrays of cardinality estimators.\n\n## Why\n\nA _cardinality estimator_, AKA _(probabilistic) sketch_, AKA _probabilistic\ncounter_ is a probabilistic data structure that has an \"add\" primitive similarly\nto a dictionary, but does not have a corresponding \"contains\": it is just\npossible to query the _size_ of the estimator. In other words, a cardinality\nestimator remembers the number of distinct elements that have been added to it.\n\nThe returned size is only an approximation of the real size, and the precision\ncan be tuned, but in exchange cardinality estimators use very little space: for\nexample, a [HyperLogLog cardinality estimator] uses\n2ᵇ⌈log log _n_⌉ bits to achieve an average relative error of 1.04/√2ᵇ (log log _n_\n≤ 6 for all practical datasets).\n\nIt is common, for example, to use cardinality estimators to measure the number\nof unique users in click streams. But more interesting applications use the fact\nthat many cardinality estimators are _[mergeable]_, which means that given two\nestimators it is possible to compute (in time linear in the size of the estimators)\na new estimator containing the union of the elements that have been added to the\ntwo original estimators.\n\nThis idea is at the core of [ANF], an algorithm for the computation of the\nneighborhood function (the function telling how many pairs of nodes are within\ndistance _t_) that used [Flajolet–Martin cardinality estimators (then called\nprobabilistic counters)]. The technique was then extended to [log-logarithmic\ncardinality estimators], with a significant reduction of the memory footprint,\nusing [broadword programming] to merge such estimators; it became also evident\nthat it could be used to compute many other interesting properties, such as the\ndistance distribution and all centralities based on the node neighborhood\nfunctions (the functions telling, for each node, how many other nodes are within\ndistance _t_). The [HyperBall algorithm], distributed with the [WebGraph\nframework], is a highly engineered implementation of these ideas. It has been\nused, for example, to compute the [degrees of separation of Facebook].\n\nThe purpose of this crate is to lay the foundation of the infrastructure that is\nnecessary to implement HyperBall in the [Rust port of the WebGraph framework].\nWe provide implementations of cardinality estimators and structures handling\nlarge arrays of estimators sharing the same parameters and logic with minimal\nmemory overhead. Sharing parameters is essential for scaling to billions of\nestimators, and this is why a separate structure for arrays of estimators is\nnecessary—just putting a billion estimators in a vector would waste a large\namount of space.\n\nFor this reason, sometimes the traits give a low-level feeling. While single\nestimators are encapsulated in suitable structures, arrays of estimators are\nmade of the concatenation of estimator backends, and each estimator is manipulated\napplying suitable methods to the backend. But it is the responsibility of the\nuser that backends and logic are matched.\n\nNote that this setup is not restricted to arrays, as it makes it possible to\nhave collections of any kind containing backends sharing the same logic (e.g.,\nhash maps or sets of backends).\n\n## Provided estimators\n\nWe provide two implementations of HyperLogLog counters (with optional [LogLog-β\ncorrection]). They provide [`EstimationLogic`] implementations that can be used\nto manage arrays of estimators.\n\n- [`HyperLogLog`] is a compact, bit-based implementation;\n- [`HyperLogLog8`] is a faster byte-based implementation.\n\n## Acknowledgments\n\nThis software has been partially supported by project SERICS (PE00000014) under\nthe NRRP MUR program funded by the EU - NGEU, and by project ANR COREGRAPHIE,\ngrant ANR-20-CE23-0002 of the French Agence Nationale de la Recherche. Views and\nopinions expressed are however those of the authors only and do not necessarily\nreflect those of the European Union or the Italian MUR. Neither the European\nUnion nor the Italian MUR can be held responsible for them.\n\n[HyperLogLog cardinality estimator]: https://algo.inria.fr/flajolet/Publications/FlFuGaMe07.pdf\n[mergeable]: https://docs.rs/card-est-array/latest/card_est_array/traits/trait.MergeEstimationLogic.html\n[ANF]: https://doi.org/10.1145/775047.775059\n[Flajolet–Martin cardinality estimators (then called probabilistic counters)]: https://doi.org/10.1016%2F0022-0000%2885%2990041-8\n[log-logarithmic cardinality estimators]: https://algo.inria.fr/flajolet/Publications/FlFuGaMe07.pdf\n[broadword programming]: https://doi.org/10.1145/1963405.1963493\n[HyperBall algorithm]: https://doi.ieeecomputersociety.org/10.1109/ICDMW.2013.10\n[WebGraph framework]: https://webgraph.di.unimi.it/\n[degrees of separation of Facebook]: https://doi.org/10.1145/2380718.2380723\n[`EstimationLogic`]: https://docs.rs/card-est-array/latest/card_est_array/traits/trait.EstimationLogic.html\n[`HyperLogLog`]: https://docs.rs/card-est-array/latest/card_est_array/impls/hyper_log_log/struct.HyperLogLog.html\n[`HyperLogLog8`]: https://docs.rs/card-est-array/latest/card_est_array/impls/hyper_log_log8/struct.HyperLogLog8.html\n[LogLog-β correction]: https://docs.rs/card-est-array/latest/card_est_array/impls/hyper_log_log/fn.beta_horner.html\n[Rust port of the WebGraph framework]: https://crates.io/crates/webgraph\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvigna%2Fcard-est-array-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvigna%2Fcard-est-array-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvigna%2Fcard-est-array-rs/lists"}