{"id":15032354,"url":"https://github.com/akvize/reconcile-rs","last_synced_at":"2025-04-09T21:23:19.881Z","repository":{"id":206030890,"uuid":"703952008","full_name":"Akvize/reconcile-rs","owner":"Akvize","description":"A reconciliation service to sync a key-value map over multiple instances.","archived":false,"fork":false,"pushed_at":"2023-11-24T11:00:43.000Z","size":571,"stargazers_count":3,"open_issues_count":12,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-23T23:16:14.918Z","etag":null,"topics":["distributed-storage","key-value-store","reconciliation","rust","rust-lang","sync"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/reconcile","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Akvize.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-12T08:38:46.000Z","updated_at":"2023-11-08T09:21:26.000Z","dependencies_parsed_at":"2023-11-10T18:26:11.381Z","dependency_job_id":"9935548c-20ba-4e10-9654-8514d133f5c0","html_url":"https://github.com/Akvize/reconcile-rs","commit_stats":null,"previous_names":["akvize/reconcile-rs"],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akvize%2Freconcile-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akvize%2Freconcile-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akvize%2Freconcile-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Akvize%2Freconcile-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Akvize","download_url":"https://codeload.github.com/Akvize/reconcile-rs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248113093,"owners_count":21049782,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-storage","key-value-store","reconciliation","rust","rust-lang","sync"],"created_at":"2024-09-24T20:18:09.531Z","updated_at":"2025-04-09T21:23:19.833Z","avatar_url":"https://github.com/Akvize.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# reconcile-rs\n\n[![Crates.io][crates-badge]][crates-url]\n[![MIT licensed][mit-badge]][mit-url]\n[![Apache licensed][apache-badge]][apache-url]\n[![Build Status][actions-badge]][actions-url]\n\n[crates-badge]: https://img.shields.io/crates/v/reconcile.svg\n[crates-url]: https://crates.io/crates/reconcile\n[mit-badge]: https://img.shields.io/badge/license-MIT-blue.svg\n[mit-url]: https://github.com/Akvize/reconcile-rs/blob/master/LICENSE-MIT\n[apache-badge]: https://img.shields.io/badge/license-APACHE-blue.svg\n[apache-url]: https://github.com/Akvize/reconcile-rs/blob/master/LICENSE-APACHE\n[actions-badge]: https://github.com/Akvize/reconcile-rs/actions/workflows/master.yml/badge.svg\n[actions-url]: https://github.com/Akvize/reconcile-rs/actions/workflows/master.yml\n\n[Docs](https://docs.rs/reconcile/latest/reconcile/)\n\nThis crate provides a key-data map structure `HRTree` that can be used together\nwith the reconciliation `Service`. Different instances can talk together over\nUDP to efficiently reconcile their differences.\n\nAll the data is available locally on all instances, and the user can be\nnotified of changes to the collection with an insertion hook.\n\nThe protocol allows finding a difference over millions of elements with a limited\nnumber of round-trips. It should also work well to populate an instance from\nscratch from other instances.\n\nThe intended use case is a scalable Web service with a non-persistent and\neventually consistent key-value store. The design enable high performance by\navoiding any latency related to using an external service such as Redis.\n\n![Architecture diagram of a scalable Web service using reconcile-rs](img/illustration.png)\n\nIn code, this would look like this:\n\n```rust\nlet tree = HRTree::new();\nlet mut service = Service::new(tree, port, listen_addr, peer_net).await;\ntokio::spawn(service.clone().run());\n// use the reconciliation service as a key-value store in the API\n```\n\n## HRTree\n\nThe core of the protocol is made possible by the `HRTree` (Hash-Range Tree) data structure, which\nallows `O(log(n))` access, insertion and removal, as well as `O(log(n))`\ncumulated hash range-query. The latter property enables querying\nthe cumulated (XORed) hash of all key-value pairs between two keys.\n\nAlthough we did come we the idea independently, it exactly matches a paper\npublished on Arxiv in February 2023: [Range-Based Set\nReconciliation](https://arxiv.org/abs/2212.13567), by Aljoscha Meyer\n\nOur implementation of this data structure is based on a B-Trees that we wrote\nourselves. Although we put a limited amount of effort in this, did not use\n`unsafe` and have to maintain more invariants, we stay within a factor 2 of the\nstandard `BTreeMap` from the standard library:\n\n![Graph of the time needed to insert N elements in an empty tree](img/perf-fill.png)\n\nThe graph above shows the amount of time **in milliseconds** (ordinate, left\naxis) needed to **insert N elements** (abscissa, bottom axis) in a tree\n(initially empty). Note that both axes use a logarithmic scale.\n\nThe performance of our `HRTree` implementation follows closely that of\n`BTreeMap`. When looking at each value of N, we see that the average throughput\nof the `HRTree` is between one third and one half that of `BTreeMap`.\n\n![Graph of the time needed to insert and remove 1 element in a tree of size N](img/perf-insert.png)\n\nThe graph above shows the amount of time **in nanoseconds** (abscissa, bottom\naxis) needed to **insert a single element** (and remove it) in a tree\ncontaining N elements (ordinate, bottom axis). Note that both axes use a\nlogarithmic scale.\n\nThe most important thing to notice is that the average insertion/removal time\nonly grows from 80 ns to 700 s although the size of the tree changes from 10 to\n1,000,000 elements.\n\n![Graph of the time needed to remove and restore 1 element in a tree of size N](img/perf-remove.png)\n\nThe graph above shows the amount of time **in nanoseconds** (abscissa, bottom\naxis) needed to **remove a single element** (and restore it) from a tree\ncontaining N elements (ordinate, bottom axis). Note that both axes use a\nlogarithmic scale.\n\nThe most important thing to notice is that the average removal/insertion time\nonly grows from 100 ns to 800 s although the size of the tree changes from 10 to\n1,000,000 elements.\n\n![Graph of the time needed to compute 1 hash of a range of elements in a tree of size N](img/perf-hash.png)\n\nThe graph above shows the amount of time **in microseconds** (abscissa, bottom\naxis) needed to compute **1 cumulated hash** over a random range of elements in a\ntree of size N (ordinate, bottom axis). Note that both axes use a logarithmic\nscale.\n\nThe average time per cumulated hash grows from 30 ns to 1,200 ns as the size of\nthe tree changes from 10 to 1,000,000 elements.\n\nAlthough there is likely still a lot of room for improvement regarding the\nperformance of the `HRTree`, it is quite enough for our purposes, since we\nexpect network delays to be orders of magnitude longer.\n\n## Service\n\nThe service exploits the properties of `HRTree` to conduct a binary-search-like\nsearch in the collections of the two instances. Once difference are found, the\ncorresponding key-value pairs are exchanged and conflicts are resolved.\n\n![Graph of the time needed to send 1 insertion and 1 removal](img/perf-send.png)\n\nThe graph above shows the amount of time **in microseconds** (abscissa, bottom\naxis) needed to send 1 insertion, then 1 removal** between two instances of\n`Service` that contain the same N elements (ordinate, left axis). Note that\nboth axes use a logarithmic scale.\n\nThe times are very consistent, hovering around 122 µs, showing that the\nreconciliation time is entirely bounded by the local network transmission. This\nis made possible by the immediate transmission of the element at\ninsertion/removal.\n\n![Graph of the time to reconcile 1 difference between two instances](img/perf-reconcile.png)\n\nThe graph above shows the amount of time **in milliseconds** (abscissa, bottom\naxis) needed to reconcile 1 insertion, then 1 removal** between two instances of\n`Service` that contain the same other N elements (ordinate, left axis). Note that\nboth axes use a logarithmic scale.\n\nThis time, the full reconciliation protocol must be run to identify the\ndifference. The times grow from 240 µs to 640 µs as the size of the collection\nchanges from 10 to 1,000,000 elements.\n\n**Note:** These benchmarks are performed locally on the loop-back network\ninterface. On a real network, transmission delays will make the values larger.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakvize%2Freconcile-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fakvize%2Freconcile-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fakvize%2Freconcile-rs/lists"}