{"id":13672459,"url":"https://github.com/vmware/node-replication","last_synced_at":"2025-04-09T06:32:19.613Z","repository":{"id":37885750,"uuid":"201998747","full_name":"vmware/node-replication","owner":"vmware","description":"An operation-log based approach for data replication.","archived":false,"fork":false,"pushed_at":"2023-04-27T00:28:29.000Z","size":1243,"stargazers_count":62,"open_issues_count":5,"forks_count":21,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-03-24T00:08:22.224Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vmware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE-APACHE","code_of_conduct":"CODE-OF-CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-12T19:50:08.000Z","updated_at":"2025-03-09T11:56:56.000Z","dependencies_parsed_at":"2024-11-06T15:53:03.997Z","dependency_job_id":null,"html_url":"https://github.com/vmware/node-replication","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fnode-replication","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fnode-replication/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fnode-replication/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vmware%2Fnode-replication/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vmware","download_url":"https://codeload.github.com/vmware/node-replication/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247992570,"owners_count":21029921,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T09:01:36.198Z","updated_at":"2025-04-09T06:32:18.467Z","avatar_url":"https://github.com/vmware.png","language":"Rust","readme":"# node-replication\n\nThe node-replication library is used to implement concurrent and replicated\nversions of (single-threaded) data structures by combining a set of different\nconcurrency techniques: flat combining, operation logging, readers-writer locks,\nand partitioning.\n\n## Implementations\n\nThe crate contains two variants of node-replication: NR for transforming\nsequential data-structures and CNR for already concurrent or partitionable data\nstructures.\n\n* [`nr`](node-replication/src/nr) converts a sequental data structure to its\n  NUMA-aware concurrent version.\n* [`cnr`](node-replication/src/cnr) converts a concurrent (or partitioned) data\n  structure to its NUMA-aware concurrent version.\n\n## Background\n\n* The implementation of NR and a detailed evaluation can be found in this paper:\n  [Black-box Concurrent Data Structures for NUMA\n  Architectures](https://dl.acm.org/citation.cfm?id=3037721) which explains the\n  NR library in more details.\n* The implementation of CNR and a detailed evaluation can be found in this\n  paper: [NrOS: Effective Replication and Sharing in an Operating\n  System](https://www.usenix.org/conference/osdi21/presentation/bhardwaj). The\n  [NrOS\n  documentation](https://nrkernel.systems/book/architecture/NodeReplication.html)\n  also has a brief overview of NR and CNR as these are the main concurrency\n  mechanisms in the nrkernel.\n* The NR code in this repo was [formally verified by porting it to\n  Dafny](https://github.com/vmware-labs/verified-betrfs/tree/concurrency-experiments/concurrency/node-replication),\n  including a proof that it ensures linearizability.\n\n## How does it work (NR)\n\nTo replicate a single-threaded data structure, one needs to implement `Dispatch`\n(from node-replication). As an example, we implement `Dispatch` for the\nsingle-threaded HashMap from `std`.\n\n```rust\n#![feature(generic_associated_types)]\nuse std::collections::HashMap;\nuse node_replication::nr::Dispatch;\n\n/// The node-replicated hashmap uses a std hashmap internally.\n#[derive(Default)]\nstruct NrHashMap {\n    storage: HashMap\u003cu64, u64\u003e,\n}\n\n/// We support mutable put operation on the hashmap.\n#[derive(Clone, Debug, PartialEq)]\nenum Modify {\n    Put(u64, u64),\n}\n\n/// We support an immutable read operation to lookup a key from the hashmap.\n#[derive(Clone, Debug, PartialEq)]\nenum Access {\n    Get(u64),\n}\n\n/// The Dispatch traits executes `ReadOperation` (our Access enum)\n/// and `WriteOperation` (our `Modify` enum) against the replicated\n/// data-structure.\nimpl Dispatch for NrHashMap {\n    type ReadOperation\u003c'rop\u003e = Access;\n    type WriteOperation = Modify;\n    type Response = Option\u003cu64\u003e;\n\n    /// The `dispatch` function applies the immutable operations.\n    fn dispatch\u003c'rop\u003e(\u0026self, op: Self::ReadOperation\u003c'rop\u003e) -\u003e Self::Response {\n        match op {\n            Access::Get(key) =\u003e self.storage.get(\u0026key).map(|v| *v),\n        }\n    }\n\n    /// The `dispatch_mut` function applies the mutable operations.\n    fn dispatch_mut(\u0026mut self, op: Self::WriteOperation) -\u003e Self::Response {\n        match op {\n            Modify::Put(key, value) =\u003e self.storage.insert(key, value),\n        }\n    }\n}\n```\n\nThe full example (using `HashMap` as the underlying data-structure) can be found\n[here](node-replication/examples/nr_hashmap.rs).\nTo run, execute: `cargo run --example nr_hashmap`\n\n## How does it perform (NR)\n\nThe library often makes your single-threaded implementation work better than, or\ncompetitive with fine-grained locking or lock free implementations of the same\ndata-structure.\n\nIt works especially well if\n\n* Your data-structure exceeds the L3 cache-size of your system (you may not see\n  any gain from replication if your data can always remain in the cache).\n* All your threads need to issue mixed, mutable and immutable operations (if\n  not alternative techniques like RCU may work better).\n* You have enough DRAM to take advantage of the replication (i.e., it's\n  typically best to use one replica per-NUMA node which means your original\n  memory foot-print is multiplied with the amount of NUMA nodes in the system).\n\nAs an example, the following benchmark uses Rust' the hash-table with the\n`Dispatch` implementation from above (`nr`), and compares it against concurrent\nhash table implementations from [crates.io](https://crates.io) (chashmap,\ndashmap, flurry), a HashMap protected by an `RwLock` (`std`), and\n[urcu](https://liburcu.org/).\n\n\u003ctable\u003e\n  \u003ctr\u003e\n    \u003ctd valign=\"top\"\u003e\u003ca href=\"./graphs/skylake4x-throughput-vs-cores.png?raw=true\"\u003e\n    \u003cimg src=\"./graphs/skylake4x-throughput-vs-cores.png?raw=true\" alt=\"Throughput of node-replicated HT\" /\u003e\n\u003c/a\u003e\u003c/td\u003e\n    \u003ctd valign=\"top\"\u003e\u003ca href=\"./graphs/skylake4x-throughput-vs-cores.png?raw=true\"\u003e\n    \u003cimg src=\"./graphs/skylake4x-throughput-vs-wr.png?raw=true\" alt=\"Different write ratios with 196 threads\" /\u003e\u003c/td\u003e\n  \u003c/tr\u003e\n\u003c/table\u003e\n\nThe figures show a benchmark using hash tables pre-filled with 67M entires (8\nbyte keys and values) and uses a uniform key distribution for operations. On the\nleft graphs, different write ratios (0%, 10% and 80%) are shown. On the right\ngraph, we vary the write ratio (x-axis) with 192 threads. The system has 4 NUMA\nnodes, so it uses 4 replicas (at `x=96`, a replica gets added every 24 cores).\nAfter `x=96`, the remaining hyper-threads are used.\n\n## Compile the library\n\nThe library works with `no_std` and currently requires a nightly rust compiler.\n\n```bash\ncargo build\n```\n\nAdd it as a dependency in your `Cargo.toml`:\n\n```toml\nnode-replication = \"*\"\n```\n\nThe code should currently be treated as an early release and is still work in\nprogress.\n\n## Testing\n\nThere are a range of unit tests as part of the implementation and a few\n[integration tests](node-replication/tests) that check various aspects of the\nimplementations.\n\nYou can run the tests by executing: `cargo test`\n\n## Benchmarks\n\nThe benchmarks (and how to execute them) are explained in more detail in the\n[benches](node-replication/benches/README.md) folder.\n\n## Supported Platforms\n\nThe code should be treated as an early release and is still work in progress. In\nits current form, the library is only known to work on x86-64 platforms (other\nplatforms will require some changes and are untested).\n\n## Contributing\n\nThe node-replication project team welcomes contributions from the community. If\nyou wish to contribute code and you have not signed our contributor license\nagreement (CLA), our bot will update the issue when you open a Pull Request. For\nany questions about the CLA process, please refer to our\n[FAQ](https://cla.vmware.com/faq).\n","funding_links":[],"categories":["Rust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmware%2Fnode-replication","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvmware%2Fnode-replication","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvmware%2Fnode-replication/lists"}