{"id":13578812,"url":"https://github.com/cloudflare/mmap-sync","last_synced_at":"2025-05-14T11:11:12.417Z","repository":{"id":175573461,"uuid":"654195529","full_name":"cloudflare/mmap-sync","owner":"cloudflare","description":"Rust library for concurrent data access, using memory-mapped files, zero-copy deserialization, and wait-free synchronization.","archived":false,"fork":false,"pushed_at":"2024-12-26T16:19:52.000Z","size":82,"stargazers_count":543,"open_issues_count":6,"forks_count":40,"subscribers_count":17,"default_branch":"main","last_synced_at":"2025-04-13T01:54:45.872Z","etag":null,"topics":["interprocess-communication","memory-mapping","synchronization","wait-free","zero-copy"],"latest_commit_sha":null,"homepage":"https://crates.io/crates/mmap-sync","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudflare.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-06-15T15:28:49.000Z","updated_at":"2025-04-10T01:48:36.000Z","dependencies_parsed_at":"2024-01-16T20:29:04.620Z","dependency_job_id":"61a9c1e1-c489-41e2-956d-a9b23a4ad89c","html_url":"https://github.com/cloudflare/mmap-sync","commit_stats":null,"previous_names":["cloudflare/mmap-sync"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fmmap-sync","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fmmap-sync/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fmmap-sync/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudflare%2Fmmap-sync/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudflare","download_url":"https://codeload.github.com/cloudflare/mmap-sync/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254121345,"owners_count":22018145,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["interprocess-communication","memory-mapping","synchronization","wait-free","zero-copy"],"created_at":"2024-08-01T15:01:34.010Z","updated_at":"2025-05-14T11:11:07.396Z","avatar_url":"https://github.com/cloudflare.png","language":"Rust","readme":"# mmap-sync\n![build](https://img.shields.io/github/actions/workflow/status/cloudflare/mmap-sync/ci.yml?branch=main)\n[![docs.rs](https://docs.rs/mmap-sync/badge.svg)](https://docs.rs/mmap-sync)\n[![crates.io](https://img.shields.io/crates/v/mmap-sync.svg)](https://crates.io/crates/mmap-sync)\n[![License](https://img.shields.io/badge/license-Apache%202.0-blue)](LICENSE)\n\n`mmap-sync` is a Rust crate designed to manage high-performance, concurrent data access between a single writer process and multiple reader processes, leveraging the benefits of memory-mapped files, wait-free synchronization, and zero-copy deserialization.\nWe're using `mmap-sync` for large-scale machine learning, detailed in our blog post: [\"Every Request, Every Microsecond: Scalable machine learning at Cloudflare\"](http://blog.cloudflare.com/scalable-machine-learning-at-cloudflare).\n\n## Overview\nAt the core of `mmap-sync` is a `Synchronizer` structure that offers a simple interface with \"write\" and \"read\" methods, allowing users to read and write any Rust struct (`T`) that implements or derives certain rkyv traits.\n\n```rust\nimpl Synchronizer {\n    /// Write a given `entity` into the next available memory mapped file.\n    pub fn write\u003cT\u003e(\u0026mut self, entity: \u0026T, grace_duration: Duration) -\u003e Result\u003c(usize, bool), SynchronizerError\u003e {\n        …\n    }\n\n    /// Reads and returns `entity` struct from mapped memory wrapped in `ReadResult`\n    pub fn read\u003cT\u003e(\u0026mut self) -\u003e Result\u003cReadResult\u003cT\u003e, SynchronizerError\u003e {\n        …\n    }\n}\n```\n\nData is stored in shared mapped memory, allowing the `Synchronizer` to \"write\" and \"read\" from it concurrently.\nThis makes `mmap-sync` a highly efficient and flexible tool for managing shared, concurrent data access.\n\n## Mapped Memory\nThe use of memory-mapped files offers several advantages over other inter-process communication (IPC) mechanisms.\nIt allows different processes to access the same memory space, bypassing the need for costly serialization and deserialization.\nThis design allows `mmap-sync` to provide extremely fast, low-overhead data sharing between processes.\n\n## Wait-free Synchronization\nOur wait-free data access pattern draws inspiration from [Linux kernel's Read-Copy-Update (RCU) pattern](https://www.kernel.org/doc/html/next/RCU/whatisRCU.html) and the [Left-Right concurrency control technique](https://github.com/pramalhe/ConcurrencyFreaks/blob/master/papers/left-right-2014.pdf).\nIn our solution, we maintain two copies of the data in separate memory-mapped files.\nWrite access to this data is managed by a single writer, with multiple readers able to access the data concurrently.\n\nWe store the synchronization state, which coordinates access to these data copies, in a third memory-mapped file, referred to as \"state\".\nThis file contains an atomic 64-bit integer, which represents an `InstanceVersion` and a pair of additional atomic 32-bit variables, tracking the number of active readers for each data copy.\nThe `InstanceVersion` consists of the currently active data file index (1 bit), the data size (39 bits, accommodating data sizes up to 549 GB), and a data checksum (24 bits).\n\n## Zero-copy Deserialization\nTo efficiently store and fetch data, `mmap-sync` utilizes zero-copy deserialization with the help of the [rkyv](https://rkyv.org/) library, directly referencing bytes in the serialized form.\nThis significantly reduces the time and memory required to access and use data.\nThe templated type `T` for `Synchronizer` can be any Rust struct implementing specified `rkyv` traits.\n\n## Getting Started\nTo use `mmap-sync`, add it to your `Cargo.toml` under `[dependencies]`:\n```toml\n[dependencies]\nmmap-sync = \"2.0.0\"\n```\nThen, import `mmap-sync` in your Rust program:\n```rust\nuse mmap_sync::synchronizer::Synchronizer;\n```\n\nCheck out the provided examples for detailed usage:\n* [Writer process example](examples/writer.rs): This example demonstrates how to define a Rust struct and write it into shared memory using `mmap-sync`.\n* [Reader process example](examples/reader.rs): This example shows how to read data written into shared memory by a writer process.\n\nThese examples share a [common](examples/common/mod.rs) module that defines the data structure being written and read.\n\nTo run these examples, follow these steps:\n\n1. Open a terminal and navigate to your project directory.\n2. Execute the writer example with the command `cargo run --example writer`.\n3. In the same way, run the reader example using `cargo run --example reader`.\n\nUpon successful execution of these examples, the terminal output should resemble:\n```shell\n# Writer example\nwritten: 36 bytes | reset: false\n# Reader example\nversion: 7 messages: [\"Hello\", \"World\", \"!\"]\n```\n\nMoreover, the following files will be created:\n```shell\n$ stat -c '%A %s %n' /tmp/hello_world_*\n-rw-r----- 36 /tmp/hello_world_data_0\n-rw-r----- 36 /tmp/hello_world_data_1\n-rw-rw---- 16 /tmp/hello_world_state\n```\n\nWith these steps, you can start utilizing `mmap-sync` in your Rust applications for efficient concurrent data access across processes.\n\n## Tuning performance\nUsing `tmpfs` volume will reduce the disk I/O latency since it operates directly on RAM, offering faster read and write capabilities compared to conventional disk-based storage:\n```rust\nlet mut synchronizer = Synchronizer::new(\"/dev/shm/hello_world\".as_ref());\n```\nThis change points the synchronizer to use a shared memory object located in a `tmpfs` filesystem, which is typically mounted at `/dev/shm` on most Linux systems. This should help alleviate some of the bottlenecks associated with disk I/O.\nIf `/dev/shm` does not provide enough space or if you want to create a dedicated `tmpfs` instance, you can set up your own with the desired size. For example, to create a 1GB `tmpfs` volume, you can use the following command:\n```shell\nsudo mount -t tmpfs -o size=1G tmpfs /mnt/mytmpfs\n```\n\n## Benchmarks\nTo run benchmarks you first need to install `cargo-criterion` binary:\n```shell\ncargo install cargo-criterion\n```\n\nThen you'll be able to run benchmarks with the following command:\n```shell\ncargo criterion --bench synchronizer\n```\n\nBenchmarks presented below are executed on Linux laptop with `13th Gen Intel(R) Core(TM) i7-13800H` processor and compiler flags set to `RUSTFLAGS=-C target-cpu=native`.\n```shell\nsynchronizer/write\n    time:   [250.71 ns 251.42 ns 252.41 ns]\n    thrpt:  [3.9619 Melem/s 3.9774 Melem/s 3.9887 Melem/s]\n\nsynchronizer/write_raw\n    time:   [145.25 ns 145.53 ns 145.92 ns]\n    thrpt:  [6.8531 Melem/s 6.8717 Melem/s 6.8849 Melem/s]\n\nsynchronizer/read/check_bytes_true\n    time:   [40.114 ns 40.139 ns 40.186 ns]\n    thrpt:  [24.884 Melem/s 24.914 Melem/s 24.929 Melem/s]\n\nsynchronizer/read/check_bytes_false\n    time:   [26.658 ns 26.673 ns 26.696 ns]\n    thrpt:  [37.458 Melem/s 37.491 Melem/s 37.512 Melem/s]\n```","funding_links":[],"categories":["Rust"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudflare%2Fmmap-sync","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudflare%2Fmmap-sync","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudflare%2Fmmap-sync/lists"}