{"id":13595298,"url":"https://github.com/chinedufn/dipa","last_synced_at":"2025-04-05T05:07:13.288Z","repository":{"id":38892061,"uuid":"211506719","full_name":"chinedufn/dipa","owner":"chinedufn","description":"dipa makes it easy to efficiently delta encode large Rust data structures.","archived":false,"fork":false,"pushed_at":"2022-01-24T08:47:21.000Z","size":884,"stargazers_count":264,"open_issues_count":2,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-10-18T07:11:21.782Z","etag":null,"topics":["delta","delta-compression","delta-encoding","diff","patch","state"],"latest_commit_sha":null,"homepage":"https://chinedufn.github.io/dipa","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/chinedufn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-09-28T13:43:11.000Z","updated_at":"2024-08-20T07:42:22.000Z","dependencies_parsed_at":"2022-09-18T04:40:33.327Z","dependency_job_id":null,"html_url":"https://github.com/chinedufn/dipa","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinedufn%2Fdipa","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinedufn%2Fdipa/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinedufn%2Fdipa/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/chinedufn%2Fdipa/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/chinedufn","download_url":"https://codeload.github.com/chinedufn/dipa/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247289428,"owners_count":20914464,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["delta","delta-compression","delta-encoding","diff","patch","state"],"created_at":"2024-08-01T16:01:47.296Z","updated_at":"2025-04-05T05:07:13.264Z","avatar_url":"https://github.com/chinedufn.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# dipa [![Actions Status](https://github.com/chinedufn/dipa/workflows/ci/badge.svg)](https://github.com/chinedufn/dipa/actions) [![docs](https://docs.rs/dipa/badge.svg)](https://docs.rs/dipa)\n\n\u003e dipa makes it easy to efficiently delta encode large Rust data structures.\n\nIn some applications, the data that you are sending to the client is often almost exactly the same as the data\nthat you last sent to them.\n\nRather than repeatedly sending nearly identical state objects to a client, an application might calculate\nwhat has changed since the last time data was sent and then only send down those changes.\n\nThis approach can dramatically reduce both your and your users' bandwidth requirements and network traffic\ncosts.\n\nThe process of determining what has changed between two instances of a data structure is known as delta encoding.\n\nHistorically, delta encoding code would become more and more difficult to maintain as your application's\ndata structures grew more and more complex.\n\nThis made it a tedious optimization reserved for only the most bandwidth sensitive applications, such as networked\ngames.\n\ndipa eliminates the maintainability challenges of efficient delta encoding code by generating all of the code for you.\n\ndipa is designed to generate very tiny diffs by default. In the most sensitive cases where you have application specific\nknowledge about your data structures that could help you generate even tinier diffs, you can implement the traits\nfor that type yourself and let dipa's derive macro take care of the rest.\n\n_Note that **dipa does not know anything about networks and has no networking code**.\nIt is only focused on encoding deltas, not transmitting them._\n\n## [The dipa Book][book]\n\n[The dipa Book][book] will introduce you to the library and teach you how to use it.\n\nIt is also available offline:\n\n```sh\n# Do this once while online\ngit clone git@github.com:chinedufn/dipa.git \u0026\u0026 cd dipa\ncargo install mdbook\n\n# This works offline\n./bin/serve-book.sh\n```\n\n## Quickstart\n\nThe easiest way to get started with dipa is by using the `#[derive(DiffPatch)]` macro. Here's a quick peek.\n\n\u003cdetails\u003e\n\u003csummary\u003e\nClick to show Cargo.toml.\n\u003c/summary\u003e\n\n```toml\n[dependencies]\n\nbincode = \"1\"\ndipa = { version = \"0.1\", features = [\"derive\"] }\nserde = { version = \"1\", features = [\"derive\"] }\n```\n\u003c/details\u003e\n\u003cp\u003e\u003c/p\u003e\n\n```rust\nuse dipa::{DiffPatch};\nuse std::borrow::Cow;\n\n#[derive(DiffPatch)]\nstruct MyClientState {\n    id: u32,\n    friends: Option\u003cu8\u003e,\n    position: Position,\n    notifications: Vec\u003cCow\u003c\u0026'static, str\u003e\u003e,\n    emotional_state: EmotionalState\n}\n\n#[derive(DiffPatch)]\nstruct Position {\n    x: f32,\n    y: f32,\n    z: f32\n}\n\n#[derive(DiffPatch)]\nenum EmotionalState {\n    Peace { calibration: u128 },\n    Love(u64),\n    Courage(u32),\n}\n\nfn main() {\n    let mut old_client_state = MyClientState {\n        id: 308,\n        friends: None,\n        position: Position { x: 1., y: 2., z: 3. }\n        notifications: vec![Cow::Borrowed(\"let\"), Cow::Owned(\"go\".to_string())],\n        emotional_state: EmotionalState::Love(100),\n    };\n\n    let new_client_state = MyClientState {\n        id: 308,\n        friends: Some(1),\n        position: Position { x: 4., y: 2., z: 3. }\n        notifications: vec![Cow::Borrowed(\"free\")]\n        emotional_state: EmotionalState::Peace { calibration: 10_000 },\n    };\n\n    let delta_created = old_client_state.create_delta_towards(\u0026new_client_state);\n\n    // Consider using bincode to serialize your diffs on the server side.\n    // You can then send them over the wire and deserialize them on the client side.\n    //\n    // For the tiniest diffs, be sure to use variable integer encoding.\n    let bin = bincode::options().with_varint_encoding();\n\n    let serialized = bin.serialize(\u0026delta_created.delta).unwrap();\n\n    // ... Pretend you send the data to the client ...\n\n    let deserialized: \u003cMyClientState as dipa::Diffable\u003c'_, '_, MyClientState\u003e\u003e::DeltaOwned = \n        bin.deserialize(\u0026serialized).unwrap();\n\n    old_client_state.apply_patch(deserialized);\n\n    // All of the fields are now equal.\n    assert_eq!(\n      old_client_state.notifications,\n      new_client_state.notifications\n    );\n}\n```\n\n[See the full API Documentation](https://docs.rs/dipa)\n\n## Advanced Usage\n\nFor applications where incredibly small payloads are a top priority, you may wish to take advantage of knowledge about how your application works in order to \ngenerate even smaller diffs.\n\nFor example, say you have the following client state data structure.\n\n```rust\n#[derive(DiffPatch)]\nstruct ClientState {\n    hair_length: u128\n}\n```\n\nIf the hair length hasn't changed the diff will be a single byte.\n\nHowever, whenever the client's hair length changes there will be up to an additional 17\\* bytes in the payload to variable integer\nencode the new `u128` value.\n\nBut, what if you knew that it was impossible for a client's hair length to ever change by more than `100` units in between state updates?\n\nAnd, your application requirements mean that saving every byte matters and so it is worth your time to customize your hair\nlength delta encoding.\n\nIn this case, you could go for something like:\n\n```rust\nuse dipa::{CreatedDelta, Diffable, Patchable};\n\n#[derive(DiffPatch)]\n// Debug + PartialEq are used by DipaImplTester below.\n// They are not required otherwise.\n#[cfg_attr(test, derive(Debug, PartialEq))]\nstruct ClientState {\n    hair_length: DeltaWithI8,\n}\n\n// Debug + PartialEq are used by DipaImplTester below.\n// They are not required otherwise.\n#[cfg_attr(test, derive(Debug, PartialEq))]\nstruct DeltaWithI8(u128);\n\nimpl\u003c's, 'e\u003e Diffable\u003c's, 'e, DeltaWithI8\u003e for DeltaWithI8 {\n    type Delta = i8;\n    type DeltaOwned = Self::Delta;\n\n    fn create_delta_towards(\u0026self, end_state: \u0026Self) -\u003e CreatedDelta\u003cSelf::Delta\u003e {\n        let delta = if self.0 \u003e= end_state.0 {\n            (self.0 - end_state.0) as i8 * -1\n        } else {\n            (end_state.0 - self.0) as i8\n        };\n\n        CreatedDelta {\n            delta,\n            did_change: self.0 != end_state.0,\n        }\n    }\n}\n\nimpl Patchable\u003ci8\u003e for DeltaWithI8 {\n    fn apply_patch(\u0026mut self, patch: i8) {\n        if patch \u003e= 0 {\n            self.0 += patch as u128;\n        } else {\n            self.0 -= (-1 * patch) as u128;\n        }\n    }\n}\n\n#[cfg(test)]\nmod tests {\n    use super::*;\n    use dipa::DipaImplTester;\n\n    #[test]\n    fn delta_with_i8() {\n        DipaImplTester {\n            label: Some(\"Unchanged DeltaWithI8\"),\n            start: \u0026mut DeltaWithI8(300),\n            end: \u0026DeltaWithI8(300),\n            expected_delta: 0,\n            expected_serialized_patch_size: 1,\n            expected_did_change: false,\n        }\n        .test();\n\n        DipaImplTester {\n            label: Some(\"Increase DeltaWithI8\"),\n            start: \u0026mut DeltaWithI8(5_000),\n            end: \u0026DeltaWithI8(5_050),\n            expected_delta: 50,\n            expected_serialized_patch_size: 1,\n            expected_did_change: true,\n        }\n        .test();\n\n        DipaImplTester {\n            label: Some(\"Decrease DeltaWithI8\"),\n            start: \u0026mut DeltaWithI8(400),\n            end: \u0026DeltaWithI8(320),\n            expected_delta: -80,\n            expected_serialized_patch_size: 1,\n            expected_did_change: true,\n        }\n        .test();\n    }\n}\n```\n\nThis approach would reduce hair length delta from 17 bytes down to just a single byte.\n\n\\* - _17, not 16, since integers larger than `u8/i8` are wrapped in `Option` by their default `DiffPatch` implementation. This optimizes for the case when the integer does not change since `None` serializes to 1 byte._\n\n## Questions\n\nIf you have a question that you can't find the answer to within five minutes then this is considered a documentation bug.\n\nPlease [open an issue](https://github.com/chinedufn/dipa/issues/new) with your question.\n\nOr, even better, a work-in-progress pull request with a skeleton of a code example,\nAPI documentation or area in the book where your question could have been answered.\n\n## Contributing\n\nIf you have a use case that isn't supported, a question, a patch, or anything else, go right ahead and open an issue or submit a pull request.\n\n## To Test\n\nTo run the test suite.\n\n```sh\n# Clone the repository\ngit clone git@github.com:chinedufn/dipa.git\ncd dipa\n\n# Run tests\ncargo test --all\n```\n\n#### License\n\n\u003csup\u003e\nLicensed under either of \u003ca href=\"LICENSE-APACHE\"\u003eApache License, Version\n2.0\u003c/a\u003e or \u003ca href=\"LICENSE-MIT\"\u003eMIT license\u003c/a\u003e at your option.\n\u003c/sup\u003e\n\n\u003cbr\u003e\n\n\u003csub\u003e\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in this project by you, as defined in the Apache-2.0 license,\nshall be dual licensed as above, without any additional terms or conditions.\n\u003c/sub\u003e\n\n[book]: https://chinedufn.github.io/dipa\n[apache]: ./LICENSE-APACHE\n[mit]: ./LICENSE-MIT\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchinedufn%2Fdipa","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fchinedufn%2Fdipa","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fchinedufn%2Fdipa/lists"}