{"id":21100220,"url":"https://github.com/prataprc/ppar","last_synced_at":"2025-05-16T17:30:26.554Z","repository":{"id":42696275,"uuid":"308348157","full_name":"prataprc/ppar","owner":"prataprc","description":"Persistent / Immutable array in Rust.","archived":false,"fork":false,"pushed_at":"2022-05-22T16:52:15.000Z","size":151,"stargazers_count":6,"open_issues_count":1,"forks_count":3,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-05-11T01:04:36.945Z","etag":null,"topics":["array","concurrent-data-structure","immutable-collections","immutable-datastructures","list","persistent-data-structure","rope","vector"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/prataprc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-10-29T14:09:42.000Z","updated_at":"2024-10-14T19:43:43.000Z","dependencies_parsed_at":"2022-08-23T22:51:17.160Z","dependency_job_id":null,"html_url":"https://github.com/prataprc/ppar","commit_stats":null,"previous_names":["bnclabs/ppar"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fppar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fppar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fppar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/prataprc%2Fppar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/prataprc","download_url":"https://codeload.github.com/prataprc/ppar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254576282,"owners_count":22094329,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["array","concurrent-data-structure","immutable-collections","immutable-datastructures","list","persistent-data-structure","rope","vector"],"created_at":"2024-11-19T23:06:39.332Z","updated_at":"2025-05-16T17:30:26.233Z","avatar_url":"https://github.com/prataprc.png","language":"Rust","readme":"[![Documentation](https://docs.rs/ppar/badge.svg?style=flat-square)](https://docs.rs/ppar)\n\nPackage implement persistent array using a variant of rope data structure.\n\nWhy would you want it ?\n-----------------------\n\nArray is implemented as [std::vec][std_vec] in rust-standard library.\nFor most cases that should be fine. But when we start working with arrays\nthat are super-large and/or step into requirements like  non-destructive\nwrites and concurrent access, we find [std::vec][std_vec] insufficient.\n[im][im] is a popular alternative, but has `insert()` and `delete()`\npenalties similar to `std::vec` for large arrays. While most implementation\nprefer to use [RRB-Tree][rrb], `ppar` is based on [Rope data structure][rope].\n\nHere is a quick list of situation that might require using `ppar`.\n\n* When array size is too large with repeated insert and remove operation.\n* When shared ownership is required.\n* When shared ownership across concurrent threads.\n* To support undo/redo operation for array modifications.\n* When splitting up of array and/or joining arrays are frequently done.\n* Lazy clone of array using copy-on-write.\n\n**Check out our [performance metric](#performance-metric)**\n\nAlgorithm\n---------\n\nFundamentally, it can be viewed as a binary-tree of array-blocks, where\neach leaf-node is a contiguous-block of type `T` items, while intermediate\nnodes only hold references to the child nodes - `left` and `right`.\nTo be more precise, intermediate nodes in the tree are organised similar\nto rope structure, as a tuple of `(weight, left, right)` where weight is\nthe sum of all items present in the leaf-nodes under the left-branch.\n\nA list of alternatives can be found [here](#alternate-solutions). If you\nfind good alternatives please add it to the list and raise a PR.\n\nIf you are planning to use `ppar` for your project, do let us know.\n\nGoals\n-----\n\n- [x] Vector parametrized over type T.\n- [x] Immutable / Persistent collection of Vector\u003cT\u003e.\n- [x] CRUD operation, get(), set(), delete(), insert(), all are persistent.\n- [x] Convert from Vec\u003cT\u003e to ppar::Vector\u003cT\u003e.\n- [x] Convert from ppar::Vector\u003cT\u003e to Vec\u003cT\u003e.\n- [x] Thread safe operations.\n- [x] [std::vec::Vec][std_vector] like mutable API.\n- [ ] Iteration over collection, item-wise, chunk-wise, reverse.\n- [ ] Deduplication.\n- [ ] Membership.\n- [x] Joining collections, splitting into collections.\n- [ ] Partial collection.\n- [ ] Extending collection.\n- [ ] Queue operations, like pop(), push().\n- [ ] Functional ops, like filter, map, reduce.\n- [ ] Sort and search operations.\n- [ ] Trait implementations.\n  - [x] Clone\n  - [ ] Eq, PartialEq, Ord, PartialOrd\n  - [ ] Extend\n  - [ ] From, FromIterator, IntoIterator\n  - [ ] Hash\n  - [ ] Index, IndexMut\n  - [ ] Write\n- [ ] Parallel iteration with [rayon][rayon].\n- [x] Fuzzy tested.\n\nThe basic algorithm is fairly tight. Contributions are welcome to make the\n`ppar::Vector` type as rich as [std::vec::Vec][std_vector] and\n[im::Vector][im_vector].\n\nPerformance metric\n------------------\n\nOn a 2008 core2-duo machine with 8GB RAM. The measurements are taken using\nthe following command and all the numbers are **op** Vs **latency**.\n\n```bash\ncargo run --release --bin perf --features=perf -- --load 100000 --ops 10000\n```\n\n![reads](./reports/2020-11-13/read-ops.png)\n\u003csup\u003eLower numbers are better\u003c/sup\u003e\n![write](./reports/2020-11-13/write-ops.png)\n\u003csup\u003eLower number are better\u003c/sup\u003e\n\nWe are loading the array with an initial data set of `100_000` u64 numbers.\nThen applying each operation in a tight loop, measuring the elapsed time\nfor the entire loop and finally computing per-op latency. `insert`,\n`insert_mut`, `remove`, `remove_mut`, `update`, `update_mut`, and `get` use\nrandom offset into the array. For iteration, we compute the elapsed time\nfor iterating each item.\n\n* Using the logarithm scale implies an order-of-magniture difference in\n  performance.\n* Between `arc` and `rc` variant of `ppar`, performance difference is not much.\n  Though it might make a difference for cache-optimized algorithms.\n* `ppar` gains order-of-magniture improvent over `vec` and `im` for **insert**,\n  **remove**, **load**, **split**, **append**, **clone** operations.\n* And it falls behind on **update**, **get**, **iter** operations. But it may\n  be worth noting that where-ever it falls behind it is still on the lower end\n  of the log-scale, that is, the difference is well within 100ns.\n\nIt is also worth noting that, maximum size of the operated data-set is less\nthan 2MB, which means the entire array footprint can be held inside the\nCPU cache. As far as my understanding goes, when we increase the size of the\narray, the performance difference will only be more pronounced, that is, fast-op\nmight look faster and slow-op might look slower.\n\nUseful links\n------------\n\n* [im][im] contains an alternate array implementation, among other things.\n* [rpds][rpds], another alternate implementation using ``RRB-Tree``.\n* [rope datastructure][rope] used by this package.\n\nContributions\n-------------\n\n* Simple workflow. Fork - Modify - Pull request.\n* Before creating a PR,\n  * Run `make build` to confirm all versions of build is passing with\n    0 warnings and 0 errors.\n  * Run `check.sh` with 0 warnings, 0 errors and all testcases passing.\n  * Run `perf.sh` with 0 warnings, 0 errors and all testcases passing.\n  * [Install][spellcheck] and run `cargo spellcheck` to remove common spelling mistakes.\n* [Developer certificate of origin][dco] is preferred.\n\n[im]: https://github.com/bodil/im-rs\n[im_vector]: https://docs.rs/im/15.0.0/im/struct.Vector.html\n[rope]: https://en.wikipedia.org/wiki/Rope_(data_structure)\n[rpds]: https://github.com/orium/rpds\n[std_vec]: https://doc.rust-lang.org/beta/std/vec/index.html\n[std_vector]: https://doc.rust-lang.org/beta/std/vec/struct.Vec.html\n[rrb]: https://infoscience.epfl.ch/record/213452/files/rrbvector.pdf\n[rayon]: https://crates.io/crates/rayon\n[quickcheck]: https://crates.io/crates/quickcheck\n[dco]: https://developercertificate.org/\n[spellcheck]: https://github.com/drahnr/cargo-spellcheck\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprataprc%2Fppar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fprataprc%2Fppar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fprataprc%2Fppar/lists"}