{"id":13648528,"url":"https://github.com/jonhoo/atone","last_synced_at":"2025-05-16T08:04:25.464Z","repository":{"id":53228964,"uuid":"277157288","full_name":"jonhoo/atone","owner":"jonhoo","description":"A `VecDeque` (and `Vec`) variant that spreads resize load across pushes.","archived":false,"fork":false,"pushed_at":"2024-12-31T10:26:37.000Z","size":165,"stargazers_count":107,"open_issues_count":1,"forks_count":10,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-15T01:49:08.430Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jonhoo.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-04T17:36:59.000Z","updated_at":"2025-04-05T21:08:07.000Z","dependencies_parsed_at":"2024-01-14T10:59:24.899Z","dependency_job_id":"d0e85952-16be-4a1a-9cc2-fe1dc1c41903","html_url":"https://github.com/jonhoo/atone","commit_stats":{"total_commits":115,"total_committers":12,"mean_commits":9.583333333333334,"dds":"0.20869565217391306","last_synced_commit":"d5d0d747ceb3710a2be08b0dadaa87f6b6c78504"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Fatone","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Fatone/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Fatone/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jonhoo%2Fatone/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jonhoo","download_url":"https://codeload.github.com/jonhoo/atone/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254493378,"owners_count":22080126,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:04:19.600Z","updated_at":"2025-05-16T08:04:20.455Z","avatar_url":"https://github.com/jonhoo.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"[![Crates.io](https://img.shields.io/crates/v/atone.svg)](https://crates.io/crates/atone)\n[![Documentation](https://docs.rs/atone/badge.svg)](https://docs.rs/atone/)\n[![codecov](https://codecov.io/gh/jonhoo/atone/graph/badge.svg?token=1HqvHNePdL)](https://codecov.io/gh/jonhoo/atone)\n![Maintenance](https://img.shields.io/badge/maintenance-experimental-blue.svg)\n\nA `VecDeque` (and `Vec`) variant that spreads resize load across pushes.\n\nMost vector-like implementations, such as `Vec` and `VecDeque`, must\noccasionally \"resize\" the backing memory for the vector as the number of\nelements grows. This means allocating a new vector (usually of twice the\nsize), and moving all the elements from the old vector to the new one.\nAs your vector gets larger, this process takes longer and longer.\n\nFor most applications, this behavior is fine — if some very small number\nof pushes take longer than others, the application won't even notice.\nAnd if the vector is relatively small anyway, even those \"slow\" pushes\nare quite fast. Similarly, if your vector grow for a while, and then\n_stops_ growing, the \"steady state\" of your application won't see any\nresizing pauses at all.\n\nWhere resizing becomes a problem is in applications that use vectors to\nkeep ever-growing state where tail latency is important. At large scale,\nit is simply not okay for one push to take 30 milliseconds when most\ntake double-digit **nano**seconds. Worse yet, these resize pauses can\ncompound to create [significant spikes] in tail latency.\n\nThis crate implements a technique referred to as \"incremental resizing\",\nin contrast to the common \"all-at-once\" approached outlined above. At\nits core, the idea is pretty simple: instead of moving all the elements\nto the resized vector immediately, move a couple each time a push\nhappens. This spreads the cost of moving the elements so that _each_\npush becomes a little slower until the resize has finished, instead of\n_one_ push becoming a _lot_ slower.\n\nThis approach isn't free, however. While the resize is going on, the old\nvector must be kept around (so memory isn't reclaimed immediately), and\niterators and other vector-wide operations must access both vectors,\nwhich makes them slower. Only once the resize completes is the old\nvector reclaimed and full performance restored.\n\nTo help you decide whether this implementation is right for you, here's\na handy reference for how this implementation compares to the standard\nlibrary vectors:\n\n - Pushes all take approximately the same time.\n   After a resize, they will be slower for a while, but only by a\n   relatively small factor.\n - Memory is not reclaimed immediately upon resize.\n - Access operations are marginally slower as they must check two\n   vectors.\n - The incremental vector is slightly larger on the stack.\n - The \"efficiency\" of the resize is slightly lower as the all-at-once\n   resize moves the items from the small vector to the large one in\n   batch, whereas the incremental does a series of pushes.\n\nAlso, since this crate must keep two vectors, it cannot guarantee that\nthe elements are stored in one contiguous chunk of memory. Since it must\nmove elements between then without losing their order, it is backed by\n`VecDeque`s, which means that this is the case even after the resize has\ncompleted. For this reason, this crate presents an interface that\nresembles `VecDeque` more so than `Vec`. Where possible though, it\nprovides `Vec`-like methods. If you need contiguous memory, there's no\ngood way to do incremental resizing without low-level memory mapping\nmagic that I'm aware of.\n\n## Benchmarks\n\nThere is a silly, but illustrative benchmark in `benches/vroom.rs`. It\njust runs lots of pushes back-to-back, and measures how long each one\ntakes. The problem quickly becomes apparent:\n\n```console\n$ cargo bench --bench vroom \u003e vroom.dat\nVec max: 2.440556ms, mean: 25ns\nVecDeque max: 4.512806ms, mean: 25ns\natone::Vc max: 25.789µs, mean: 26ns\n```\n\nYou can see that the standard library implementations have some pretty\nsevere latency spikes. This is more readily visible through a timeline\nlatency plot (`misc/vroom.plt`):\n\n![latency spikes on resize](https://raw.githubusercontent.com/jonhoo/atone/master/misc/vroom.png)\n\nResizes happen less frequently as the vector grows, but they also take\nlonger _when_ they occur. With atone, those spikes are mostly gone.\n\n## A note on in-place resizing\n\nSome memory allocators have an API for increasing the size of an\nallocation _in-place_. This operation doesn't always succeed, in which\ncase it falls back to a regular allocation plus a `memcpy`, but when it\ndoes, the resize is basically free. `Vec` and `VecDeque` are\noccasionally able to take advantage of this kind of in-place resizing,\nwhereas `atone::Vc` is not.\n\nIn practice, you are unlikely to get in-place resizing for vectors that\nfit the use-case for `atone::Vc`. If the running application continues\nto allocate memory elsewhere, chances are _something_ will get allocated\nin the space on the heap after the current backing memory for the\nvector, in which case in-place reallocation isn't possible.\n\n`Vec` specifically also has a further optimization (I'm not sure what\nit's called), in which it can perform in-place resizing even when the\nallocation is surrounded on the heap. I assume that this involves some\nmemory remapping trickery. This only works on particular allocators\nthough (it does not work with `jemalloc` for example).\n\n## Implementation\n\natone is backed by two `VecDeque`s. One is the \"current\" vector, and one\nholds any leftovers from the last resize. Logically, the leftovers are\ntreated as the head of the vector, and the current vector is treated as\nthe tail. The \"incremental\" piece on push is then implemented by popping\nelements from the leftovers and pushing them onto the front of the\ncurrent vector. It's this shifting, combined with the need to support\n`push`, that made me go with `VecDeque`, since with a `Vec`, pushing to\nthe front would have to shift all the elements.\n\natone aims to stick as closely to `Vec` and `VecDeque` as it can, both\nin terms of code and API. `src/lib.rs` is virtually\nidentical to [`src/liballoc/collections/vec_deque.rs` in `std`][src] (I\nencourage you to diff them!).\n\n## Why \"atone\"?\n\nWe make the vector atone with more expensive pushes for the sin it committed by resizing..?\n\n[significant spikes]: https://twitter.com/jonhoo/status/1277618908355313670\n[src]: https://github.com/rust-lang/rust/blob/master/src/liballoc/collections/vec_deque.rs\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0\n   ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license\n   ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonhoo%2Fatone","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjonhoo%2Fatone","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjonhoo%2Fatone/lists"}