{"id":13822416,"url":"https://github.com/paulkernfeld/contiguous-data-in-rust","last_synced_at":"2025-10-10T05:51:43.801Z","repository":{"id":141279252,"uuid":"264683259","full_name":"paulkernfeld/contiguous-data-in-rust","owner":"paulkernfeld","description":"An opinionated guide that tries to help you choose the best way to store contiguous data in Rust","archived":false,"fork":false,"pushed_at":"2020-08-09T18:42:25.000Z","size":48,"stargazers_count":123,"open_issues_count":2,"forks_count":4,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-05-16T17:40:38.065Z","etag":null,"topics":["rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paulkernfeld.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-05-17T14:17:14.000Z","updated_at":"2025-05-15T13:19:33.000Z","dependencies_parsed_at":"2024-08-04T08:08:23.039Z","dependency_job_id":null,"html_url":"https://github.com/paulkernfeld/contiguous-data-in-rust","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/paulkernfeld/contiguous-data-in-rust","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulkernfeld%2Fcontiguous-data-in-rust","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulkernfeld%2Fcontiguous-data-in-rust/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulkernfeld%2Fcontiguous-data-in-rust/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulkernfeld%2Fcontiguous-data-in-rust/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paulkernfeld","download_url":"https://codeload.github.com/paulkernfeld/contiguous-data-in-rust/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulkernfeld%2Fcontiguous-data-in-rust/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279002866,"owners_count":26083468,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-10T02:00:06.843Z","response_time":62,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rust"],"created_at":"2024-08-04T08:01:59.455Z","updated_at":"2025-10-10T05:51:43.784Z","avatar_url":"https://github.com/paulkernfeld.png","language":"Rust","funding_links":[],"categories":["Rust","rust"],"sub_categories":[],"readme":"# A Guide to Contiguous Data in Rust\n\nMany modern languages have collections called \"array,\" \"slice,\" or \"vector.\" Rust has all three, plus many third-party libraries! This is an opinionated guide that tries to help you choose the best way to store contiguous data in Rust. It covers tools from Rust's core and standard library as well as third-party crates that meet more specific needs.\n\nContiguous data is when multiple pieces of data are stored next to each other in memory.  It is often a great way to store collections, because it can provide great cache locality and branch prediction.\n\n# Tradeoffs\n\nThese are things to think about when choosing a technique for storing contiguous data.\n\n## Mutability\n\nThere are a few broad levels of mutability for contiguous data in Rust:\n\n- Completely immutable\n- Fixed length, mutable contents\n- Mutable length and contents\n\n## Allocation\n\nSome solutions can be used with `const` and `static`, some solutions can use the memory of the stack, and some need to use memory allocated by an allocator (I'll call this \"the heap\" although I don't think that it technically has to be a heap).\n\n## Splitting\n\nThere are several ways that you can split contiguous data in Rust. Reviewing [References and Borrowing](https://doc.rust-lang.org/book/ch04-02-references-and-borrowing.html) from TRPL might be helpful to understanding this better. Since any contiguous data can be turned into a slice, the slice splitting rules generally apply for borrowed data. The `bytes` crate provides an interesting way to split owned data. \n\n## Extend vs. refuse\n\nWhen you run out of memory in your contiguous data, should the data structure ask to allocate more memory or should it refuse to give you more memory? If you want to intentionally use a capped amount of memory, it might be better to refuse to extend the memory. This could be useful in embedded and/or real-time programming.\n\n## Types\n\nMost of the solutions below can store any type but there are a couple exceptions.\n\n# Solutions\n\nThe solutions are roughly in order of increasing power, according to the [Principle of Least Power](https://www.lihaoyi.com/post/StrategicScalaStylePrincipleofLeastPower.html#dependency-injection). So, consider using the first solution that will solve your problem.\n\n## Fixed-size array: `[T; N]`\n\nWhen you create a [fixed-size array](https://doc.rust-lang.org/std/primitive.array.html) in Rust, you must specify the size (`N`) at compile time.\n\nGiven a mutable fixed-size array, you can change the content, but there is no way to change the size.\n\n```rust\nconst MY_DATA: [i8; 3] = [1, 2, 3];\n\nfn main() {\n    let mut my_data = [2; 3];\n    my_data[0] = 1;\n    my_data[2] = 3;\n    assert_eq!(MY_DATA, my_data);\n}\n```\n\nBecause the size is known at compile time, a fixed-size array can be stored anywhere, even to be used as a `const`. However, it might not be a good idea to store large fixed-size arrays on the stack because it could cause a stack overflow.\n\nThe length of the array is actually part of its type. This means that the following code, for example, won't compile because you can't compare two variables with different types:\n\n```\nconst MY_DATA_3: [i8; 3] = [1, 2, 3];\nconst MY_DATA_4: [i8; 4] = [1, 2, 3, 4];\n\nfn main() {\n    assert_eq!(MY_DATA_3, MY_DATA_4);\n}\n```\n\nAn array cannot be split.\n\nSee also [Array types](https://doc.rust-lang.org/reference/types/array.html) from The Rust Reference.\n\n## Slice: `\u0026[T]` or `\u0026mut [T]`\n\nIn Rust, a [slice](https://doc.rust-lang.org/std/primitive.slice.html) is \"a dynamically-sized view into a contiguous sequence.\" In contrast to an array, the length of a slice isn't known at compile time.\n\nGiven a mutable slice, you can change the content, but there is no way to change the length. When we take a mutable slice to an array, we're actually modifying the data in the array itself. That's why `my_array` needs to be declared as `mut` below.\n\n```rust\nfn main() {\n    let mut my_array: [i8; 3] = [1, 2, 0];\n    let my_slice: \u0026mut [i8] = \u0026mut my_array[1..];\n    my_slice[1] = 3;\n    assert_eq!(my_array, [1, 2, 3]);\n}\n```\n\nUnlike with an an array, two slices of different lengths _are_ the same type, so the code below works:\n\n```rust\nconst MY_SLICE: \u0026[i8] = \u0026[1, 2, 3];\n\nfn main() {\n    let my_slice: \u0026[i8] = \u0026[1, 2, 3, 4];\n    assert_ne!(MY_SLICE, my_slice);\n}\n```\n\nA slice can refer to memory anywhere including as a `const`.\n\nYou can always create as many overlapping shared slices (`\u0026[T]`) as you want, although you can't mutate them:\n\n```rust\nconst MY_DATA: [i8; 3] = [2; 3];\n\nfn main() {\n    // Compare two overlapping slices\n    assert_eq!(\u0026MY_DATA[..2], \u0026MY_DATA[1..]);\n}\n```\n\nYou can split a mutable slice into multiple non-overlapping mutable slices:\n\n```rust\nfn main() {\n    let my_data = \u0026mut [0, 2, 3, 0];\n    let (left, right) = my_data.split_at_mut(2);\n    left[0] = 1;\n    right[1] = 4;\n    assert_eq!(my_data, \u0026[1, 2, 3, 4]);\n}\n```\n\nSee also [TRPL on slices](https://doc.rust-lang.org/book/ch04-03-slices.html).\n\n## Boxed slice: `Box\u003c[T]\u003e`\n\nWith a boxed slice, you can create arrays at run time without knowing the length at compile time. In most cases, you could probably just use `Vec` instead. However, boxed slices do have a few use cases:\n\n- Writing a custom buffer (e.g. [`std::io::BufReader`](https://doc.rust-lang.org/std/io/struct.BufReader.html))\n- If you want to store data slightly more efficiently than a `Vec` (a boxed slice doesn't store a capacity)\n- If you generally want to prevent users from resizing the data\n\nThe code below won't compile; you can't collect an iterator into a fixed-size array because the number of elements in an iterator isn't generally known at compile time.\n\n```\nfn main() {\n    let my_data: [i8; 3] = [1, 2, 3].iter().cloned().collect();\n}\n```\n\n...but you can collect into a boxed slice:\n\n```rust\nconst MY_DATA: [i8; 3] = [1, 2, 3];\n\n\nfn main() {\n    let my_data: Box\u003c[i8]\u003e = MY_DATA.iter().cloned().collect();\n\n    // We need to take a slice b/c we can't compare boxed slices and arrays directly\n    assert_eq!(\u0026my_data[..], MY_DATA);\n}\n```\n\nBecause the length isn't known at compile time, the data for a boxed slice can _only_ be allocated with an allocator. \n\n## `Vec`\n\n[`std::vec::Vec`](https://doc.rust-lang.org/std/vec/struct.Vec.html) is the `std` way to do variable-length contiguous data. When you try to add data and there's not enough room, the data structure will allocate more memory for the data.\n\n```rust\nfn main() {\n    let mut vec = Vec::new();\n    vec.push(1);\n    vec.push(2);\n    vec.push(3);\n    \n    assert_eq!(vec.len(), 3);\n    assert_eq!(vec, [1, 2, 3]);\n}\n``` \n\nBecause the length is dynamic and not known at compile time, the data for a `Vec` can only live on the heap.\n\nWe can split a `Vec` into two separate `Vec`s using the `split_off` method. However, doing this (surprisingly?) copies the data. [According to user hanna-kruppe on the Rustlang forum,](https://users.rust-lang.org/t/split-owned-vec-t-without-reallocation/6346/2):\n\n\u003e But as far as I know, there’s currently no way to tell the allocator “Hey I got this piece of memory from you, now please pretend you gave it to me as two separate, contiguous allocations”\n\nSee also [Storing Lists of Values with Vectors](Storing Lists of Values with Vectors) from _The Rust Programming Language_.\n\n\n## `smallvec`\n\nThe [`smallvec`](https://github.com/servo/rust-smallvec) provides an interface that is very close to Vec, but it will store small numbers of items on the stack instead of allocating on the heap. This is good if you suspect your data structure will normally be quite small but it may need to grow occasionally.\n\nWhereas an array with type `[T; 4]` stores exactly 4 elements, a `SmallVec` of type `SmallVec[T; 4]` can store more than 4 elements. If it has 4 or fewer elements, it will be stored on the stack; otherwise, it will be stored on the heap.\n\n```rust\nuse smallvec::{SmallVec, smallvec};\n    \nfn main() {\n    // This SmallVec can hold up to 4 items on the stack:\n    let mut v: SmallVec\u003c[i32; 4]\u003e = smallvec![1, 2, 0, 4];\n    \n    // It will automatically move its contents to the heap if\n    // contains more than four items:\n    v.push(5);\n    \n    // SmallVec points to a slice, so you can use normal slice\n    // indexing and other methods to access its contents:\n    v[2] = 3;\n    assert_eq!(\u0026[1, 2, 3, 4, 5], \u0026v[..]);\n}\n```\n\nSee also [When is it “morally” correct to use smallvec?](https://users.rust-lang.org/t/when-is-it-morally-correct-to-use-smallvec/46375) from The Rust Programming Language Forum.\n\n## `arrayvec`\n\nThis crate will let you store a Vec inside of an array, but it won't let you exceed the length of the underlying array. This means that the data can live in the data segment, on the stack, or on the heap.\n\n[arrayvec](https://docs.rs/arrayvec)\n\n```rust\nuse arrayvec::ArrayVec;\n\nfn main() {\n    let mut array = ArrayVec::\u003c[_; 2]\u003e::new();\n    assert_eq!(array.try_push(1), Ok(()));\n    assert_eq!(array.try_push(2), Ok(()));\n    assert!(array.try_push(3).is_err());\n    assert_eq!(\u0026array[..], \u0026[1, 2]);\n}\n```\n\n# `tinyvec`\n\n`tinyvec` provides 100%-safe alternatives to both `arrayvec` and `smallvec`. It works pretty much the same way except that the types must implement `Default`. Note that it doesn't provide the exact same APIs.\n\n```rust\nuse tinyvec::ArrayVec;\n\nfn main() {\n    let mut array = ArrayVec::\u003c[_; 2]\u003e::new();\n    array.push(1);\n    array.push(2);\n}\n```\n\n```rust,should_panic\nuse tinyvec::ArrayVec;\n\nfn main() {\n    let mut array = ArrayVec::\u003c[_; 2]\u003e::new();\n    array.push(1);\n    array.push(2);\n    array.push(3);\n}\n```\n\n## `VecDeque`\n\n[`std::collections::VecDeque`](https://doc.rust-lang.org/std/collections/vec_deque/struct.VecDeque.html) is \"a double-ended queue implemented with a growable ring buffer.\" It's pretty similar to a `Vec` except that it can efficiently push and pop from both front and back. This means that `VecDeque` can be used as a queue or as a double-ended queue. See [`std::collections` docs](https://doc.rust-lang.org/stable/std/collections/index.html) for more details.\n\n```rust\nuse std::collections::VecDeque;\n\nfn main() {\n    let mut vec_deque = VecDeque::with_capacity(3);\n    vec_deque.push_back(0);\n    vec_deque.push_back(2);\n    vec_deque.push_back(3);\n    assert_eq!(Some(0), vec_deque.pop_front());\n    vec_deque.push_front(1);\n\n    assert_eq!(\u0026vec_deque, \u0026[1, 2, 3]);\n}\n```\n\nLike a `Vec`, the `VecDeque` data lives on the heap.\n\n## The `bytes` crate \n\n[bytes](https://docs.rs/bytes) provides `Bytes`, \"an efficient container for storing and operating on contiguous slices of memory.\" One of its signature features is that, unlike `Vec`, it allows you to split ownership of data without copying. Unlike the other tools in this guide, the `bytes` crate can't store types `T`, it can only store `u8`.\n\n```rust\nuse bytes::{BytesMut, BufMut};\n\nfn main() {\n    let mut whole = BytesMut::with_capacity(1024);\n    whole.put(\u0026[1u8, 2, 3, 4] as \u0026[u8]);\n    \n    let mut right = whole.split_off(2);\n    let mut left = whole;\n    \n    std::thread::spawn(move || {\n        right[1] = 6;\n        assert_eq!(\u0026right[..], [3, 6]);\n    });\n    \n    left[0] = 5;\n    assert_eq!(\u0026left[..], [5, 2]);\n}\n```\n\nThe data for the `bytes` data structures will live on the heap.\n\n## Honorable Mention\n\nThese are a few techniques that are good to know about, but the use cases are pretty narrow or there are major drawbacks.\n\n- A boxed array (`Box\u003c[T; N]\u003e`) is _not_ the same as a boxed slice (`Box\u003c[T]\u003e`). You probably don't want to use a boxed array; among other things, there are a couple [major ergonomics drawbacks](https://users.rust-lang.org/t/when-would-you-want-to-use-a-boxed-array/46658/4) at the time of writing. See also [stack overflow](https://github.com/rust-lang/rust/issues/53827).\n- [Slice Deque](https://crates.io/crates/slice-deque) is a double-ended queue that uses operating system virtual memory trickery to let you see the entire contents of the queue as a slice.\n- [tendril](https://crates.io/crates/tendril) is a crate from the Servo org boasting really advanced features for zero-copy parsing.\n- [ndarray](https://crates.io/crates/ndarray) and [nalgebra](https://www.nalgebra.org/) for math-heavy and multidimensional arrays.\n\n# More Resources\n\n- [Smart Pointers](https://doc.rust-lang.org/book/ch15-00-smart-pointers.html) from _The Rust Programming Language_.\n- [Arrays and Slices](https://doc.rust-lang.org/stable/rust-by-example/primitives/array.html) from _Rust By Example_\n- [Slice types](https://doc.rust-lang.org/reference/types/slice.html) from The Rust Reference\n\n## Thanks\n\nThanks to soruh_c10 on Twitch and s3bk and mejrs on users.rust-lang.org for suggestions and corrections!\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulkernfeld%2Fcontiguous-data-in-rust","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaulkernfeld%2Fcontiguous-data-in-rust","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulkernfeld%2Fcontiguous-data-in-rust/lists"}