https://github.com/geeknoid/dst-factory
A crate that provides C-like flexible array members in Rust.
https://github.com/geeknoid/dst-factory
dst memory-allocation rust
Last synced: 3 months ago
JSON representation
A crate that provides C-like flexible array members in Rust.
- Host: GitHub
- URL: https://github.com/geeknoid/dst-factory
- Owner: geeknoid
- License: mit
- Created: 2025-05-22T03:47:42.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2026-02-17T21:01:12.000Z (4 months ago)
- Last Synced: 2026-02-17T22:39:07.241Z (4 months ago)
- Topics: dst, memory-allocation, rust
- Language: Rust
- Homepage: https://crates.io/crates/dst-factory
- Size: 137 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# dst-factory
[](https://crates.io/crates/dst-factory)
[](https://docs.rs/dst-factory)
[](https://github.com/geeknoid/dst-factory/actions)
[](https://codecov.io/gh/geeknoid/dst-factory)
[]()
[](./LICENSE)
* [Summary](#summary)
* [Why Should You Care?](#why-should-you-care)
* [Where to Use DSTs?](#where-to-use-dsts)
* [Examples](#examples)
* [Smart Pointers](#smart-pointers)
* [Attribute Features](#attribute-features)
* [Trait Implementations](#trait-implementations)
* [Zero-Initialized Buffers](#zero-initialized-buffers)
* [Serde Support](#serde-support)
* [Other Features](#other-features)
* [Error Conditions](#error-conditions)
* [Acknowledgments](#acknowledgments)
## Summary
Rich support to safely create instances of [Dynamically Sized Types](https://doc.rust-lang.org/reference/dynamically-sized-types.html).
This crate lets you allocate variable data inline at the end of a struct. If you have a
struct that gets allocated on the heap and has some variable-length data associated with it
(like a string or an array), then you can allocate this data directly inline with the struct.
This saves memory by avoiding the need for a pointer and a separate allocation, and saves CPU
cycles by eliminating the need for indirection when accessing the data.
Rust supports the notion of [Dynamically Sized Types](https://doc.rust-lang.org/reference/dynamically-sized-types.html), known as DSTs,
which are types that have a size not known at compile time. DSTs are perfect to implement
flexible array members. But unfortunately, Rust doesn't provide an out-of-the-box way to allocate
instances of such types. This is where this crate comes in.
You can apply the #[[`macro@make_dst_factory`]] attribute to your DST structs, which generates factory
functions that let you easily and safely create instances of your DSTs.
## Why Should You Care?
Dynamically sized types aren't for everyone. You can't use them as local variables
or put them in arrays or vectors, so they can be inconvenient to use. However, their value
lies in situations where you have a lot of heap-allocated objects, as they can substantially
reduce the memory footprint of your application. If you're building graphs, trees, or other
dynamic data structures, you can often leverage DSTs to keep your individual nodes smaller
and more efficient.
## Where to Use DSTs?
It can be hard or tedious to discover where in your codebase there are opportunities to use DSTs.
You can give the following prompt to your favorite AI to have it find candidate structs that
can be upgraded to a DST.
```text
Analyze this Rust codebase for DST (Dynamically Sized Type) optimization opportunities using the dst_factory crate pattern.
A DST struct has one trailing unsized field that is co-located with the struct header in a single allocation (behind Arc, Box, or Rc), eliminating one heap indirection. There are three forms:
- String field → trailing str (eliminates the String's heap buffer)
- Vec field → trailing [T] (eliminates the Vec's heap buffer)
- Box field → trailing dyn Trait (eliminates the Box's heap allocation and pointer indirection)
A struct is a candidate if ALL of these are true:
1. It has at least one String, Vec, or Box field
2. It is used behind Arc, Box, or Rc — NOT stored inline in a Vec, HashMap, array, or by value on the stack
3. The candidate field is not resized, replaced, or swapped after construction. In-place mutation of elements (e.g., writing to [u8] slots, calling &mut self methods on a dyn Trait) is fine — only
operations that change the length or swap the entire value are disqualifying (e.g., push(), pop(), clear(), resize(), reassigning the field to a new String/Vec/Box)
A struct with multiple candidate fields is still a valid candidate — the user picks one field as the trailing unsized tail, the others stay as-is. When listing candidates, note ALL eligible tail fields and
recommend which one to pick (typically the largest or most frequently populated).
For each crate, systematically:
1. Find all struct definitions (both pub and private) that have String, Vec, or Box fields
2. For each, grep for Arc, Box, Vec to determine usage pattern
3. Check whether candidate fields are resized, replaced, or swapped after construction (in-place element mutation is OK)
4. Report: file path, full struct definition, all candidate tail fields, usage pattern (Arc/Box/Vec/value), mutability assessment, and recommended tail choice
Exclude:
- Structs stored in Vec or HashMap values (DSTs are !Sized)
- Structs where the candidate field is resized or replaced after construction (e.g., push(), clear(), field reassignment)
- Structs not behind Arc/Box/Rc (no allocation to optimize)
- Test-only structs
Output format per candidate:
### StructName
File: path/to/file.rs:line
Arc/Box usage: N sites (list key files)
Candidate tail fields:
- field_name: Type → unsized_type (recommended: yes/no, reason)
- field_name: Type → unsized_type
Resized/replaced after construction: no (cite evidence) / yes (disqualifying method)
Volume: how often constructed per request/operation
Savings: 1 allocation per instance × volume
```
## Examples
Here's an example using an array as the last field of a struct:
```rust
use dst_factory::make_dst_factory;
#[make_dst_factory]
struct User {
age: u32,
signing_key: [u8],
}
// allocate one user with a 4-byte key
let a = User::build(33, [0, 1, 2, 3]);
// allocate another user with a 5-byte key
let b = User::build_from_slice(33, &[0, 1, 2, 3, 4]);
// allocate another user, this time using an iterator
let v = vec![0, 1, 2, 3, 4];
let c = User::build(33, v.iter().copied());
// destructure this user and compare its key to the vector
// this has the advantage of iterating over u8, not &u8 or &mut u8.
let (_age, signing_key) = User::destructure(c);
assert!(signing_key.eq(v.into_iter()));
```
Here's another example, this time using a string as the last field of a struct:
```rust
use dst_factory::make_dst_factory;
#[make_dst_factory]
struct User {
age: u32,
name: str,
}
// allocate one user with a 5-character string
let a = User::build(33, "Alice");
// allocate another user with a 3-character string
let b = User::build(33, "Bob");
```
And finally, here's an example using a trait object as the last field of a struct:
```rust
use dst_factory::make_dst_factory;
// a trait we'll use in our DST
trait NumberProducer {
fn get_number(&self) -> u32;
}
// an implementation of the trait we're going to use
struct FortyTwoProducer;
impl NumberProducer for FortyTwoProducer {
fn get_number(&self) -> u32 {
42
}
}
// another implementation of the trait we're going to use
struct TenProducer;
impl NumberProducer for TenProducer {
fn get_number(&self) -> u32 {
10
}
}
#[make_dst_factory]
struct Node {
count: u32,
producer: dyn NumberProducer,
}
// allocate an instance with one implementation of the trait
let a = Node::build(33, FortyTwoProducer{});
assert_eq!(42, a.producer.get_number());
// allocate an instance with another implementation of the trait
let b = Node::build(33, TenProducer{});
assert_eq!(10, b.producer.get_number());
```
Because DSTs don't have a known size at compile time, you can't store them on the stack,
and you can't pass them by value. As a result of these constraints, the factory functions
return smart-pointer-wrapped instances of the structs.
## Smart Pointers
The macro generates factory functions for three smart pointer types:
| Pointer | Factory suffix | Use case |
|---------|---------------|----------|
| [`Box`] | *(none)* | Unique ownership |
| [`Arc`](std::sync::Arc) | `_arc` | Shared ownership, thread-safe (atomic refcount) |
| [`Rc`](std::rc::Rc) | `_rc` | Shared ownership, single-threaded (non-atomic refcount) |
```rust
use dst_factory::make_dst_factory;
use std::sync::Arc;
use std::rc::Rc;
#[make_dst_factory]
struct User {
age: u32,
name: str,
}
// Unique ownership
let boxed: Box = User::build(33, "Alice");
// Thread-safe shared ownership
let shared: Arc = User::build_arc(33, "Bob");
let clone = Arc::clone(&shared);
assert_eq!(&clone.name, "Bob");
// Single-threaded shared ownership
let local: Rc = User::build_rc(33, "Carol");
let clone = Rc::clone(&local);
assert_eq!(&clone.name, "Carol");
// Convert an existing Box into Arc or Rc
let boxed: Box = User::build(33, "Dora");
let shared: Arc = User::into_arc(boxed);
assert_eq!(&shared.name, "Dora");
```
As shown above, in addition to the factory methods, the macro always generates the `into_arc` and `into_rc`
associated functions that convert a `Box` into `Arc` or `Rc` while
preserving the inline DST layout.
## Attribute Features
The common use case for the #[[`macro@make_dst_factory`]] attribute is to not pass any arguments.
This results in functions called `build`, `build_arc`, and `build_rc` when using a string
or dynamic trait as the last field of the struct, and additionally `build_from_slice`,
`build_arc_from_slice`, `build_rc_from_slice`, and `destructure` when using an array as the
last field of the struct.
The generated functions are private by default and have the following approximate signatures:
```rust
// for slices
fn build(field1, field2, ..., last_field: G) -> Box
where
G: IntoIterator,
::IntoIter: ExactSizeIterator,
fn build_from_slice(field1, field2, ..., last_field: &[last_field_type]) -> Box
where
last_field_type: Copy + Sized;
fn build_arc(field1, field2, ..., last_field: G) -> Arc
where
G: IntoIterator,
::IntoIter: ExactSizeIterator,
fn build_arc_from_slice(field1, field2, ..., last_field: &[last_field_type]) -> Arc
where
last_field_type: Copy + Sized;
fn build_rc(field1, field2, ..., last_field: G) -> Rc
where
G: IntoIterator,
::IntoIter: ExactSizeIterator,
fn build_rc_from_slice(field1, field2, ..., last_field: &[last_field_type]) -> Rc
where
last_field_type: Copy + Sized;
fn destructure(this: Box) -> (Type1, Type2, ..., SelfIter);
// when zeroable flag is set (slice tails only, requires bytemuck)
fn build_zeroed(field1, field2, ..., len: usize) -> Box
where
last_field_type: bytemuck::Zeroable;
fn build_arc_zeroed(field1, field2, ..., len: usize) -> Arc
where
last_field_type: bytemuck::Zeroable;
fn build_rc_zeroed(field1, field2, ..., len: usize) -> Rc
where
last_field_type: bytemuck::Zeroable;
// for strings
fn build(field1, field2, ..., last_field: impl AsRef) -> Box;
fn build_arc(field1, field2, ..., last_field: impl AsRef) -> Arc;
fn build_rc(field1, field2, ..., last_field: impl AsRef) -> Rc;
// for trait objects
fn build(field1, field2, ..., last_field: G) -> Box
where
G: TraitName + Sized;
fn build_arc(field1, field2, ..., last_field: G) -> Arc
where
G: TraitName + Sized;
fn build_rc(field1, field2, ..., last_field: G) -> Rc
where
G: TraitName + Sized;
// generated for all DSTs
fn into_arc(this: Box) -> Arc;
fn into_rc(this: Box) -> Rc;
```
The attribute lets you control the name of the generated functions, their
visibility, and whether to generate code for the `no_std` environment, along with
which traits to automatically implement for your type. The general grammar is:
```rust
#[make_dst_factory(
[, destructurer=]
[, iterator=]
[, generic=]
[, ]
[, no_std]
[, deserialize]
[, clone]
[, debug]
[, eq]
[, ord]
[, hash]
[, zeroable]
)]
```
Some examples:
```rust
// Make all generated functions public.
#[make_dst_factory(pub)]
// Custom base name for the generated functions giving `create`, `create_from_slice`, `create_arc`, `create_arc_from_slice`,
// `create_rc`, and `create_rc_from_slice`.
#[make_dst_factory(create)]
// Custom destructurer name.
#[make_dst_factory(create, destructurer = destroy)]
// Public functions with custom name.
#[make_dst_factory(create, pub)]
// Support the `no_std` environment.
#[make_dst_factory(create, no_std)]
// Custom generic type name.
#[make_dst_factory(create, no_std, generic=X)]
```
### Trait Implementations
Rust's standard `#[derive(...)]` often doesn't work for DST structs. The #[[`macro@make_dst_factory`]] attribute provides
flags to generate these trait implementations:
| Flag | Trait(s) generated | Notes |
|------|-------------------|-------|
| `clone` | `Clone` for `Box` | Deep copy via factory function |
| `debug` | `Debug` for the struct | Named fields or tuple formatting |
| `eq` | `PartialEq` and `Eq` for the struct | Compares all fields |
| `ord` | `PartialOrd` and `Ord` for the struct | Compares all fields lexicographically |
| `hash` | `Hash` for the struct | Hashes all fields |
When the last field of the struct is a `dyn Trait`, the generated `where` clauses require the trait object to
implement the relevant trait (e.g. `dyn MyTrait: Debug`). The `clone` flag is the
exception; it is not supported for `dyn Trait` since there is no way to clone
a concrete type through a trait object reference.
```rust
use dst_factory::make_dst_factory;
#[make_dst_factory(clone, debug, eq, ord, hash)]
struct Message {
id: u32,
text: str,
}
let msg = Message::build(1, "hello");
let cloned = msg.clone();
assert_eq!(msg, cloned);
assert_eq!(format!("{:?}", &*msg), "Message { id: 1, text: \"hello\" }");
```
### Zero-Initialized Buffers
When the last struct field is a `[T]`, the `zeroable` flag generates `build_zeroed`, `build_arc_zeroed`,
and `build_rc_zeroed` factories that allocate the DST with a zero-initialized slice of a
given length.
```rust
use dst_factory::make_dst_factory;
#[make_dst_factory(zeroable)]
struct Buffer {
cursor: usize,
data: [u8],
}
// Allocate a 1MB buffer with zero-initialized payload — no per-element initialization.
let buf = Buffer::build_zeroed(0, 1_000_000);
assert_eq!(buf.data.len(), 1_000_000);
assert!(buf.data.iter().all(|&b| b == 0));
```
### Serde Support
DST structs work naturally with serde's `#[derive(Serialize)]`, since serialization
only requires a reference. However, `#[derive(Deserialize)]` does not work with DSTs,
so special support is needed instead.
Passing the `deserialize` flag in the attribute generates a
[`Deserialize`](https://docs.rs/serde/latest/serde/trait.Deserialize.html) implementation
for `Box`.
```rust
use dst_factory::make_dst_factory;
use serde::Serialize;
#[derive(Serialize)]
#[make_dst_factory(deserialize)]
struct Message {
id: u32,
text: str,
}
// Serialize
let msg = Message::build(1, "hello");
let json = serde_json::to_string(&*msg).unwrap();
// Deserialize
let restored: Box = serde_json::from_str(&json).unwrap();
assert_eq!(restored.id, 1);
assert_eq!(&restored.text, "hello");
```
Rust's orphan rules prevent implementing `Deserialize` for `Arc` or `Rc`.
Instead, the `deserialize` flag generates helper functions `deserialize_arc` and
`deserialize_rc` that can be used with serde's `#[serde(deserialize_with = "...")]`
attribute:
```rust
use dst_factory::make_dst_factory;
use serde::{Serialize, Deserialize};
use std::sync::Arc;
use std::rc::Rc;
#[derive(Serialize)]
#[make_dst_factory(deserialize)]
struct Message {
id: u32,
text: str,
}
#[derive(Serialize, Deserialize)]
struct Dashboard {
#[serde(deserialize_with = "Message::deserialize_arc")]
shared_msg: Arc,
#[serde(deserialize_with = "Message::deserialize_rc")]
local_msg: Rc,
}
```
Deserialization is not supported when the last field of the struct is a `dyn Trait`
since there is no way to reconstruct the concrete type from serialized data.
## Other Features
You can use the #[[`macro@make_dst_factory`]] attribute on structs with the normal Rust
representation or C representation (`#[repr(C)]`), with any padding and alignment
specification. See the Rust reference on [Type Layout](https://doc.rust-lang.org/reference/type-layout.html)
for more details.
## Error Conditions
The #[[`macro@make_dst_factory`]] attribute produces a compile-time error if:
- It's applied to anything other than a regular struct or a tuple struct.
- Its arguments are malformed (e.g., incorrect visibility keyword, too many arguments, etc.).
- The struct has no fields.
- The last field of the struct is not a slice (`[T]`), a string (`str`), or a trait object (`dyn Trait`).
- The resulting struct exceeds the maximum size allowed of `isize::MAX`.
- The `deserialize` or `clone` flags are used on a struct whose last field is a trait object.
- The `zeroable` flag is used on a struct whose last field is not a slice (`[T]`).
## Acknowledgments
Many thanks to for his invaluable help getting the factory methods
in top shape.