{"id":17644431,"url":"https://github.com/vigna/epserde-rs","last_synced_at":"2025-05-16T01:04:11.707Z","repository":{"id":176508891,"uuid":"657492192","full_name":"vigna/epserde-rs","owner":"vigna","description":"An ε-copy serialization/deserialization framework for Rust","archived":false,"fork":false,"pushed_at":"2025-03-05T20:41:02.000Z","size":618,"stargazers_count":144,"open_issues_count":4,"forks_count":8,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-04-08T12:12:31.050Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/vigna.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-06-23T07:20:01.000Z","updated_at":"2025-03-28T17:52:50.000Z","dependencies_parsed_at":null,"dependency_job_id":"8f1bc910-e040-4e1e-924f-164d8131415f","html_url":"https://github.com/vigna/epserde-rs","commit_stats":{"total_commits":385,"total_committers":5,"mean_commits":77.0,"dds":0.2623376623376623,"last_synced_commit":"f539366b63cae169f99c304bc55d829e78b558aa"},"previous_names":["vigna/epserde-rs"],"tags_count":16,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fepserde-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fepserde-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fepserde-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/vigna%2Fepserde-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/vigna","download_url":"https://codeload.github.com/vigna/epserde-rs/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254448579,"owners_count":22072764,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-23T10:05:25.754Z","updated_at":"2025-05-16T01:04:11.674Z","avatar_url":"https://github.com/vigna.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ε-serde\n\n[![downloads](https://img.shields.io/crates/d/epserde)](https://crates.io/crates/epserde)\n[![dependents](https://img.shields.io/librariesio/dependents/cargo/epserde)](https://crates.io/crates/epserde/reverse_dependencies)\n![GitHub CI](https://github.com/vigna/epserde-rs/actions/workflows/rust.yml/badge.svg)\n![license](https://img.shields.io/crates/l/epserde)\n[![](https://tokei.rs/b1/github/vigna/epserde-rs?type=Rust,Python)](https://github.com/vigna/epserde-rs)\n[![Latest version](https://img.shields.io/crates/v/epserde.svg)](https://crates.io/crates/epserde)\n[![Documentation](https://docs.rs/epserde/badge.svg)](https://docs.rs/epserde)\n\nε-serde is a Rust framework for *ε*-copy *ser*ialization and *de*serialization.\n\n## Why\n\nLarge immutable data structures need time to be deserialized using the [serde]\napproach. A possible solution for this problem is given by frameworks such as\n[Abomonation], [rkiv], and [zerovec], which provide *zero-copy* deserialization:\nthe stream of bytes serializing the data structure can be used directly as a\nRust structure. In particular, this approach makes it possible to map into\nmemory an on-disk data structure, making it available instantly. It also makes\nit possible to load the data structure in a memory region with particular\nattributes, such as transparent huge pages on Linux. Even when using standard\nmemory load and deserialization happen much faster as the entire structure can\nbe loaded with a single read operation.\n\nε-serde has the same goals as the zero-copy frameworks above but provides\ndifferent tradeoffs.\n\n## How\n\nSince in these data structures typically most of the data is given by large\nchunks of memory in the form of slices or vectors, at deserialization time one\ncan build quickly a proper Rust structure whose referenced memory, however, is\nnot copied. We call this approach *ε-copy deserialization*, as typically a\nminuscule fraction of the serialized data is copied to build the structure. The\nresult is similar to that of the frameworks above, but the performance of the\ndeserialized structure will be identical to that of a standard, in-memory Rust\nstructure, as references are resolved at deserialization time.\n\nWe provide procedural macros implementing serialization and deserialization\nmethods, basic (de)serialization for primitive types, vectors, etc., convenience\nmemory-mapping methods based on [mmap_rs], and a [`MemCase`] structure that\ncouples a deserialized structure with its backend (e.g., a slice of memory or a\nmemory-mapped region).\n\n## Who\n\nTommaso Fontana, while working at INRIA under the supervision of Stefano\nZacchiroli, came up with the basic idea for ε-serde, that is, replacing\nstructures with equivalent references. The code was developed jointly with\nSebastiano Vigna, who came up with the [`MemCase`] and the\n[`ZeroCopy`]/[`DeepCopy`] logic.\n\n## Cons\n\nThese are the main limitations you should be aware of before choosing to use\nε-serde:\n\n- Your types cannot contain references. For example, you cannot use ε-serde on a\n  tree.\n\n- While we provide procedural macros that implement serialization and\n  deserialization, they require that your type is written and used in a specific\n  way; in particular, the fields you want to ε-copy must be type parameters\n  implementing [`DeserializeInner`], to which a [deserialized type] is\n  associated. For example, we provide implementations for `Vec\u003cT\u003e`/`Box\u003c[T]\u003e`,\n  where `T` is zero-copy, or `String`/`Box\u003cstr\u003e`, which have associated\n  deserialized type `\u0026[T]` or `\u0026str`, respectively. Vectors and boxed slices of\n  types that are not zero-copy will be deserialized recursively in memory\n  instead.\n\n- After deserialization of a type `T`, you will obtain an associated\n  deserialized type [`DeserType\u003c'_,T\u003e`], which will usually reference the\n  underlying serialized support (e.g., a memory-mapped region); hence the need for\n  a lifetime. If you need to store the deserialized structure in a field of a new\n  structure you will need to couple permanently the deserialized structure with\n  its serialized support, which is obtained by putting it in a [`MemCase`] using\n  the convenience methods [`Deserialize::load_mem`], [`Deserialize::load_mmap`],\n  and [`Deserialize::mmap`]. A [`MemCase`] will deref to its contained type, so it\n  can be used transparently as long as fields and methods are concerned, but if\n  your original type is `T` the field of the new structure will have to be of type\n  `MemCase\u003cDeserType\u003c'static, T\u003e\u003e`, not `T`.\n\n## Pros\n\n- Almost instant deserialization with minimal allocation provided that you\n  designed your type following the ε-serde guidelines or that you use standard\n  types.\n\n- The structure you get by deserialization is the same structure you serialized,\n  except that type parameters will be replaced by their associated deserialization\n  type (e.g., vectors will become references to slices). This is not the\n  case with [rkiv], which requires you to reimplement all methods on the\n  deserialized type.\n\n- The structure you get by deserialization has exactly the same performance as\n  the structure you serialized. This is not the case with [zerovec] or [rkiv].\n\n- You can serialize structures containing references to slices, or even\n  exact-size iterators, and they will be deserialized as if you had written a\n  vector. It is thus possible to serialize structures larger than the available\n  memory.\n\n- You can deserialize from read-only supports, as all dynamic information\n  generated at deserialization time is stored in newly allocated memory. This is\n  not the case with [Abomonation].\n\n## Example: Zero copy of standard types\n\nLet us start with the simplest case: data that can be zero-copy deserialized. In\nthis case, we serialize an array of a thousand zeros, and get back a reference\nto such an array:\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\n\nlet s = [0_usize; 1000];\n\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized0\");\ns.serialize(\u0026mut std::fs::File::create(\u0026file)?)?;\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// The type of t will be inferred--it is shown here only for clarity\nlet t: \u0026[usize; 1000] =\n    \u003c[usize; 1000]\u003e::deserialize_eps(b.as_ref())?;\n\nassert_eq!(s, *t);\n\n// You can derive the deserialization type, with a lifetime depending on b\nlet t: DeserType\u003c'_, [usize; 1000]\u003e =\n    \u003c[usize; 1000]\u003e::deserialize_eps(b.as_ref())?;\n\nassert_eq!(s, *t);\n\n// This is a traditional deserialization instead\nlet t: [usize; 1000] = \n    \u003c[usize; 1000]\u003e::deserialize_full(\n        \u0026mut std::fs::File::open(\u0026file)?\n    )?;\nassert_eq!(s, t);\n\n// In this case we map the data structure into memory\n//\n// Note: requires the `mmap` feature.\nlet u: MemCase\u003c\u0026[usize; 1000]\u003e = \n    \u003c[usize; 1000]\u003e::mmap(\u0026file, Flags::empty())?;\n\nassert_eq!(s, **u);\n\n// When using a MemCase, the lifetime of the derived deserialization type is 'static\nlet u: MemCase\u003cDeserType\u003c'static, [usize; 1000]\u003e\u003e = \n    \u003c[usize; 1000]\u003e::mmap(\u0026file, Flags::empty())?;\n\nassert_eq!(s, **u);\n#     Ok(())\n# }\n```\n\nNote how we serialize an array, but we deserialize a reference. The reference\npoints inside `b`, so there is no copy performed. The call to\n[`deserialize_full`] creates a new array instead. The third call maps the data\nstructure into memory and returns a [`MemCase`] that can be used transparently\nas a reference to the array; moreover, the [`MemCase`] can be passed to other\nfunctions or stored in a structure field, as it contains both the structure and\nthe memory-mapped region that supports it.\n\nThe type alias [`DeserType`] can be used to derive the deserialized type\nassociated with a type. It contains a lifetime, which is the lifetime of the\nmemory region containing the serialized data. When deserializing into a\n[`MemCase`], however, the lifetime is `'static`, as [`MemCase`] is an owned\ntype.\n\n## Examples: ε-copy of standard structures\n\nZero-copy deserialization is not that interesting because it can be applied only\nto data whose memory layout and size are fixed and known at compile time. This\ntime, let us serialize a `Vec` containing a thousand zeros: ε-serde will\ndeserialize its associated deserialization type, which is a reference to a\nslice.\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\n\nlet s = vec![0; 1000];\n\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized1\");\ns.serialize(\u0026mut std::fs::File::create(\u0026file)?)?;\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// The type of t will be inferred--it is shown here only for clarity\nlet t: DeserType\u003c'_, Vec\u003cusize\u003e\u003e =\n    \u003cVec\u003cusize\u003e\u003e::deserialize_eps(b.as_ref())?;\n\nassert_eq!(s, *t);\n\n// This is a traditional deserialization instead\nlet t: Vec\u003cusize\u003e = \n    \u003cVec\u003cusize\u003e\u003e::load_full(\u0026file)?;\nassert_eq!(s, t);\n\n// In this case we map the data structure into memory\nlet u: MemCase\u003cDeserType\u003c'static, Vec\u003cusize\u003e\u003e\u003e = \n    \u003cVec\u003cusize\u003e\u003e::mmap(\u0026file, Flags::empty())?;\nassert_eq!(s, **u);\n#     Ok(())\n# }\n```\n\nNote how we serialize a vector, but we deserialize a reference to a slice; the\nsame would happen when serializing a boxed slice. The reference points inside\n`b`, so there is very little copy performed (in fact, just a field containing\nthe length of the slice). All this is because `usize` is a zero-copy type. Note\nalso that we use the convenience method [`Deserialize::load_full`].\n\nIf your code must work both with the original and the deserialized version,\nhowever, it must be written for a trait that is implemented by both types, such\nas `AsRef\u003c[usize]\u003e`.\n\n## Example: Zero-copy structures\n\nYou can define your types to be zero-copy, in which case they will work like\n`usize` in the previous examples. This requires the structure to be made of\nzero-copy fields, and to be annotated with `#[zero_copy]` and `#[repr(C)]`\n(which means that you will lose the possibility that the compiler reorders the\nfields to optimize memory usage):\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\n\n#[derive(Epserde, Debug, PartialEq, Copy, Clone)]\n#[repr(C)]\n#[zero_copy]\nstruct Data {\n    foo: usize,\n    bar: usize,\n}\n\nlet s = vec![Data { foo: 0, bar: 0 }; 1000];\n\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized2\");\ns.serialize(\u0026mut std::fs::File::create(\u0026file)?)?;\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// The type of t will be inferred--it is shown here only for clarity\nlet t: DeserType\u003c'_, Vec\u003cData\u003e\u003e =\n    \u003cVec\u003cData\u003e\u003e::deserialize_eps(b.as_ref())?;\n\nassert_eq!(s, *t);\n\n// This is a traditional deserialization instead\nlet t: Vec\u003cData\u003e = \n    \u003cVec\u003cData\u003e\u003e::load_full(\u0026file)?;\nassert_eq!(s, t);\n\n// In this case we map the data structure into memory\nlet u: MemCase\u003cDeserType\u003c'static, Vec\u003cData\u003e\u003e\u003e = \n    \u003cVec\u003cData\u003e\u003e::mmap(\u0026file, Flags::empty())?;\nassert_eq!(s, **u);\n#     Ok(())\n# }\n```\n\nIf a structure is not zero-copy, vectors of structures will be always\ndeserialized into vectors.\n\n## Example: Structures with parameters\n\nMore flexibility can be obtained by defining structures with fields whose types\nare defined by parameters. In this case, ε-serde will deserialize the structure\nreplacing its type parameters with the associated deserialized type.\n\nLet us design a structure that will contain an integer, which will be copied,\nand a vector of integers that we want to ε-copy:\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\n\n#[derive(Epserde, Debug, PartialEq)]\nstruct MyStruct\u003cA\u003e {\n    id: isize,\n    data: A,\n}\n\n// Create a structure where A is a Vec\u003cisize\u003e\nlet s: MyStruct\u003cVec\u003cisize\u003e\u003e = MyStruct { id: 0, data: vec![0, 1, 2, 3] };\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized3\");\ns.store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// The type of t will be inferred--it is shown here only for clarity\nlet t: MyStruct\u003c\u0026[isize]\u003e = \n    \u003cMyStruct\u003cVec\u003cisize\u003e\u003e\u003e::deserialize_eps(b.as_ref())?;\n\nassert_eq!(s.id, t.id);\nassert_eq!(s.data, Vec::from(t.data));\n\n// This is a traditional deserialization instead\nlet t: MyStruct\u003cVec\u003cisize\u003e\u003e = \n    \u003cMyStruct\u003cVec\u003cisize\u003e\u003e\u003e::load_full(\u0026file)?;\nassert_eq!(s, t);\n\n// In this case we map the data structure into memory\nlet u: MemCase\u003cMyStruct\u003c\u0026[isize]\u003e\u003e = \n    \u003cMyStruct\u003cVec\u003cisize\u003e\u003e\u003e::mmap(\u0026file, Flags::empty())?;\nassert_eq!(s.id, u.id);\nassert_eq!(s.data, u.data.as_ref());\n#     Ok(())\n# }\n```\n\nNote how the field originally containing a `Vec\u003cisize\u003e` now contains a\n`\u0026[isize]` (this replacement is generated automatically). The reference points\ninside `b`, so there is no need to copy the field. Nonetheless, deserialization\ncreates a new structure `MyStruct`, ε-copying the original data. The second call\ncreates a full copy instead. We can write methods for our structure that will\nwork for the ε-copied version: we just have to take care that they are defined\nin a way that will work both on the original type parameter and on its\nassociated deserialized type; we can also use `type` to reduce the clutter:\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\n\n#[derive(Epserde, Debug, PartialEq)]\nstruct MyStructParam\u003cA\u003e {\n    id: isize,\n    data: A,\n}\n\n/// This method can be called on both the original and the ε-copied structure\nimpl\u003cA: AsRef\u003c[isize]\u003e\u003e MyStructParam\u003cA\u003e {\n    fn sum(\u0026self) -\u003e isize {\n        self.data.as_ref().iter().sum()\n    }\n}\n\ntype MyStruct = MyStructParam\u003cVec\u003cisize\u003e\u003e;\n\n// Create a structure where A is a Vec\u003cisize\u003e\nlet s = MyStruct { id: 0, data: vec![0, 1, 2, 3] };\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized4\");\ns.store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\nlet t = MyStruct::deserialize_eps(b.as_ref())?;\n// We can call the method on both structures\nassert_eq!(s.sum(), t.sum());\n\nlet t = \u003cMyStruct\u003e::mmap(\u0026file, Flags::empty())?;\n\n// t works transparently as a MyStructParam\u003c\u0026[isize]\u003e\nassert_eq!(s.id, t.id);\nassert_eq!(s.data, t.data.as_ref());\nassert_eq!(s.sum(), t.sum());\n#     Ok(())\n# }\n```\n\n## Example: Deep-copy structures with internal parameters\n\nInternal parameters, that is, parameters used by the types of your fields but\nthat do not represent the type of your fields are left untouched. However, to be\nserializable they must be classified as deep-copy or zero-copy, and in the first\ncase they must have a `'static` lifetime. The only exception to this rule is\nfor types inside a [`PhantomData`], which do not even need to be serializable.\nFor example,\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\n\n#[derive(Epserde, Debug, PartialEq)]\nstruct MyStruct\u003cA: DeepCopy + 'static\u003e(Vec\u003cA\u003e);\n\n// Create a structure where A is a Vec\u003cisize\u003e\nlet s: MyStruct\u003cVec\u003cisize\u003e\u003e = MyStruct(vec![vec![0, 1, 2, 3]]);\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized5\");\ns.store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// The type of t is unchanged\nlet t: MyStruct\u003cVec\u003cisize\u003e\u003e = \n    \u003cMyStruct\u003cVec\u003cisize\u003e\u003e\u003e::deserialize_eps(b.as_ref())?;\n#     Ok(())\n# }\n```\n\nNote how the field originally of type `Vec\u003cVec\u003cisize\u003e\u003e` remains of the same\ntype.\n\n## Example: Zero-copy structures with parameters\n\nFor zero-copy structures, things are slightly different because types are not\nsubstituted, even if they represent the type of your fields. So all parameters\nmust be zero-copy. This must hold even for types inside a [`PhantomData`].\nFor example,\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\n\n#[derive(Epserde, Debug, PartialEq, Clone, Copy)]\n#[repr(C)]\n#[zero_copy]\nstruct MyStruct\u003cA: ZeroCopy\u003e {\n    data: A,\n}\n\n// Create a structure where A is a Vec\u003cisize\u003e\nlet s: MyStruct\u003ci32\u003e = MyStruct { data: 0 };\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized6\");\ns.store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// The type of t is unchanged\nlet t: \u0026MyStruct\u003ci32\u003e = \n    \u003cMyStruct\u003ci32\u003e\u003e::deserialize_eps(b.as_ref())?;\n#     Ok(())\n# }\n```\n\nNote how the field originally of type `i32` remains of the same type.\n\n## Example: Enums\n\nEnums are supported, but there are two caveats: first, if you want them to be\nzero-copy, they must be `repr(C)`, and thus you will lose the possibility that\nthe compiler optimize their memory representation; second, if you have type\nparameters that are not used by all variants you must be careful to specify\nalways the same type parameter when serializing and deserializing. This is\nobvious for non-enum types, but with enum types with default type parameters it\ncan become tricky. For example,\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\n\n#[derive(Epserde, Debug, PartialEq, Clone, Copy)]\nenum Enum\u003cT=Vec\u003cusize\u003e\u003e {\n    A,\n    B(T),\n}\n\n// This enum has T=Vec\u003ci32\u003e by type inference\nlet e = Enum::B(vec![0, 1, 2, 3]);\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized7\");\ne.store(\u0026file);\n// Deserializing using just Enum will fail, as the type parameter \n// by default is Vec\u003cusize\u003e\nassert!(\u003cEnum\u003e::load_full(\u0026file).is_err());\n#     Ok(())\n# }\n```\n\n## Example: (Structures containing references to) slices\n\nFor convenience, ε-serde can serialize references to slices, and will\ndeserialize them as if they were vectors. You must however be careful to\ndeserialize with the correct type. For example,\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\n\nlet v = vec![0, 1, 2, 3];\n// This is a slice\nlet s: \u0026[i32] = v.as_ref();\n// Serialize it\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized8\");\ns.store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// We must deserialize as a vector, even if we are getting back a reference\nlet t: \u0026[i32] = \u003cVec\u003ci32\u003e\u003e::deserialize_eps(b.as_ref())?;\nlet t: Vec\u003ci32\u003e = \u003cVec\u003ci32\u003e\u003e::deserialize_full(\n        \u0026mut std::fs::File::open(\u0026file)?\n    )?;\nlet t: MemCase\u003c\u0026[i32]\u003e = \u003cVec\u003ci32\u003e\u003e::mmap(\u0026file, Flags::empty())?;\n\n// Within a structure\n#[derive(Epserde, Debug, PartialEq, Eq, Default, Clone)]\nstruct Data\u003cA\u003e {\n    s: A,\n}\n\nlet d = Data { s };\n// Serialize it\nd.store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// We must deserialize the field as a vector, even if we are getting back a reference\nlet t: Data\u003c\u0026[i32]\u003e = \u003cData\u003cVec\u003ci32\u003e\u003e\u003e::deserialize_eps(b.as_ref())?;\nlet t: Data\u003cVec\u003ci32\u003e\u003e = \u003cData\u003cVec\u003ci32\u003e\u003e\u003e::deserialize_full(\n        \u0026mut std::fs::File::open(\u0026file)?\n    )?;\nlet t: MemCase\u003cData\u003c\u0026[i32]\u003e\u003e = \u003cData\u003cVec\u003ci32\u003e\u003e\u003e::mmap(\u0026file, Flags::empty())?;\n#     Ok(())\n# }\n```\n\n## Example: (Structures containing) iterators\n\nε-serde can serialize iterators returning references to a type. The resulting\nfield can be deserialized as a vector. In this case we need to wrap the iterator\nin a [`SerIter`], as ε-serde cannot implement the serialization traits directly\non [`Iterator`]. For example,\n\n```rust\n# fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\nuse epserde::prelude::*;\nuse epserde_derive::*;\nuse std::slice::Iter;\n\nlet v = vec![0, 1, 2, 3];\n// This is an iterator\nlet i: Iter\u003c'_, i32\u003e = v.iter();\n// Serialize it by wrapping it in a SerIter\nlet mut file = std::env::temp_dir();\nfile.push(\"serialized9\");\nSerIter::from(i).store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// We must deserialize as a vector, even if we are getting back a reference\nlet t: \u0026[i32] = \u003cVec\u003ci32\u003e\u003e::deserialize_eps(b.as_ref())?;\nlet t: Vec\u003ci32\u003e = \u003cVec\u003ci32\u003e\u003e::deserialize_full(\n        \u0026mut std::fs::File::open(\u0026file)?\n    )?;\nlet t: MemCase\u003c\u0026[i32]\u003e = \u003cVec\u003ci32\u003e\u003e::mmap(\u0026file, Flags::empty())?;\n\n// Within a structure\n#[derive(Epserde, Debug, PartialEq, Eq, Default, Clone)]\nstruct Data\u003cA\u003e {\n    s: A,\n}\n\nlet d = Data { s: SerIter::from(v.iter()) };\n// Serialize it\nd.store(\u0026file);\n// Load the serialized form in a buffer\nlet b = std::fs::read(\u0026file)?;\n\n// We must deserialize the field as a vector, even if we are getting back a reference\nlet t: Data\u003c\u0026[i32]\u003e = \u003cData\u003cVec\u003ci32\u003e\u003e\u003e::deserialize_eps(b.as_ref())?;\nlet t: Data\u003cVec\u003ci32\u003e\u003e = \u003cData\u003cVec\u003ci32\u003e\u003e\u003e::deserialize_full(\n        \u0026mut std::fs::File::open(\u0026file)?\n    )?;\nlet t: MemCase\u003cData\u003c\u0026[i32]\u003e\u003e = \u003cData\u003cVec\u003ci32\u003e\u003e\u003e::mmap(\u0026file, Flags::empty())?;\n#     Ok(())\n# }\n```\n\n\n## Example: `sux-rs`\n\nThe [`sux-rs`] crate provides several data structures that use ε-serde.\n\n## Design\n\nEvery type serializable with ε-serde has three features that are in principle\northogonal, but that in practice often condition one another:\n\n- the type has an *associated serialization type*, which is the type we\n  write when serializing;\n- the type has an *associated deserialization type*, which is the type you\n  obtain upon deserialization;\n- the type can be either [`ZeroCopy`] or [`DeepCopy`]; it can also be neither.\n\nThere is no constraint on the associated (de)serialization type: it can be\nliterally anything. In general, however, one tries to have a deserialization\ntype that is somewhat compatible with the original type, in the sense that they\nboth satisfy a trait for which implementations can be written: for example,\nε-serde deserializes vectors as references to slices, so implementations can be\nwritten for references to slices and will work both on the original and the\ndeserialized type. And, in general, [`ZeroCopy`] types deserialize into\nthemselves. Presently the associated serialization type is almost always `Self`,\nwith the notable exception of references to slices, which are serialized for\nconvenience as vectors.\n\nBeing [`ZeroCopy`] or [`DeepCopy`] decides instead how the type will be treated\nwhen serializing and deserializing sequences, such as arrays, slices, boxed\nslices, and vectors. Sequences of zero-copy types are deserialized using a\nreference, whereas sequences of deep-copy types are recursively deserialized in\nallocated memory (to sequences of the associated deserialization types). It is\nimportant to remark that *you cannot serialize a sequence whose elements are of\na type that is neither* [`ZeroCopy`] *nor* [`DeepCopy`] (see the [`CopyType`]\ndocumentation for a deeper explanation).\n\nLogically, zero-copy types should be deserialized to references, and this indeed\nhappens in most cases, and certainly in the derived code: however, *primitive\ntypes are always fully deserialized*. There are two reasons behind this\nnon-orthogonal choice:\n\n- primitive types occupy so little space that deserializing them as a reference\n  is not efficient;\n- if a type parameter `T` is a primitive type, writing generic code for\n  `AsRef\u003cT\u003e` is really not nice;\n- deserializing primitive types to a reference would require further padding to\n  align them.\n\nSince this is true only of primitive types, when deserializing a 1-tuple\ncontaining a primitive type one obtains a reference (and indeed this workaround\ncan be used if you really need to deserialize a primitive type as a reference).\nThe same happens if you deserialize a zero-copy struct containing a single field\nof primitive type.\n\nDeep-copy types instead are serialized and deserialized recursively, field by\nfield. The basic idea in ε-serde is that *if a field has a type that is a\nparameter, during ε-copy deserialization the type will be replaced with its\ndeserialization type*. Since the deserialization type is defined recursively,\nreplacement can happen at any depth level. For example, a field of type `A =\nVec\u003cVec\u003cVec\u003cusize\u003e\u003e\u003e` will be deserialized as a `A = Vec\u003cVec\u003c\u0026[usize]\u003e\u003e`.\n\nThis approach makes it possible to write ε-serde-aware structures that hide from\nthe user the substitution. A good example is the `BitFieldVec` structure from\n[`sux-rs`], which exposes an array of fields of fixed bit width using (usually)\na `Vec\u003cusize\u003e` as a backend; except for extension methods, all methods of\n`BitFieldVec` come from the trait `BitFieldSlice`. If you have your own struct\nand one of the fields is of type `A`, when serializing your struct with `A`\nequal to `BitFieldVec\u003cVec\u003cusize\u003e\u003e`, upon ε-copy deserialization you will get a\nversion of your struct with `BitFieldVec\u003c\u0026[usize]\u003e`. All this will happen under\nthe hood because `BitFieldVec` is ε-serde-aware, and in fact you will not even\nnotice the difference if you access both versions using the trait\n`BitFieldSlice`.\n\n## MemDbg / MemSize\n\nAll ε-serde structures implement the [`MemDbg`] and [`MemSize`] traits.\n\n## Derived and hand-made implementations\n\nWe strongly suggest using the procedural macro [`Epserde`] to make your own\ntypes serializable and deserializable. Just invoking the macro on your structure\nwill make it fully functional with ε-serde. The attribute `#[zero_copy]` can be\nused to make a structure zero-copy, albeit it must satisfy [a few\nprerequisites].\n\nYou can also implement manually the traits [`CopyType`], [`MaxSizeOf`],\n[`TypeHash`], [`ReprHash`], [`SerializeInner`], and [`DeserializeInner`], but\nthe process is error-prone, and you must be fully aware of ε-serde's\nconventions. The procedural macro [`TypeInfo`] can be used to generate\nautomatically at least [`MaxSizeOf`], [`TypeHash`], and [`ReprHash`]\nautomatically.\n\n## Acknowledgments\n\nThis software has been partially supported by project SERICS (PE00000014) under\nthe NRRP MUR program funded by the EU - NGEU, and by project ANR COREGRAPHIE,\ngrant ANR-20-CE23-0002 of the French Agence Nationale de la Recherche.\nViews and opinions expressed are however those of the authors only and do not\nnecessarily reflect those of the European Union or the Italian MUR. Neither the\nEuropean Union nor the Italian MUR can be held responsible for them.\n\n[`MemCase`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/mem_case/struct.MemCase.html\u003e\n[`ZeroCopy`]: \u003chttps://docs.rs/epserde/latest/epserde/traits/copy_type/trait.ZeroCopy.html\u003e\n[`DeepCopy`]: \u003chttps://docs.rs/epserde/latest/epserde/traits/copy_type/trait.DeepCopy.html\u003e\n[`CopyType`]: \u003chttps://docs.rs/epserde/latest/epserde/traits/copy_type/trait.CopyType.html\u003e\n[`MaxSizeOf`]: \u003chttps://docs.rs/epserde/latest/epserde/traits/type_info/trait.MaxSizeOf.html\u003e\n[`TypeHash`]: \u003chttps://docs.rs/epserde/latest/epserde/traits/type_info/trait.TypeHash.html\u003e\n[`ReprHash`]: \u003chttps://docs.rs/epserde/latest/epserde/traits/type_info/trait.ReprHash.html\u003e\n[`DeserializeInner`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/trait.DeserializeInner.html\u003e\n[`SerializeInner`]: \u003chttps://docs.rs/epserde/latest/epserde/ser/trait.SerializeInner.html\u003e\n[`TypeInfo`]: \u003chttps://docs.rs/epserde/latest/epserde/derive.TypeInfo.html\u003e\n[`Epserde`]: \u003chttps://docs.rs/epserde/latest/epserde_derive/derive.Epserde.html\u003e\n[`Deserialize::load_full`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/trait.Deserialize.html#method.load_full\u003e\n[`deserialize_full`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/trait.Deserialize.html#tymethod.deserialize_full\u003e\n[`DeserType`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/type.DeserType.html\u003e\n[`Deserialize::load_mem`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/trait.Deserialize.html#method.load_mem\u003e\n[`Deserialize::load_mmap`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/trait.Deserialize.html#method.load_mmap\u003e\n[`Deserialize::mmap`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/trait.Deserialize.html#method.mmap\u003e\n[a few prerequisites]: \u003chttps://docs.rs/epserde/latest/epserde/traits/copy_type/trait.CopyType.html\u003e\n[deserialized type]: \u003chttps://docs.rs/epserde/latest/epserde/deser/trait.DeserializeInner.html#associatedtype.DeserType\u003e\n[`DeserType\u003c'_,T\u003e`]: \u003chttps://docs.rs/epserde/latest/epserde/deser/type.DeserType.html\u003e\n[`sux-rs`]: \u003chttp://crates.io/sux/\u003e\n[serde]: \u003chttps://serde.rs/\u003e\n[Abomonation]: \u003chttps://crates.io/crates/abomonation\u003e\n[rkiv]: \u003chttps://crates.io/crates/rkyv/\u003e\n[zerovec]: \u003chttps://crates.io/crates/zerovec\u003e\n[mmap_rs]: \u003chttps://crates.io/crates/mmap-rs\u003e\n[`MemDbg`]: https://docs.rs/mem_dbg/latest/mem_dbg/trait.MemDbg.html\n[`MemSize`]: https://docs.rs/mem_dbg/latest/mem_dbg/trait.MemSize.html\n[`PhantomData`]: \u003chttps://doc.rust-lang.org/std/marker/struct.PhantomData.html\u003e\n[`Iterator`]: \u003chttps://doc.rust-lang.org/std/iter/trait.Iterator.html\u003e\n[`SerIter`]: \u003chttps://docs.rs/epserde/latest/epserde/impls/iter/struct.SerIter.html\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvigna%2Fepserde-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvigna%2Fepserde-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvigna%2Fepserde-rs/lists"}