{"id":19333458,"url":"https://github.com/zakarumych/alkahest","last_synced_at":"2025-04-05T10:10:10.687Z","repository":{"id":57483426,"uuid":"387237106","full_name":"zakarumych/alkahest","owner":"zakarumych","description":"Fantastic serialization library","archived":false,"fork":false,"pushed_at":"2024-05-16T20:05:51.000Z","size":481,"stargazers_count":159,"open_issues_count":8,"forks_count":9,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-03-29T09:12:25.658Z","etag":null,"topics":["network-programming","protocols","rust","serialization"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zakarumych.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"COPYING","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-07-18T17:53:21.000Z","updated_at":"2025-01-05T22:25:16.000Z","dependencies_parsed_at":"2023-11-08T10:41:52.545Z","dependency_job_id":"aeecae60-b5a5-4d9c-95c3-a313fba2c999","html_url":"https://github.com/zakarumych/alkahest","commit_stats":{"total_commits":115,"total_committers":3,"mean_commits":"38.333333333333336","dds":"0.27826086956521734","last_synced_commit":"47e162b82356220f4a303475ea0a48b3b0e4eaa4"},"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakarumych%2Falkahest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakarumych%2Falkahest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakarumych%2Falkahest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zakarumych%2Falkahest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zakarumych","download_url":"https://codeload.github.com/zakarumych/alkahest/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247318745,"owners_count":20919484,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["network-programming","protocols","rust","serialization"],"created_at":"2024-11-10T02:52:29.301Z","updated_at":"2025-04-05T10:10:10.662Z","avatar_url":"https://github.com/zakarumych.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Alkahest - Fantastic serialization library.\n\n[![crates](https://img.shields.io/crates/v/alkahest.svg?style=for-the-badge\u0026label=alkahest)](https://crates.io/crates/alkahest)\n[![docs](https://img.shields.io/badge/docs.rs-alkahest-66c2a5?style=for-the-badge\u0026labelColor=555555\u0026logoColor=white)](https://docs.rs/alkahest)\n[![actions](https://img.shields.io/github/actions/workflow/status/zakarumych/alkahest/badge.yml?branch=main\u0026style=for-the-badge)](https://github.com/zakarumych/alkahest/actions/workflows/badge.yml)\n[![MIT/Apache](https://img.shields.io/badge/license-MIT%2FApache-blue.svg?style=for-the-badge)](./COPYING)\n![loc](https://img.shields.io/tokei/lines/github/zakarumych/alkahest?style=for-the-badge)\n\n*Alkahest* is blazing-fast, zero-deps, zero-overhead, zero-unsafe, schema-based\nserialization library.\nIt is suitable for broad range of use-cases, but tailored for\ncustom high-performance network protocols.\n\n### Benchmarks\n\nThis benchmark that mimics some game networking protocol.\n\n|                 | `alkahest`               | `bincode`                       | `rkyv`                        | `speedy`                        |\n| :-------------- | :----------------------- | :------------------------------ | :---------------------------- | :------------------------------ |\n| **`serialize`** | `10.69 us` (✅ **1.00x**) | `11.08 us` (✅ **1.04x slower**) | `12.43 us` (❌ *1.16x slower*) | `11.13 us` (✅ **1.04x slower**) |\n| **`read`**      | `1.19 us` (✅ **1.00x**)  | `9.19 us` (❌ *7.74x slower*)    | `2.10 us` (❌ *1.77x slower*)  | `1.54 us` (❌ *1.30x slower*)    |\n\n---\nMade with [criterion-table](https://github.com/nu11ptr/criterion-table)\n\n\nSee also [benchmark results](./BENCHMARKS.md) from \u003chttps://github.com/djkoloski/rust_serialization_benchmark\u003e (in draft until 0.2 release).\n\n## Features\n\n* **Schema-based serialization**.\n  Alkahest uses data schemas called `Formula`s to serialize and deserialize data.\n  Thus controlling data layout independently from data types that are serialized\n  or deserialized.\n\n* **Support wide variety of formulas**.\n  Integers, floats, booleans, tuples, arrays, slices, strings and\n  user-defined formulas with custom data layout using `derive` macro\n  that works for structs and enums of any complexity and supports generics.\n\n* **Zero-overhead serialization of sequences**.\n  Alkahest support serializing iterators directly into slice formulas.\n  No more allocation of a `Vec` to serialize and drop immediately.\n\n* **Lazy deserialization**.\n  Alkahest provides `Lazy\u003cF\u003e` type to deserialize any formula `F` lazily.\n  `Lazy` can be used later to perform actual deserialization.\\\n  `Lazy\u003c[F]\u003e` can also produce iterator that deserializes elements on demand.\\\n  Laziness is controlled on type level and can be applied to any element\n  of a larger formula.\n\n* **Infallible serialization**.\n  Given large enough or growing buffer any value that implements `Serialize`\n  can be serialized without error.\n  No more unnecessary unwraps or puzzles \"what to do if serialization fails?\".\n  The only error condition for serialization is \"data doesn't fit\".\n\n### Planned features\n\n* Serializable formula descriptors\n* Compatibility rules\n* External tool for code-generation for formula descriptors for C and Rust.\n\n## How it works. In more details\n\n*Alkahest* separates data schema definition (aka `Formula`) from\nserialization and deserialization code.\nDoing so, this library provides better guarantees for cases\nwhen serializable data type and deserializable data type\nare different.\nIt also supports serializing from iterators instead of collections\nand deserialization into lazy wrappers that defers costly process\nand may omit it entirely if value is never accessed.\nUser controls laziness on type level by choosing appropriate `Deserialize` impls.\nFor instance deserializing into `Vec\u003cT\u003e` is eager because `Vec\u003cT\u003e` is constructed\nwith all `T` instances and memory allocated for them.\nWhile `alkahest::SliceIter` implements `Iterator` and deserializes\nelements in `Iterator::next` and other methods. And provides constant-time\nrandom access to any element.\n\nFlexibility comes at cost of using only byte slices for\nserialization and deserialization.\nAnd larger footprint of serialized data than some other binary formats.\n\nQuestion about support of dense data packing is open.\nIt may be desireable to control on type level.\n\n### Errors and panics\n\nThe API is designed with following principles:\nAny value can be serialized successfully given large enough buffer.\nData can't cause panic, incorrect implementation of a trait can.\n\nThere is *zero* unsafe code in the library on any code it generates.\nNo UB is possible given that `std` is not unsound.\n\n### Forward and backward compatibility\n\nNo data schemas stays the same.\nNew fields and variants are added,\nothers are deprecated and removed.\n\nThere's set of rules that ensures forward compatibility between formulas.\nAnd another set or rules for backward compatibility.\n\nVerification of compatibility is not implemented yet.\n\n### Forward compatibility\n\nForward compatibility is an ability to deserialize data\nthat was serialized with newer formulas.\n\nTODO: List all rules\n\n### Backward compatibility\n\nBackward compatibility is an ability to deserialize data\nthat was serialized with older formulas.\n\nTODO: List all rules\n\n## Formula, Serialize and Deserialize traits.\n\nThe crate works using three fundamental traits.\n`Formula`, `Serialize` and `Deserialize`.\nThere's also supporting trait - `BareFormula`.\n\n*Alkahest* provides proc-macro `alkahest` for deriving `Formula`, `Serialize` and `Deserialize`.\n\n### Formula\n\n`Formula` trait is used to allow types to serve as data schemas.\nAny value serialized with given formula should be deserializable with the same\nformula. Sharing only `Formula` type allows modules and crates\neasily communicate.\n`Formula` dictates binary data layout and it *must* be platform-independent.\n\nPotentially `Formula` types can be generated from separate files,\nopening possibility for cross-language communication.\n\n`Formula` is implemented for a number of types out-of-the-box.\nPrimitive types like `bool`, integers and floating point types all implement `Formula`.\n*!Caveat!*:\n  Serialized size of `isize` and `usize` is controlled by a feature-flag.\n  Sizes and addresses are serialized as `usize`.\n  Truncating `usize` value if it was too large.\n  This may result in broken data generated and panic in debug.\nIt is also implemented for tuples, array and slice, `Option` and `Vec` (the later requires `\"alloc\"` feature).\n\nThe easiest way to define a new formula is to derive `Formula` trait for a struct or an enum.\nGenerics are supported, but may require complex bounds specified in attributes for\n`Serialize` and `Deserialize` derive macros.\nThe only constrain is that all fields must implement `Formula`.\n### Serialize\n\n`Serialize\u003cFormula\u003e` trait is used to implement serialization\naccording to a specific formula.\nSerialization writes to mutable bytes slice and *should not*\nperform dynamic allocations.\nBinary result of any type serialized with a formula must follow it.\nAt the end, if a stream of primitives serialized is the same,\nbinary result should be the same.\nTypes may be serializable with different formulas producing\ndifferent binary result.\n\n`Serialize` is implemented for many types.\nMost notably there's implementation `T: Serialize\u003cT\u003e`\nand `\u0026T: Serialize\u003cT\u003e` for all primitives `T` (except `usize` and `isize`).\nAnother important implementation is\n`Serialize\u003cF\u003e for I where I: IntoIterator, I::Item: Serialize\u003cF\u003e`,\nallowing serializing into slice directly from both iterators and collections.\nSerialization with formula `Ref\u003cF\u003e` uses serialization with formula `F`\nand then stores relative address and size. No dynamic allocations is required.\n\nDeriving `Serialize` for a type will generate `Serialize` implementation,\nformula is specified in attribute `#[alkahest(FormulaRef)]` or\n`#[alkahest(serialize(FormulaRef))]`. `FormulaRef` is typically a type.\nWhen generics are used it also contains generic parameters and bounds.\nIf formula is not specified - `Self` is assumed.\n`Formula` should be derived for the type as well.\nIt is in-advised to derive `Serialize` for formulas with\nmanual `Formula` implementation,\n`Serialize` derive macro generates code that uses non-public items\ngenerated by `Formula` derive macro.\nSo either both *should have* manual implementation or both derived.\n\nFor structures `Serialize` derive macro requires that all fields\nare present on both `Serialize` and `Formula` structure and has the same\norder (trivially if this is the same structure).\n\nFor enums `Serialize` derive macro checks that for each variant there\nexists variant on `Formula` enum.\nVariants content is compared similar to structs.\nSerialization inserts variant ID and serializes variant as struct.\nThe size of variants may vary. Padding is inserted by outer value serialization\nif necessary.\n\n`Serialize` can be derived for structure where `Formula` is an enum.\nIn this case variant should be specified using\n`#[alkahest(@variant_ident)]` or `#[alkahest(serialize(@variant_ident))]`\nand then `Serialize` derive macro will produce serialization code that works\nas if this variant was a struct `Formula`,\nexcept that variant's ID will be serialized before fields.\n\n`Serialize` can be derived for enum only if `Formula` is enum as well.\nSerializable enum may omit some (or all) variants from `Formula`.\nIt may not have variants missing in `Formula`.\nEach variant then follows rules for structures.\n\nFor convenience `Infallible` implements `Serialize` for enum formulas.\n\n### Deserialize\n\n`Deserialize\u003c'de, Formula\u003e` trait is used to implement deserialization \naccording to a specific formula.\nDeserialization reads from bytes slice constructs deserialized value.\nDeserialization *should not* perform dynamic allocations except those\nthat required to construct and initialize deserialized value.\nE.g. it is allowed to allocate when `Vec\u003cT\u003e` is produced if non-zero\nnumber of `T` values are deserialized. It *should not* over-allocate.\n\nSimilar to `Serialize` *alkahest* provides a number of out-of-the-box\nimplementations of `Deserialize` trait.\n`From\u003cT\u003e` types can be deserialized with primitive formula `T`.\n\nValues that can be deserialized with formula `F`\ncan also deserialize with `Ref\u003cF\u003e`, it reads address and length\nand proceeds with formula `F`.\n\n`Vec\u003cT\u003e` may deserialize with slice formula.\n`Deserialize\u003c'de, [F]\u003e` is implemented for `alkahest::SliceIter\u003c'de, T\u003e` type\nthat implements `Iterator` and lazily deserialize elements of type\n`T: Deserialize\u003c'de, F\u003e`. `SliceIter` is cloneable,\ncan be iterated from both ends and skips elements for in constant time.\nFor convenience `SliceIter` also deserializes with array formula.\n\nDeriving `Deserialize` for a type will generate `Deserialize` implementation,\nformula is specified in attribute `#[alkahest(FormulaRef)]` or\n`#[alkahest(deserialize(FormulaRef))]`. `FormulaRef` is typically a type.\nWhen generics are used it also contains generic parameters and bounds.\nIf formula is not specified - `Self` is assumed.\n`Formula` should be derived for the type as well.\nIt is in-advised to derive `Deserialize` for formulas with\nmanual `Formula` implementation,\n`Deserialize` derive macro generates code that uses non-public items\ngenerated by `Formula` derive macro.\nSo either both *should have* manual implementation or both derived.\n\n## Interoperability with `serde`\n\n*Alkahest* is cool but `serde` is almost universally used, and for good reasons.\nWhile designing a `Formula` it may be desireable to include existing type\nthat supports serialization `serde`, especially if it comes from another crate.\nThis crate provides `Bincode` and `Bincoded\u003cT\u003e` formulas to cover this.\nAnything with `serde::Serialize` implementation can be serialized with `Bincode`\nformula, naturally it will be serialized using `bincode` crate.\n`Bincoded\u003cT\u003e` is a restricted version of `Bincode` that works only for `T`.\n\n# Usage example\n\n```rust\n// This requires two default features - \"alloc\" and \"derive\".\n#[cfg(all(feature = \"derive\", feature = \"alloc\"))]\nfn main() {\n  use alkahest::{alkahest, serialize_to_vec, deserialize};\n\n  // Define simple formula. Make it self-serializable.\n  #[derive(Clone, Debug, PartialEq, Eq)]\n  #[alkahest(Formula, SerializeRef, Deserialize)]\n  struct MyDataType {\n    a: u32,\n    b: Vec\u003cu8\u003e,\n  }\n\n  // Prepare data to serialize.\n  let value = MyDataType {\n    a: 1,\n    b: vec![2, 3],\n  };\n\n  // Use infallible serialization to `Vec`.\n  let mut data = Vec::new();\n\n  // Note that this value can be serialized by reference.\n  // This is default behavior for `Serialized` derive macro.\n  // Some types required ownership transfer for serialization.\n  // Notable example is iterators.\n  let (size, _) = serialize_to_vec::\u003cMyDataType, _\u003e(\u0026value, \u0026mut data);\n\n  let de = deserialize::\u003cMyDataType, MyDataType\u003e(\u0026data[..size]).unwrap();\n  assert_eq!(de, value);\n}\n\n#[cfg(not(all(feature = \"derive\", feature = \"alloc\")))]\nfn main() {}\n```\n\n# Benchmarking\n\nAlkahest comes with a benchmark to test against other popular serialization crates.\nSimply run `cargo bench --all-features` to see results.\n\n## License\n\nLicensed under either of\n\n* Apache License, Version 2.0, ([license/APACHE](./license/APACHE) or \u003chttp://www.apache.org/licenses/LICENSE-2.0\u003e)\n* MIT license ([license/MIT](./license/MIT) or \u003chttp://opensource.org/licenses/MIT\u003e)\n\nat your option.\n\n## Contributions\n\nUnless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakarumych%2Falkahest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzakarumych%2Falkahest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzakarumych%2Falkahest/lists"}