{"id":16340850,"url":"https://github.com/casey/x-serialization-format","last_synced_at":"2026-02-18T15:01:23.351Z","repository":{"id":81088371,"uuid":"263517372","full_name":"casey/x-serialization-format","owner":"casey","description":null,"archived":false,"fork":false,"pushed_at":"2021-03-18T00:50:06.000Z","size":195,"stargazers_count":1,"open_issues_count":129,"forks_count":1,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-25T14:40:37.727Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/casey.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-05-13T03:40:02.000Z","updated_at":"2023-04-21T22:16:18.000Z","dependencies_parsed_at":null,"dependency_job_id":"345b0542-ded7-4b80-a8cd-f3349aed7a87","html_url":"https://github.com/casey/x-serialization-format","commit_stats":null,"previous_names":["casey/x-serialization-format"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/casey/x-serialization-format","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casey%2Fx-serialization-format","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casey%2Fx-serialization-format/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casey%2Fx-serialization-format/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casey%2Fx-serialization-format/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/casey","download_url":"https://codeload.github.com/casey/x-serialization-format/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/casey%2Fx-serialization-format/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29582807,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T13:56:48.962Z","status":"ssl_error","status_checked_at":"2026-02-18T13:54:34.145Z","response_time":162,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-10T23:58:02.861Z","updated_at":"2026-02-18T15:01:23.326Z","avatar_url":"https://github.com/casey.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# x\n\nThis library implements a serialization format for Rust. The name \"x\" is a\nplaceholder, hopefully someone comes up with a better one soon.\n\nThis project is currently dreamware. It exists only as documentation, issues,\nand figments of the imagination.\n\nThat being said, this readme is written as if everything is already done, so I\ndon't have to go and rewrite it later.\n\n## Features\n\n- Fast: Serialization and deserialization are efficient, making x a good\n  format for use in resource-constrained environments.\n\n- Excellent Rust integration: Standard library types are supported where\n  possible, and enum access is idiomatic and ergonomic.\n\n- No external schema language or additional build steps required: Schemas are\n  declared in Rust with procedural macros.\n\n- Schema evolution: Fields can be added and optional fields removed without\n  breaking backwards compatibility.\n\n- `nostd` support: Neither the standard library nor the `alloc` crate are\n  required to serialize or deserialize messages.\n\n- Zero parse: Accessing fields of a message only requires loading and following\n  offsets, making it very fast. There is, however, an up-front validation step,\n  albeit a very efficient one.\n\n- Zero copy: Deserialization and access do not require memory beyond that\n  used to store the message itself.\n\n- `mmap`-friendly: Messages can be mapped into memory and accessed in-place, making\n  x a good choice for large messages and technologies like LMDB.\n\n- Canonicality: For any message, a canonical serialization of that message is\n  defined. For ergonomic and efficiency reasons, this may be opt-in.\n\n- Simple: The encoding is straightforward and easy to understand. If you\n  absolutely had to, writing your own deserializer or serializer would be\n  straightforward.\n\n## Non-goals\n\n- Languages other than Rust: Currency, only Rust support is planned. If there\n  is demand, support for other languages may implemented, but it isn't a\n  near- or medium-term goal.\n\n## Prior Art\n\nX draws inspiration from many other formats.\n\nThe most similar format is probably\n[FIDL](https://fuchsia.dev/fuchsia-src/development/languages/fidl), the Fuchsia\nInterface Definition Language, a schema language and wire format used for\ninter-process communication in the Fuchsia operating system.\n\nUnfortunately, FIDL is difficult to use outside of the Fuchsia source tree,\nrequires an external schema, and use-cases outside of Fuchsia are not\nwell-supported.\n\nX is also very similar to Flatbuffers and Cap'n Proto, but differs in that\nit does not require an external schema definition, and is designed from the\nground up to have excellent Rust support.\n\n## Encoding\n\nMessage encoding is hopefully straightforward, and is intended to be both\nsimple and preferment.\n\nValues have no alignment guarantees, so sequential elements are always directly\nadjacent, with no padding. In general, modern computers have little to no\nunaligned access penalty, so the format avoids the complexity of guaranteeing\nalignment, and the space overhead of padding.\n\nIntegers are little-endian, regardless of platform, since most computers are\nlittle-endian.\n\nIn order to support efficient traversal, all variable-length data is stored\nout-of-line as a relative offset. Since the length of non-variable length data\nis known in advance, traversal never requires inspecting so as to be able to\nskip over variable length data.\n\nBy way of illustration, consider the following struct:\n\n```rust\nuse x::{Slice, U8, U16, U32};\n\n#[derive(X)]\n#[repr(C)]\nstruct Foo {\n  a: U8,\n  b: Slice\u003cU16\u003e,\n  c: U32,\n}\n```\n\nConceptually, this struct contains a `u8`, followed by a variable number of\n`u16`s, and finally followed by a `u32`.\n\nWhen serialized, the struct will have the following layout in memory:\n\n```\n0x0000 A         # 1 byte value of 'a'\n0x0001 OOOO OOOO # 8 byte offset to 'b'\n0x0009 LLLL LLLL # 8 byte length of 'b'\n0x0011 CCCC      # 4 byte value of 'c'\n0x0015 BBBBBBBB… # contents of 'b'\n```\n\nNotice that because 'b' is stored out-of-line, we can access 'c' directly.\nGiven a pointer `p` to `Foo`, `c` will always be at `p + 0x11`.\n\nTo access variable length data, in this case 'b', we calculate the pointer to\nthe offset, `p + 0x1`, load the value it points to, and add it to the pointer\nto the offset.\n\nIn pseudocode:\n\n```rust\nfn load_slice(pointer: *const u64) -\u003e \u0026[u8] {\n  let offset = *pointer;\n  let len = *(pointer + 0x8);\n  let data = pointer + offset;\n  std::slice::from_raw_parts(data, len)\n}\n```\n\nOffsets are relative to their own location, so zero is not a valid offset,\nsince that would point to the offset itself. So, zero is used as a sentinel\nvalue, for example to represent `Option::None`.\n\n### Types\n\n#### `()`\n\nSince `()`, also called the unit type, has a single value, it requires zero bits to\nrepresent, and thus the encoding is the empty byte string.\n\n#### `bool`\n\nBoolean `true` is encoded as a byte containing the value `1`, and `false` is\nencoded as a byte containing the value `0`.\n\n#### `u8`, `u16`, `u32`, `u64`, `u128`\n\nUnsigned integers are encoded as little endian, and are fixed width. `u8` is\nalways 1 byte, `u16` two bytes, and so on.\n\n#### `i8`, `i16`, `i32`, `i64`, `i128`\n\nSigned integers are encoded as little endian two's complement, and are fixed\nwidth. `i8` is always 1 byte, `i16` two bytes, and so on.\n\n#### `usize`, `isize`\n\n`usize` and `isize` are always encoded as a `u64` and a `i64`, respectively,\nregardless of platform. However, validation will fail if the encoded value is\ntoo big to fit into the platform's size type.\n\n#### `char`\n\nCharacters are encoded as as the little endian bytes of the the Unicode scalar\nvalue they represent. Since only 3 bytes are required to represent all unicode\nscalar values, `char`s are encoded as 3 bytes intead of the usual 4.\n\n#### `\u0026str`\n\nStrings are encoded as a relative offset pointing to the UTF-8 encoded contents\nof the string, and the length of the contents in bytes, encoded as a `u64`\n\n### `CStr`\n\nC strings are encoded as a relative offset pointing to their contents, and the\nlength of the string in bytes. The contents are a null-terminated byte sequence.\n\n#### `\u0026[T]`\n\nSlices are encoded as a relative offset pointing to the contents of the slice,\nand the length of the slice. The length is the number of elements in the slice,\nnot the number of bytes.\n\n#### `Option\u003cT\u003e`\n\nOptions that contain a value are encoded as a relative offset to the contained\n`T`. `None` is encoded as a zero offset.\n\n#### `Result\u003cT, E\u003e`\n\nResults are encoded as a byte containing `0` for `Ok` or `1` for `Err`,\nfollowed in either case by a relative offset to the contained `T` or `E`.\n\n#### Structs\n\nStructs are encoded as the encoding of each field, in the order they appear in\nthe struct declaration. It is not possible to make backwards compatible changes\nto structs. For structs which might need to change in the future, see flexible\nstructs.\n\n#### Enums\n\nThe different values of an enum are called its “variants”. These are identified\nby an unsigned integer called the enum's “discriminant”.\n\nEnums are encoded as a `u8` containing the enum's discriminant, followed by the\npayload, if any, encoded in the same format as structs.\n\nVariants cannot be added or removed from an enum. For enums which may need to\nchange in the future, see flexible enums.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcasey%2Fx-serialization-format","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcasey%2Fx-serialization-format","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcasey%2Fx-serialization-format/lists"}