{"id":13804529,"url":"https://github.com/atilaneves/cerealed","last_synced_at":"2026-01-05T21:04:31.672Z","repository":{"id":9676122,"uuid":"11619359","full_name":"atilaneves/cerealed","owner":"atilaneves","description":"Powerful binary serialisation library for D","archived":false,"fork":false,"pushed_at":"2023-06-02T13:36:20.000Z","size":251,"stargazers_count":92,"open_issues_count":5,"forks_count":3,"subscribers_count":9,"default_branch":"master","last_synced_at":"2025-02-05T02:11:11.773Z","etag":null,"topics":["custom-serialisation","d","dlang","dlanguage","packets","serialisation","unmarshall"],"latest_commit_sha":null,"homepage":"","language":"D","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/atilaneves.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2013-07-23T21:13:10.000Z","updated_at":"2024-08-19T06:24:40.000Z","dependencies_parsed_at":"2024-01-03T01:30:28.478Z","dependency_job_id":null,"html_url":"https://github.com/atilaneves/cerealed","commit_stats":null,"previous_names":[],"tags_count":26,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atilaneves%2Fcerealed","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atilaneves%2Fcerealed/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atilaneves%2Fcerealed/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/atilaneves%2Fcerealed/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/atilaneves","download_url":"https://codeload.github.com/atilaneves/cerealed/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245153896,"owners_count":20569408,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["custom-serialisation","d","dlang","dlanguage","packets","serialisation","unmarshall"],"created_at":"2024-08-04T01:00:49.463Z","updated_at":"2026-01-05T21:04:26.613Z","avatar_url":"https://github.com/atilaneves.png","language":"D","funding_links":[],"categories":["Data serialization"],"sub_categories":["Binary Serilization"],"readme":"cerealed\n=============\n[![Build Status](https://travis-ci.org/atilaneves/cerealed.png?branch=master)](https://travis-ci.org/atilaneves/cerealed)\n[![Coverage](https://codecov.io/gh/atilaneves/cerealed/branch/master/graph/badge.svg)](https://codecov.io/gh/atilaneves/cerealed)\n\n[My DConf 2014 talk mentioning Cerealed](https://www.youtube.com/watch?v=xpImt14KTdc).\n\nBinary serialisation library for D. Minimal to no boilerplate necessary. Example usage:\n\n```d\n    import cerealed;\n\n    assert(cerealise(5) == [0, 0, 0, 5]); // returns ubyte[]\n    cerealise!(a =\u003e assert(a == [0, 0, 0, 5]))(5); // faster than using the bytes directly\n\n    assert(decerealise!int([0, 0, 0, 5]) == 5);\n\n    struct Foo { int i; }\n    const foo = Foo(5);\n    // alternate spelling\n    assert(foo.cerealize.decerealize!Foo == foo);\n```\n\nThe example below shows off a few features. First and foremost, members are serialised\nautomatically, but can be opted out via the `@NoCereal` attribute. Also importantly,\nmembers to be serialised in a certain number of bits (important for binary protocols)\nare signalled with the `@Bits` attribute with a compile-time integer specifying the\nnumber of bits to use.\n\n```d\n    struct MyStruct {\n        ubyte mybyte1;\n        @NoCereal uint nocereal1; //won't be serialised\n        @Bits!4 ubyte nibble;\n        @Bits!1 ubyte bit;\n        @Bits!3 ubyte bits3;\n        ubyte mybyte2;\n    }\n\n    assert(MyStruct(3, 123, 14, 1, 2, 42).cerealise == [ 3, 0xea /*1110 1 010*/, 42]);\n```\n\nWhat if custom serialisation is needed and the default, even with opt-outs, won't work?\nIf an aggregate type defines a member function `void accept(C)(ref C cereal)` it will be used\ninstead. To get the usual automatic serialisation from within the custom `accept`,\nthe `grainAllMembers` member function of Cereal can be called, as shown in the\nexample below. This function takes a ref argument so rvalues need not apply.\n\nThe function to use on `Cereal` to marshall or unmarshall a particular value is `grain`.\nThis is essentially what `Cerealiser.~=` and `Decerealiser.value` are calling behind\nthe scenes (and therefore `cerealise` and `decerealise`).\n\n```d\n    struct CustomStruct {\n        ubyte mybyte;\n        ushort myshort;\n        void accept(C)(auto ref C cereal) {\n             //do NOT call cereal.grain(this), that would cause an infinite loop\n             cereal.grainAllMembers(this);\n             ubyte otherbyte = 4; //make it an lvalue\n             cereal.grain(otherbyte);\n        }\n    }\n\n    assert(CustomStruct(1, 2).cerealise == [ 1, 0, 2, 4]);\n\n    //because of the custom serialisation, passing in just [1, 0, 2] would throw\n    assert([1, 0, 2, 4].decerealise!CustomStruct == CustomStruct(1, 2));\n```\n\nThe other option when custom serialisation is needed that avoids boilerplate is to\ndefine a `void postBlit(C)(ref C cereal)` function instead of `accept`. The\nmarshalling or unmarshalling is done as it would in the absence of customisation,\nand `postBlit` is called to fix things up. It is a compile-time error to\ndefine both `accept` and `postBlit`. Example below.\n\n```d\n    struct CustomStruct {\n        ubyte mybyte;\n        ushort myshort;\n        @NoCereal ubyte otherByte;\n        void postBlit(C)(auto ref C cereal) {\n             //no need to handle mybyte and myshort, already done\n             if(mybyte == 1) {\n                 cereal.grain(otherByte);\n             }\n        }\n    }\n\n    assert(CustomStruct(1, 2).cerealise == [ 1, 0, 2, 4]);\n    assert(CustomStruct(3, 2).cerealise == [ 1, 0, 2]);\n```\n\nFor more examples of how to serialise structs, check the [tests](tests) directory\nor real-world usage in my [MQTT broker](https://github.com/atilaneves/mqtt)\nalso written in D.\n\nArrays are by default serialised with a ushort denoting array length followed\nby the array contents. It happens often enough that networking protocols\nhave explicit length parameters for the whole packet and that array lengths\nare implicitly determined from this. For this use case, the `@RestOfPacket`\nattribute tells `cerealed` to not add the length parameter. As the name implies,\nit will \"eat\" all bytes until there aren't any left.\n\n```d\n    private struct StringsStruct {\n        ubyte mybyte;\n        @RestOfPacket string[] strings;\n    }\n\n    //no length encoding for the array, but strings still get a length each\n    const bytes = [ 5, 0, 3, 'f', 'o', 'o', 0, 6, 'f', 'o', 'o', 'b', 'a', 'r',\n                    0, 6, 'o', 'h', 'w', 'e', 'l', 'l'];\n    const strs = StringStruct(5, [\"foo\", \"foobar\", \"ohwell\"]);\n    assert(strs.cerealise == bytes);\n    assert(bytes.decerealise!StringsStruct ==  strs);\n```\n\nDerived classes can be serialised via a reference to the base class, but the\nchild class must be registered first:\n\n```d\n    class BaseClass  { int a; this(int a) { this.a = a; }}\n    class ChildClass { int b; this(int b) { this.b = b; }}\n    Cereal.registerChildClass!ChildClass;\n    BaseClass obj = ChildClass(3, 7);\n    assert(obj.cerealise == [0, 0, 0, 3, 0, 0, 0, 7]);\n```\n\nThere is now support for InputRange and OutputRange objects. Examples can\nbe found in the [tests directory](tests/range.d)\n\nAdvanced Usage\n---------------\nFrequently in networking programming, the packets themselves encode the length\nof elements to follow. This happens often enough that Cerealed has two UDAs\nto automate this kind of serialisation: `@ArrayLength` and `@LengthInBytes`.\nThe former specifies how to get the length of an array (usually a variable)\nThe latter specifies how many bytes the array takes. Examples:\n\n```d\n    struct Packet {\n        ushort length;\n        @ArrayLength(\"length\") ushort[] array;\n    }\n    auto pkt = decerealise!Packet([\n        0, 3, //length\n        0, 1, 0, 2, 0, 3]); //array of 3 ushorts\n    assert(pkt.length == 3);\n    assert(pkt.array == [1, 2, 3]);\n\n    struct Packet {\n        static struct Header {\n            ubyte ub;\n            ubyte totalLength;\n        }\n        enum headerSize = unalignedSizeof!Header; //2 bytes\n\n        Header header;\n        @LengthInBytes(\"totalLength - headerSize\") ushort[] array;\n    }\n    auto pkt = decerealise!Packet([\n        7, //ub1\n        6, //totalLength in bytes\n        0, 1, 0, 2]); //array of 2 ushorts\n    assert(pkt.ub1 == 7);\n    assert(pkt.totalLength == 6);\n    assert(pkt.array == [1, 2]);\n```\n\nRelated Projects\n----------------\n- [orange](https://github.com/jacob-carlborg/orange).\n- [msgpack-d](https://github.com/msgpack/msgpack-d).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatilaneves%2Fcerealed","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fatilaneves%2Fcerealed","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fatilaneves%2Fcerealed/lists"}