{"id":23429837,"url":"https://github.com/dergoegge/btcser","last_synced_at":"2025-10-30T12:30:58.021Z","repository":{"id":268421534,"uuid":"904292797","full_name":"dergoegge/btcser","owner":"dergoegge","description":"Descriptor language for Bitcoin's serialization format","archived":false,"fork":false,"pushed_at":"2025-02-05T11:56:33.000Z","size":168,"stargazers_count":4,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-02-15T07:12:30.361Z","etag":null,"topics":["bitcoin","fuzzing","structure-aware-fuzzing"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/dergoegge.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-16T15:57:46.000Z","updated_at":"2025-02-05T11:55:52.000Z","dependencies_parsed_at":"2024-12-16T17:34:45.955Z","dependency_job_id":"ad440619-7293-4d7a-bfe6-84f4ee39d0d8","html_url":"https://github.com/dergoegge/btcser","commit_stats":null,"previous_names":["dergoegge/btcser"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dergoegge%2Fbtcser","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dergoegge%2Fbtcser/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dergoegge%2Fbtcser/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/dergoegge%2Fbtcser/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/dergoegge","download_url":"https://codeload.github.com/dergoegge/btcser/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":238962775,"owners_count":19559540,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bitcoin","fuzzing","structure-aware-fuzzing"],"created_at":"2024-12-23T08:12:58.206Z","updated_at":"2025-10-30T12:30:57.721Z","avatar_url":"https://github.com/dergoegge.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"*Note: This is a work in progress.*\n\n# btcser\n\nbtcser is a descriptor language for Bitcoin's serialization format (used\nheavily in Bitcoin Core for data storage as well as the Bitcoin p2p protocol).\nIt is designed to be simple and easy to write for those familiar with Bitcoin's\nserialization format.\n\nLet's look at a simple example:\n\n```text\nfoo { u8, u64, bytes\u003c8\u003e }\n```\n\nIn this example, a new type `foo` is defined which consists of three fields: a\n`u8`, a `u64`, and a `bytes\u003c8\u003e`. `u8` is a 8-bit unsigned integer, `u64` is a\n64-bit little-endian unsigned integer, and `bytes\u003c8\u003e` is an 8-byte byte array.\nUsing Bitcoin's serialization format, this type would always be represented as\n17 bytes. The first byte represent the `u8` field, the next 8 bytes represent\nthe `u64` field, and the last 8 bytes represent the `bytes\u003c8\u003e` field.\n\nThe equivalent C++ type would look as follows (using Bitcoin Core's\nserialization framework):\n\n```c++\nstruct Foo {\npublic:\n    uint8_t field0;\n    uint64_t field1;\n    std::array\u003cuint8_t, 8\u003e field2;\n\n    SERIALIZE_METHODS(Foo, obj)\n    {\n        READWRITE(obj.field0, obj.field1, obj.field2);\n    }\n};\n```\n\n## Built-in Types\nBesides the three basic types used above, btcser also supports the following\nbuild-in types:\n\n- `bool`: a boolean type\n- `u8`, `i8`: 8-bit unsigned and signed integers\n- `u16`, `u32`, `u64`, `u256`: unsigned integers of various sizes\n  (little-endian)\n- `i16`, `i32`, `i64`, `i256`: signed integers of various sizes (little-endian)\n- `U16`, `U32`, `U64`, `U256`: unsigned integers of various sizes (big-endian)\n- `I16`, `I32`, `I64`, `I256`: signed integers of various sizes (big-endian)\n- `vec\u003cT\u003e`: a vector of type `T`\n- `slice\u003cT, 'f'\u003e`: a slice of type `T` with its length specified by a preceding\n  field at index `f`\n- `bytes\u003cN\u003e`: a fixed-size byte array of length `N`\n- `cs64`: compact size encoding of up to 64-bit integers\n- `varint`: variable length integer encoding (default mode)\n- `varint+`: variable length integer encoding (signed non-negative mode)\n\n## Type Composition\n\nLet's look at another example:\n\n```text\nfoo { u8, u64, bytes\u003c8\u003e }\n\nbar { vec\u003cfoo\u003e }\n```\n\nIn this example, `bar` is a type that consists of a vector of `foo`s. In\nserialized form `bar` is of variable length depending on the number of `foo`s\nit contains. Note how the `foo` type is defined in the first line and then used\nas part of `bar`'s definition. This is how we can compose complex types from\nsimpler ones. For example, here is the Bitcoin `headers` protocol message\nexpressed in btcser:\n\n```text\nheader { i32,u256,u256,u32,u32,u32 }\nempty_block { header,u8(0) }\nheaders { vec\u003cempty_block\u003e }\n```\n\n## Slices\n\nThe `slice\u003cT, 'f'\u003e` type is used to specify a list of same-typed elements with\nthe list's length specified by a preceding field at index `f`. For example:\n\n```text\nbar { cs64, slice\u003cu64, '0'\u003e }\n```\n\nIn this example, `bar` is a type that consists of a `cs64` field and a\n`slice\u003cu64, '0'\u003e` field. The `cs64` field is a compact size encoding of a\n64-bit integer, and the `slice\u003cu64, '0'\u003e` field is a slice of `u64`s with its\nlength specified by the `cs64` field. Those familiar with Bitcoin's\nserialization format will recognize that this is the way that variable length\narrays are encoded. In short, it is the same as `vec\u003cu64\u003e`.\n\n## Constants\n\nbtcser also supports constants as part of type definitions. For example:\n\n```text\nbar { bytes\u003c4\u003e(0xdeadbeef), u8(0xff) }\n```\n\nIn this example, the `bar` type consists of a `bytes\u003c4\u003e` constant and a `u8`\nconstant. The `bytes\u003c4\u003e` constant is a 4-byte byte array with the value\n`0xdeadbeef`, and the `u8` constant is an 8-bit unsigned integer with the value\n`0xff`.\n\n## Alternatives\n\nbtcser also supports alternatives, which are used to express the idea that\ntypes can have alternative serialization formats. For example:\n\n```text\nbar { u8(0x01), vec\u003cu64\u003e }\nbar { u8(0x02), bytes\u003c32\u003e }\n\nfoo { bar }\n```\n\nIn this example, `foo` holds a `bar` field which can be either a `u8(0x01)` and\na `vec\u003cu64\u003e` or a `u8(0x02)` and a `bytes\u003c32\u003e`. Any object of type `foo` has to\nbe prefixed with either a `u8(0x01)` or a `u8(0x02)`.\n\n*See [examples/](btcser/examples/) for more examples.*\n\n## Applications in Fuzzing\n\nbtcser is primarily intended to be used for fuzzing Bitcoin full-node software,\nas serializable types are often part of their interfaces. It allows for\nstructure-aware fuzzing of interfaces accepting Bitcoin serialized data, as it\nprovides fuzz engines with structure and type information that is otherwise\ninvisible to them.\n\nWithout being aware of the serialization structure, fuzz engines often mutate\ninputs in ways that cause deserialization to fail (since they only operate on\nbyte arrays). When attempting to test logic that comes after inputs are\ndeserialized in a target, then violating serialization rules is a waste of CPU\ncycles.\n\nStructure and type information are very useful for cross-over mutations between\ndifferent serialized objects of the same schema. Let's consider this example:\n\n```\ntx { ... } # fields omitted for brevity (this is how comments work in btcser)\nheader { i32,u256,u256,u32,u32,u32 }\nblock { header, vec\u003ctx\u003e }\n```\n\nA byte array fuzzer, attempting to perform cross-over mutations between `block`\nobjects would frequently invalidate serialization structure. It simply isn't\naware of the individual fields and their types.\n\nWith btcser's additional structure and type information, it can now perform\ncross-over mutations between `block` objects without messing with the overall\nstructure. Type information helps crossing over values of the same type, e.g.\nadd a transaction from one block to another, copy the prev block hash from one\nheader to another, etc.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdergoegge%2Fbtcser","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdergoegge%2Fbtcser","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdergoegge%2Fbtcser/lists"}