Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/spider-gazelle/bindata
BinData - Parsing Binary Data in Crystal Lang
https://github.com/spider-gazelle/bindata
Last synced: 9 days ago
JSON representation
BinData - Parsing Binary Data in Crystal Lang
- Host: GitHub
- URL: https://github.com/spider-gazelle/bindata
- Owner: spider-gazelle
- License: mit
- Created: 2019-02-20T06:37:49.000Z (over 5 years ago)
- Default Branch: master
- Last Pushed: 2024-07-19T00:10:54.000Z (4 months ago)
- Last Synced: 2024-08-01T17:33:15.719Z (3 months ago)
- Language: Crystal
- Size: 115 KB
- Stars: 48
- Watchers: 8
- Forks: 5
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- awesome-crystal - BinData - Binary data parser helper with an [ASN.1](https://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One) parser (Data Formats)
- awesome-crystal - BinData - Binary data parser helper with an [ASN.1](https://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One) parser (Data Formats)
README
# BinData - Parsing Binary Data in Crystal Lang
BinData provides a declarative way to read and write structured binary data.
This means the programmer specifies what the format of the binary data is, and BinData works out how to read and write data in this format. It is an easier (and more readable) alternative.
[![Build Status](https://github.com/spider-gazelle/bindata/actions/workflows/CI.yml/badge.svg?branch=master)](https://github.com/spider-gazelle/bindata/actions/workflows/CI.yml)
## Usage
Firstly, it's recommended that you specify the datas endian.
```crystal
class Header < BinData
endian big
end
```Then you can specify the structures fields. There are a few different field types:
1. Core types
* `UInt8` to `UInt128` values respectively
* You can use endian-aware types to mix endianess: `, endian: IO::ByteFormat::LittleEndian`
2. Custom types
* anything that is [io serialisable](https://crystal-lang.org/api/0.27.2/IO.html#write_bytes%28object%2Cformat%3AIO%3A%3AByteFormat%3DIO%3A%3AByteFormat%3A%3ASystemEndian%29-instance-method)
3. Bit Fields
* These are a group of fields who values are defined by the number of bits used to represent their value
* The total number of bits in a bit field must be divisible by 8
4. Groups
* These are embedded BinData class with access to the parent fields
* Useful when a group of fields are related or optional
5. Enums
6. Bools
6. Arrays and Sets (fixed size and dynamic)### Examples
see the [spec helper](https://github.com/spider-gazelle/bindata/blob/master/spec/helper.cr) for all possible manipulations
```crystal
enum Inputs
VGA
HDMI
HDMI2
endclass Packet < BinData
endian big# Default sets the value at initialisation.
field start : UInt8 = 0xFF_u8# Value procs assign these values before writing to an IO, overwriting any
# existing value
field size : UInt16, value: ->{ text.bytesize + 1 }# String fields without a length use `\0` null byte termination
# Length is being calculated by the size field above
field text : String, length: ->{ size - 1 }# Bit fields should only be used when one or more fields are not byte aligned
# The sum of the bits in a bit field must be divisible by 8
bit_field do
# a bits value can be between 1 and 128 bits long
bits 5, reserved# Bool values are a single bit
bool set_input = false# This enum is represented by 2 bits
bits 2, input : Inputs = Inputs::HDMI2
end# isolated namespace
group :extended, onlyif: ->{ start == 0xFF } do
field start : UInt8 = 0xFF_u8# Supports custom objects as long as they implement `from_io`
field header : ExtHeader = ExtHeader.new
end# optionally read the remaining bytes out of io
remaining_bytes :rest
end
```The object above can then be accessed like any other object
```crystal
pack = io.read_bytes(Packet)
pack.size # => 12
pack.text # => "hello world"
pack.input # => Inputs::HDMI
pack.set_input # => true
pack.extended.start # => 255
```Additionally, BinData fields support a `verify` proc, which allows data to be verified while reading and writing io.
```crystal
class VerifyData < BinData
endian bigfield size : UInt8
field bytes : Bytes, length: ->{ size }
field checksum : UInt8, verify: ->{ checksum == bytes.reduce(0) { |acc, i| acc + i } }
end
```If the `verify` proc returns `false`, a `BinData::VerificationException` is raised with a message matching the following format.
```
Failed to verify reading basic at VerifyData.checksum
```Inheritance is also supported
## Callbacks
Callbacks can helpful for providing accessors for simplified representations of the data.
```crystal
class CallbackTest < BinData
endian littlefield integer : UInt8
property external_representation : UInt16 = 0
before_serialize { self.integer = (external_representation // 2).to_u8 }
after_deserialize { self.external_representation = integer.to_u16 * 2_u16 }
end
```## ASN.1 Helpers
Included in this library are helpers for decoding and writing ASN.1 data, such as those used in SNMP and LDAP
```crystal
require "bindata/asn1"# Build an object
ber = ASN1::BER.new
ber.tag_number = ASN1::BER::UniversalTags::Integer
ber.payload = Bytes[1]# Write it to an IO:
io.write_bytes(ber)# Read data out of an IO:
ber = io.read_bytes(ASN1::BER)
ber.tag_class # => ASN1::BER::TagClass::Universal```
## Real World Examples
* ASN.1
* https://github.com/crystal-community/jwt/blob/master/src/jwt.cr#L251
* https://github.com/spider-gazelle/crystal-ldap
* enums and bit fields
* https://github.com/spider-gazelle/knx/blob/master/src/knx/cemi.cr#L195
* variable sized arrays
* https://github.com/spider-gazelle/crystal-bacnet/blob/master/src/bacnet/virtual_link_control/secure_bvlci.cr#L54