https://github.com/omar-azmi/fbicodec_ts

A Forward and Backward Invertible binary serializer TypeScript library built on modularity and composition. When you hear the words `FBI OPEN UP!`, you'll be able to prove your innocence with any data encoded with this library.
https://github.com/omar-azmi/fbicodec_ts

binary-serialization codec decoder deno encoder es6 file-parser invertible modular oop parser schema serialization serialization-library tiny typescript unparser unserialize

Last synced: 27 days ago
JSON representation

A Forward and Backward Invertible binary serializer TypeScript library built on modularity and composition. When you hear the words `FBI OPEN UP!`, you'll be able to prove your innocence with any data encoded with this library.

• Host: GitHub
• URL: https://github.com/omar-azmi/fbicodec_ts
• Owner: omar-azmi
• License: other
• Created: 2024-02-06T17:49:53.000Z (9 months ago)
• Default Branch: main
• Last Pushed: 2024-03-26T12:54:10.000Z (8 months ago)
• Last Synced: 2024-10-11T01:10:07.417Z (about 1 month ago)
• Topics: binary-serialization, codec, decoder, deno, encoder, es6, file-parser, invertible, modular, oop, parser, schema, serialization, serialization-library, tiny, typescript, unparser, unserialize
• Language: TypeScript
• Homepage: https://omar-azmi.github.io/fbicodec_ts/
• Size: 244 KB
• Stars: 0
• Watchers: 1
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: readme.md
  • License: license.md
  • Code of conduct: .github/code_of_conduct.md

Awesome Lists containing this project

README

        
# FBIcodec

FBIcodec is so cleanly invertible, it's the last thing the FBI would investigate!

This is a typescript library for building bidirectional (encoding and decoding) codecs using composable steps.

The "FBI" part in the name stands for "Forward-Backward-Interfaced", and Codec stands for an invertible data transformation.

The main building block of the library is its general purpose `Step` interface, which ensures the ability for a forward transformation to be reversed.

The forward and backward methods take a single argument, allowing Steps to be composed into more complex Codecs.

### Features

- Composable steps for encoding and decoding binary data

- Steps for primitive types like numbers, strings, bytes

- Steps for composing records, arrays, sequences

- Invertible - encoding and decoding are symmetric

- Type safe

### Overview

The main abstractions are:

- `Step` - encodes `forward` and decodes `backward` between a source and target type.

- `Binarystep` - a `Step` for encoding/decoding binary data.

To work with binary data of primitive types, such as: numbers, strings, bytes etc..., you can use the the `BinaryPureStep`s inside the file [`binary_primitive_steps.ts`](/src/binary_primitive_steps.ts).

- `BinaryNumberStep` - supports signed/unsigned integers of 8, 16, 32 and 64 bits, and floats of 32 and 64 bits, in both little-endian and big-endian formats.

- `BinaryStringStep` - for strings with varying lengths (based on the `args.length` parameter).

- `BinaryCStringStep` - for NUL (`"\x00"`) terminated strings.

- `BinaryBytesStep` - for sequence of bytes (based on the `args.length` parameter).

- `BinaryNumberArrayStep` - for numeric array of numbers (based on the `args.length` parameter).

To work with binary data of composite types, such as: records, arrays, sequences etc..., you can use the the `BinaryPureStep`s inside the file [`binary_composition_steps`](/src/binary_composition_steps.ts).

- `BinaryRecordStep` - encodes a record with named fields, when provided with a sequence of `Steps` for each field.

- `BinaryArrayStep` - encodes an array of items, when provided with a single `Step` for the item type.

- `SequentialSteps` - chains multiple `Step`s.

For example, here is a step that decode and then repack PNG image file's metadata blocks:

```ts

class PNG_Codec_Step extends PureStep {

	forward(input: Uint8Array): GeneralChunk_schema[] {

		// parse input into chunks

	}

	backward(input: GeneralChunk_schema[]): Uint8Array {

		// encode chunks into binary

	}

}

```

## Theory

The key components that enable building robust bidirectional transformations are:

- `Step`

  - Defines symmetric forward and backward methods between source `FROM` and target `TO` types.

  Any lost information should be stored in the `LOST` private property.

- `PureStep extends Step`

  - This is a `Step` that is guaranteed to not lose information during forward transformation, allowing backward transformation to fully reconstruct the original value.

It also means that one may reuse a single instance of this kind of step for multiple forward and backward transformations, in whatever order it is deemed convenient.

Hence, these kind of steps are suitable for being applied repeatdly by larger compositional steps.

- `BinaryStep extends Step<{ bin: Uint8Array, pos: number, args: ARGS }, { val: OUT, len: number }, LOST>`

  - A step that specialized for parsing binary data.

backward encodes the wrapped `OUT` value back into a wrapped `Uint8Array` object.

- `BinaryPureStep extends PureStep<{ bin: Uint8Array, pos: number, args: ARGS }>`

  - Similar to `BinaryStep`, but is guaranteed not to lose data during forward transformation.

This is the class that is basically used by all implementations of `BinaryStep`s.

These simple interfaces allow `Step`s to be composed in a type-safe way.

Moreover, the compositions form a tree structure that preserves invertibility - as long as each `Step` inside implements their `forward` and `backward` methods correctly, the entire composition will be robustly reversible.

It also makes diagnosing issues easier - if there is a decoding error, it must be within one of the composed Steps.

The composition tree narrows down where the issue lies.