{"id":20145753,"url":"https://github.com/obsius/typed-binary-json","last_synced_at":"2025-04-09T19:18:31.436Z","repository":{"id":57383325,"uuid":"177037347","full_name":"obsius/typed-binary-json","owner":"obsius","description":"A Typed Binary JSON serializer and parser for JS.","archived":false,"fork":false,"pushed_at":"2024-12-03T02:52:25.000Z","size":585,"stargazers_count":11,"open_issues_count":2,"forks_count":1,"subscribers_count":0,"default_branch":"master","last_synced_at":"2025-03-23T21:11:22.372Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/obsius.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-03-21T23:11:56.000Z","updated_at":"2024-12-03T02:52:28.000Z","dependencies_parsed_at":"2024-06-19T05:29:21.957Z","dependency_job_id":"d48a63e8-d74a-435d-ad2f-475fc19a1543","html_url":"https://github.com/obsius/typed-binary-json","commit_stats":{"total_commits":98,"total_committers":1,"mean_commits":98.0,"dds":0.0,"last_synced_commit":"b3822ab6f740fb57cf9946dc0e0d031caf8b7fb2"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsius%2Ftyped-binary-json","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsius%2Ftyped-binary-json/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsius%2Ftyped-binary-json/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/obsius%2Ftyped-binary-json/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/obsius","download_url":"https://codeload.github.com/obsius/typed-binary-json/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247601480,"owners_count":20964863,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-13T22:18:01.335Z","updated_at":"2025-04-09T19:18:31.415Z","avatar_url":"https://github.com/obsius.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Typed Binary JSON\n\nTyped Binary JSON or TBJSON, is a binary serialization format that is compatible with JSON.\nIt stores known object prototypes in a JSON header, and serializes the data in a binary format following the header.  \n  \nTBJSON is useful for serializing known objects, classes, or types, otherwise it will offer little advantage if any in terms of size or performance over JSON.  \n\nFor a browser compatible version of this package, use [TBJSON in the Browser](https://www.npmjs.com/package/typed-binary-json-browser).\n\n## Format\n\n### File Format\n\nEach file starts off with `.tbj` to singinify that it is a `Typed Binary JSON` file, followed by a `unit32` which is the length of the header.\n```\n         length of header                 raw binary data\n   .tbj                header in JSON             \n.  t  b  j  [ uint32 ]  {  .  .  .  }  .  .  d  a  t  a  .  .\n0  1  2  3  4  5  6  7  8  9  0  1  2  3  4  5  6  7  8  9  0\n0                             10                            20\n```\n\nOffset | Value  | Meaning\n---    | ---    | ---\n0      | .tbj   | States the file type.\n4      | uint32 | Size of the JSON header.\n8      | JSON   | A utf-8 serialized JSON map of the binary data to follow.\nx      | binary | The binary data. Always the next byte after the last header byte.\n\n### Header Format\n\nThe header contains information necessary to parse the binary data. It is raw `JSON` and makes it easy to peak at the file and see how the data is structured.\n\nEntry     | Meaning\n---       | ---\ntypeRefs  | A map that translates known type names to their codes.\ntypes     | Custom primitive types that have been defined for this serialization.\nprotoRefs | A map that translates known class and object names (either passed in or the object's constructor name) to their codes.\nprotos    | Definitions for known prototypes or classes that are referenced in the root definition.\nobjs      | Definitions for unknown objects that are referenced in known prototypes.\nroot      | The object that was serialized. Contains the definition needed to decode the binary format.\n\n### Types\n\nThe types used by TBJSON.\n\nType          | Code | Definition\n---           | ---  | ---\nPrimitives    | -    | -\nNULL          | 0    | Null value.\nBOOL          | 1    | Boolean.\nUINT8         | 2    | 8 bit unsigned integer.\nINT8          | 3    | 8 bit signed integer.\nUINT16        | 4    | 16 bit unsigned integer.\nINT16         | 5    | 16 bit signed integer.\nUINT32        | 6    | 32 bit unsigned integer.\nINT32         | 7    | 32 bit signed integer.\nFLOAT32       | 8    | 32 bit floating point.\nFLOAT64       | 9    | 64 bit double precision floating point.\nComplex Types | -    | -\nSTRING        | 10   | String.\nARRAY         | 11   | Array. Used as `Tbjson.TYPES.ARRAY` or `[Tbjson.TYPES.ARRAY, \u003cTYPE\u003e]`. Like: `[Tbjson.TYPES.ARRAY, Tbjson.TYPES.FLOAT32]`.\nOBJECT        | 12   | Object. Used as `Tbjson.TYPES.OBJECT` or `[Tbjson.TYPES.OBJECT, \u003cTYPE\u003e]` if all the values in the object are the same type. Like: `[Tbjson.TYPES.OBJECT, MyClass]`.\nNULLABLE      | 13   | Nullable value. Used as `[Tbjson.TYPES.NULLABLE, \u003cTYPE\u003e]`. Like: `[Tbjson.TYPES.NULLABLE, Tbjson.TYPES.STRING]`.\nTYPED_ARRAY   | 14   | Typed array. Used as `Float32Array` or `Int16Array`. Used like `[Tbjson.TYPES.TYPED_ARRAY, \u003cTYPE\u003e`. Like: `[Tbjson.TYPES.TYPED_ARRAY, Tbjson.TYPES.INT32]`.\nUNKNOWN       | 15   | Unknown type. Wildcard that can represent a JS number, boolean, or string.\nExtras        | -    | -\nVARIABLE_DEF  | 16   | A variable definition. That is a definition that is not yet known, but will be known just before serialization. Registered by `tbjson.registerVariableDef('var1', { ... })`. Used as `Tbjson.TYPES.VARIABLE_DEF`.\nINSTANCE      | 17   | An instance of a class. Useful for subclasses. Like `[Tbjson.TYPES.INSTANCE, MySuperClass]`.\n\n### Reference\n\n```js\n// use an import\nimport Tbjson from 'typed-binary-json';\n// or require\nconst Tbjson = require('typed-binary-json');\n\n// make a new instance\nlet tbjson = new Tbjson();\n\n// serialize a plain object to a buffer\nlet serializedToBuffer = tbjson.serializeToBuffer({ a: 'a', b: 1, c: true });\n\n// buffer looks like:\n//\n// byte offset: data\n//\n// 000: .tbj\n// 004: (uint32)12\n// 008: {\n//        \"version\": 1,\n//        \"offsets\": {\n//          \"prototype\": 64,\n//          \"nullablePrototype\": 256,\n//          \"array\": 512,\n//          \"object\": 4096\n//        },\n//        \"typeRefs\": {},\n//        \"typeDefs\": {},\n//        \"protoRefs\": {},\n//        \"protoDefs\": {},\n//        \"objs\": {},\n//        \"root\": {\n//          \"a\": 10,\n//          \"b\": 9,\n//          \"c\": 1\n//        }\n//      }\n// 194: binary data\n\n// parse a buffer (deserialize)\ntbjson.parseBuffer(serializedToBuffer);\n\nclass Test {\n\tconstructor() {\n\t\tthis.x = [0, 1, 2, 3, 4, 5, 6, 7, 8];\n\t}\n}\nTest.tbjson = {\n\tdefinition: {\n\t\tx: [Tbjson.TYPES.ARRAY, Tbjson.TYPES.UINT32]\n\t}\n}; \n\n// serialize a prototyped object to a buffer\nserializedToBuffer = tbjson.serializeToBuffer(new Test());\n\n// buffer looks like:\n//\n// byte offset: data\n//\n// 000: .tbj\n// 004: (uint32)12\n// 008: {\n//        \"version\": 1,\n//        \"offsets\": {\n//          \"prototype\": 64,\n//          \"nullablePrototype\": 256,\n//          \"array\": 512,\n//          \"object\": 4096\n//        },\n//        \"typeRefs\": {},\n//        \"typeDefs\": {},\n//        \"protoRefs\": {\n//          \"Test\": 64\n//        },\n//        \"protoDefs\": {\n//          \"64\": {\n//            \"x\": 518\n//          }\n//        },\n//        \"objs\": {},\n//        \"root\": 64\n//      }\n// 199: binary data\n\n```\n\n### Working Example\n\nRefer to the `test` dir to see all possible examples.\n\n```js\nimport Tbjson from 'typed-binary-json';\n\nclass A {\n\tx = 0;\n\ty = 0;\n\tz = 0;\n}\n\n// make A a known prototype\nA.tbjson = {\n\tdefinition: {\n\t\tx: Tbjson.TYPES.FLOAT32,\n\t\ty: Tbjson.TYPES.FLOAT32,\n\t\tz: Tbjson.TYPES.FLOAT32,\n\t}\n};\n\nclass B {\n\tas = [new A()];\n\tstring = 'string';\n\tbool = false;\n\tnumber = 100.5;\n}\n\n// make B a known prototype\nB.tbjson = {\n\tdefinition: {\n\t\t// use the [ array, type ] notation to say that \"B.as\" is an array of A\n\t\tas: [Tbjson.TYPES.ARRAY, A],\n\t\tstring: Tbjson.TYPES.STRING,\n\t\tbool: Tbjson.TYPES.BOOL,\n\t\tnumber: Tbjson.TYPES.FLOAT64\n\t}\n}\n\n// make a root object (untyped)\nlet root = {\n\tb: new B()\n};\n\n(async function() {\n\n\tlet tbjson = new Tbjson();\n\n\t// serialize to a file\n\tawait tbjson.serializeToFile('test.tbj', root);\n\n\t// parse from a file\n\tlet obj = tbjson.parseFileAsBuffer('test.tbj');\n\n\tconsole.log(obj);\n})();\n```\n\n## Methods\n\n### Serialization\n\n**serializeToBuffer(obj)**\n\nSerialize `obj`. Create a buffer and write to it.\n\n**serializeToStream(stream, obj)**\n\nSerialize `obj`. Write to `stream`.\n\n**serializeToFile(filename, obj)**\n\nSerialize `obj`. Create a write stream for `filename` and write out to the file stream.\n\n### Parsing\n\n**parseBuffer(buffer)**\n\nParse the `buffer`. Return the parsed object.\n\n**parseStream(stream)**\n\nParse the `stream`. Return the parsed object.\n\n**parseFileAsBuffer(filename)**\n\nRead the file `filename` into memory and parse its conents. Preferred for performance. Return the parsed object.\n\n**parseFileAsStream(filename)**\n\nCreate a read stream for `filename` and parse its contents. Useful for very large files, but slower. Return the parsed object.\n\n### Registrations\n\nMost of these functions are not necessary to call if `tbjson` is set on the class level.\n\n**finalizePrototypes()**\n\n*See `test/inheritance` for an example.*\n\nMust be called if inheritance or referenced definitions are used or registered.\n\n```js\nlet tbson = new Tbjson();\ntbjson.registerPrototype({\n\treference: 'X',\n\tdefinition: {\n\t\tx: Tbjson.TYPES.STRING\n\t}\n});\n```\n\n**registerPrototype(obj)**\n\n*See `test/type` and `test/inheritance` for examples.*\n\n*Not needed if tbjson is set statically on a class.*\n\nRegister a prototype. `obj` is the definition for the prototype.\n\n```js\nlet tbson = new Tbjson();\ntbjson.registerPrototype({\n\tprototype: X,\n\tdefinition: {\n\t\tx: Tbjson.TYPES.STRING\n\t}\n});\n```\n\n**registerPrototypes(array)**\n\nRegister an array of prototypes. `array` is an array of prototypes.\n\n**registerPseudoPrototype(id, def)**\n\nRegister a pseudo prototype. That is a plain (non-prototyped) object that will have a known structure just before serialization.\n`id` is a number of string to identify the definition. `ref` is the definition. Can be qualified, like marked `nullable`.\n\n```js\nlet tbjson = new Tbjson();\ntbjson.registerPseudoPrototype('x', {\n\tx: Tbjson.TYPES.STRING\n});\n\nclass X {\n\tconstructor() {\n\t\tthis.x = null;\n\t}\n}\nX.tbjson = {\n\tdefinition: {\n\t\tx: [Tbjson.TYPES.NULLABLE, 'x']\n\t}\n};\n```\n\n**registerVariableDef(id, def)**\n\nRegister a variable definition. That is a plain (non-prototyped) object that will have a known structure just before serialization. `id` is a number or string to identify the definition. `ref` is the definition. Unlike a pseduo prototype, this cannot be qualified - like marked `nullable`.\n\n```js\nlet tbjson = new Tbjson();\ntbjson.registerVariableDef('x', {\n\tx: Tbjson.TYPES.STRING\n});\n\nclass X {\n\tconstructor() {\n\t\tthis.x = {\n\t\t\tx: 'x'\n\t\t};\n\t}\n}\nX.tbjson = {\n\tdefinition: {\n\t\tx: [Tbjson.TYPES.VARIABLE_DEF, 'x']\n\t}\n};\n```\n\n**registerType(type)**\n\nNot available yet. Register a custom type (a primitive like `int48`, etc...). `type` is the definition for the custom type.\n\n### Static\n\n**cast(obj, prototype)**\n\n*See `test/cast` and `test/protoypeCast` for examples.*\n\nCast the given `obj` as `prototype`.\n\n```js\nclass X {}\nX.tbjson = {\n\tdefinition: {\n\t\tx: Tbjson.TYPES.STRING\n\t}\n};\nTbjson.cast({ x: 'x' }, X);\n```\n\n**clone(obj)**\n\n*See `test/clone` for an example.*\n\nClone the `obj` into a prototyped object ignoring typing rules, but obeying which properties should be ignored.\n\n```js\nclass X {}\nlet cloneX = Tbjson.clone(x);\n```\n\n**definition(obj)**\n\n*See `test/definition` for an example.*\n\nHelper function to extract the definition (including parent prototypes) of `obj`.\n\n```js\nclass Y {}\ny.tbjson = {\n\tdefinition: {\n\t\ta: Tbjson.TYPES.STRING\n\t}\n};\nclass X extends Y {}\nX.tbjson = {\n\tdefinition: {\n\t\tb: Tbjson.TYPES.STRING\n\t}\n};\n\nTbjson.definition(x);\n// { a: 'a', b: 'b' }\n```\n\n**serialize(obj)**\n\n*See `test/serialize` for an example.*\n\nSerialize `obj` into a plain object ignoring typings, but obeying which properties should be ignored.\n\n```js\nclass X {\n\tconstriuctor() {\n\t\tthis.a = 'a';\n\t\tthis.b = 'b';\n\t}\n}\nX.tbjson = {\n\tdefinition: {\n\t\ta: Tbjson.TYPES.STRING\n\t}\n};\n\nlet x = new X();\n\nTbjson.serialize(x);\n// x.b is ignored because it is not part of the tbjson definition\n// {  a: 'a' }\n```\n\n## Performance\n\nPerformance varies on the data type, but you'll get best performance if your types have lots of numeric values, and even better performance if you can take advantage of `float32`, `int32`, `int16`, and `int8` to save space.  \n  \n100 of `root.first`  \n10K per each `root.first` of `first.second`\n\n```json\n{\n\t\"root\": {\n\t\t\"first\": [{\n\t\t\t\"second\": [{\n\t\t\t\t\"x\": 100000.666666666666,\n\t\t\t\t\"y\": -999999.999,\n\t\t\t\t\"z\": 1234.5678901234,\n\t\t\t\t\"details\": {\n\t\t\t\t\t\"alpha\": \"oranges\",\n\t\t\t\t\t\"beta\": 10,\n\t\t\t\t\t\"gamma\": [-3.14159, false, true, \"!@#$%^\u0026*()\"]\n\t\t\t\t}\n\t\t\t}],\n\t\t\t\"anotherString\": \"apples\",\n\t\t\t\"number\": 86,\n\t\t\t\"bool\": true,\n\t\t\t\"array\": [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]\n\t\t}]\n\t}\n}\n```\n\nBenchmark    | Filesize | Time\n---          | ---      | ---\nJSON Write   | 140 MB   | 2,648 ms\nTBJSON Write | 37 MB    | 1,154 ms\nJSON Read    | N/A      | 2,073 ms\nTBJSON Read  | N/A      | 1,453 ms\n\n## Contributing\n\nFeel free to make changes and submit pull requests whenever.\n\n## License\n\nTyped Binary JSON uses the [MIT](https://opensource.org/licenses/MIT) license.","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobsius%2Ftyped-binary-json","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fobsius%2Ftyped-binary-json","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fobsius%2Ftyped-binary-json/lists"}