{"id":18532619,"url":"https://github.com/ethan7g/protocore","last_synced_at":"2025-04-09T14:31:37.677Z","repository":{"id":33830977,"uuid":"162858357","full_name":"ethan7g/protocore","owner":"ethan7g","description":"Specify and deploy performant binary protocol structures in Node","archived":false,"fork":false,"pushed_at":"2023-01-07T04:04:38.000Z","size":307,"stargazers_count":40,"open_issues_count":11,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-18T12:26:17.451Z","etag":null,"topics":["binary","binary-parser","binary-protocol","binary-schemas","binary-serialization","nodejs","protocol","protocol-library"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ethan7g.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2018-12-23T03:10:17.000Z","updated_at":"2024-10-31T13:53:05.000Z","dependencies_parsed_at":"2023-01-15T02:49:46.141Z","dependency_job_id":null,"html_url":"https://github.com/ethan7g/protocore","commit_stats":null,"previous_names":["ethanent/protocore"],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethan7g%2Fprotocore","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethan7g%2Fprotocore/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethan7g%2Fprotocore/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ethan7g%2Fprotocore/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ethan7g","download_url":"https://codeload.github.com/ethan7g/protocore/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248055142,"owners_count":21040126,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["binary","binary-parser","binary-protocol","binary-schemas","binary-serialization","nodejs","protocol","protocol-library"],"created_at":"2024-11-06T19:07:13.536Z","updated_at":"2025-04-09T14:31:37.344Z","avatar_url":"https://github.com/ethan7g.png","language":"JavaScript","readme":"# protocore\n\u003e Specify and deploy performant binary protocol structures in Node\n\n[GitHub](https://github.com/ethanent/protocore) | [NPM](https://www.npmjs.com/package/protocore)\n\n## Install\n\n```\nnpm i protocore\n```\n\nThen include the library in your code:\n\n```js\nconst {Schema, StreamingAbstractor, types, protospec} = require('protocore')\n```\n\n## What is Protocore?\n\nProtocore makes building custom binary protocols a snap.\n\nIt's a lightweight Node library that takes the pain out of making binary protocols for games, databases, and other performance-dependent applications!\n\nProtocore allows developers to create advanced protocols with powerful functionality and maximum efficiency.\n\n![Protocore schemas are much more efficient than JSON](https://raw.githubusercontent.com/ethanent/protocore/master/media/Bench-ComplexPerson.png)\n\n[See benchmarks -\u003e](https://github.com/ethanent/protocore/blob/master/bench.js)\n\n## Define a Schema\n\n```js\nconst personSchema = new Schema([\n\t{\n\t\t'name': 'firstName',\n\t\t'type': types.string\n\t},\n\t{\n\t\t'name': 'age',\n\t\t'type': types.uint,\n\t\t'size': 8\n\t},\n\t{\n\t\t'name': 'alive',\n\t\t'type': types.boolean\n\t}\n])\n```\n\nThe above code defines a simple schema which represents a person.\n\nIt includes a `firstName` string field and an `age`, which is a UInt8.\n\n## Build a Buffer from a Schema\n\n```js\nconst ethanBuf = personSchema.build({\n\t'name': 'Ethan Davis',\n\t'age': 17,\n\t'alive': true\n})\n\n// Now ethanBuf is a buffer representation of a person!\n```\n\nHere we've built a buffer from Ethan's data using the `personSchema` Schema. The `Schema.build` method returns a [Buffer](https://nodejs.org/api/buffer.html).\n\n## Parse a Buffer from a Schema\n\n```js\n// ^ Let's say ethanBuf is a buffer created with the personSchema Schema\n\nconst parsed = personSchema.parse(ethanBuf)\n\n// parsed will now be an object with the original information about Ethan!\n// parsed = {'name': 'Ethan Davis', 'age': 17, 'alive': true}\n```\n\nAbove a buffer was parsed using `personSchema`, which returned an object representation of the data!\n\n## Lists and Maps in Schemas\n\nLists and Maps can be defined in schemas as well.\n\n```js\nconst citySchema = new Schema([\n\t{\n\t\t'name': 'name',\n\t\t'type': types.string\n\t},\n\t{\n\t\t'name': 'buildings',\n\t\t'type': types.list,\n\t\t'of': new Schema([\n\t\t\t{\n\t\t\t\t'name': 'name',\n\t\t\t\t'type': types.string\n\t\t\t},\n\t\t\t{\n\t\t\t\t'name': 'constructed',\n\t\t\t\t'type': types.uint,\n\t\t\t\t'size': 16\n\t\t\t}\n\t\t])\n\t},\n\t{\n\t\t'name': 'residentLifetimes',\n\t\t'type': types.map,\n\t\t'key': {\n\t\t\t'type': types.string\n\t\t},\n\t\t'value': {\n\t\t\t'type': types.varint\n\t\t}\n\t}\n])\n```\n\nWe've now defined `citySchema`, which represents a city with buildings and residents. Buildings have names and also contain the year they were constructed. Residents' time spent in the city are represented in the `residentLifetimes` map.\n\n### Serializing Lists and Maps in Schemas\n\n```js\nconst sanFranciscoBuf = citySchema.build({\n\t'name': 'San Francisco',\n\t'buildings': [\n\t\t{\n\t\t\t'name': 'Salesforce Tower',\n\t\t\t'constructed': 2018\n\t\t},\n\t\t{\n\t\t\t'name': 'Ferry Building',\n\t\t\t'constructed': 1898\n\t\t}\n\t],\n\t'residentLifetimes': {\n\t\t'Ethan': 8,\n\t\t'James': 9,\n\t\t'Bohn': 12\n\t}\n})\n```\n\n### Parsing Lists and Maps in Schemas\n\n```js\nconst sanFrancisco = citySchema.parse(sanFranciscoBuf)\n```\n\n`sanFrancisco` will be similar to the object we built `sanFranciscoBuf` from. It will have an array of building objects. It will also have a map (object with appropriate types) for `residentLifetimes`.\n\n## Utilizing StreamingAbstractor\n\n`StreamingAbstractor`s allow us to create duplex, event-based streaming systems for applications.\n\nLet's create a `StreamingAbstractor`.\n\n```js\nconst myAbstractor = new StreamingAbstractor()\n\nmyAbstractor.register('login', new Schema([\n\t{\n\t\t'name': 'username',\n\t\t'type': types.string\n\t},\n\t{\n\t\t'name': 'number',\n\t\t'type': types.uint,\n\t\t'size': 16\n\t}\n]))\n\n// Now we can bind myAbstractor to a stream using myAbstractor.bind(stream)\n```\n\nAbove we've registered an event called 'login' in our abstractor. Now it can recieve login events from a stream connected to another `StreamingAbstractor`.\n\n### Recieving Events Through StreamingAbstractor\n\nNow that we have a `StreamingAbstractor` (`myAbstractor`) with the `login` event registered, we'll listen for `login` on our end.\n\n```js\nmyAbstractor.on('login', (data) =\u003e {\n\tconsole.log('Login with username ' + data.username + ' and number ' + data.number + '.')\n})\n```\n\n### Sending Events Through StreamingAbstractor\n\nBecause we've registered the `login` event, we can send `login` events using `myAbstractor`.\n\n```js\nmyAbstractor.send('login', {\n\t'username': 'ethan',\n\t'number': 5135\n})\n```\n\n## Creating Custom Types\n\nIt's possible to build custom types for Protocore schemas to use, and it's not too complex either.\n\nProtocore ships with its own built in types (ex. string, buffer, int, double, etc), and those are available for inspection in the [types directory](https://github.com/ethanent/protocore/tree/master/lib/types).\n\n## Writing Protocols with Protospec\n\nProtospec is Protocore's protocol specification format. It is nice to write.\n\n```\n// my.pspec\n\ndef player private\nstring username\nvarint score\nint x size=16\nint y size=16\n\ndef join\ninstance player of=player\n\ndef updateAllPlayers\nlist players of=player\n```\n\nTo import a Protospec as a `StreamingAbstractor`:\n\n```js\n// ... load spec, ex. fs.readFileSync(path.join(__dirname, 'my.pspec'))\n\nconst myAbstractor = protospec.importAbstractor(spec)\n\nmyAbstractor.on('updateAllPlayers', (data) =\u003e {\n\t// Do something with data.players\n})\n```\n\nTo import a Protospec as an `Object` of `Schema`s:\n\n```js\nconst mySchemas = protospec.importAll(spec).schemas\n\nconst builtJoin = mySchemas.join.build({\n\t'player': {\n\t\t'username': 'a',\n\t\t'score': 2,\n\t\t'x': 100,\n\t\t'y': 200\n\t}\n})\n```\n\n## Exchanges in a protocol\n\nProtocore has a built-in protocol feature called an `Exchange` which allows protocols to define transfers of information in a \"request-response\" manner.\n\n```\ndef ReqUser private\nstring username\n\ndef ResUser private\nstring username\nuint age size=8\naliasUses map key=string;value=varint\n\nexchange getUser\nrequest ReqUser\nresponse ResUser\n```\n\nNote that in this Protospec, the `ReqUser` and `ResUser` definitions are private. This means that they should not be directly sent over the `StreamingAbstractor`.\n\nServer handles requests:\n\n```js\nperClientAbstractor.on('getUser', (req, res) =\u003e {\n\tres({\n\t\t'username': req.username,\n\t\t'age': Math.floor(Math.random() * 100),\n\t\t'aliasUses': {\n\t\t\t'test': 5,\n\t\t\t'testB': 7\n\t\t}\n\t})\n})\n```\n\nClient sends requests:\n\n```js\nconst userRes = await clientAbstractor.request('getUser', {\n\t'username': 'ethan'\n})\n\nconsole.log(userRes)\n// =\u003e {'username': 'ethan', 'age': 18, 'aliasUses': {'test': 5, 'testB': 7}}\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethan7g%2Fprotocore","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fethan7g%2Fprotocore","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fethan7g%2Fprotocore/lists"}