{"id":19319235,"url":"https://github.com/oramasearch/seqproto","last_synced_at":"2025-04-22T17:31:32.325Z","repository":{"id":206047152,"uuid":"715695029","full_name":"oramasearch/seqproto","owner":"oramasearch","description":"Extremely fast, compact, binary serialization/deserialization for your structured data","archived":false,"fork":false,"pushed_at":"2024-03-27T16:12:07.000Z","size":476,"stargazers_count":104,"open_issues_count":1,"forks_count":3,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-10-18T18:11:23.001Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oramasearch.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-11-07T16:41:37.000Z","updated_at":"2024-07-24T17:05:28.000Z","dependencies_parsed_at":"2024-10-18T18:11:34.849Z","dependency_job_id":"2733ec46-c50d-49db-be3b-4813d60fd7da","html_url":"https://github.com/oramasearch/seqproto","commit_stats":null,"previous_names":["oramasearch/seqproto","askorama/seqproto"],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oramasearch%2Fseqproto","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oramasearch%2Fseqproto/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oramasearch%2Fseqproto/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oramasearch%2Fseqproto/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oramasearch","download_url":"https://codeload.github.com/oramasearch/seqproto/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":223901852,"owners_count":17222259,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T01:22:48.484Z","updated_at":"2024-11-10T01:22:48.987Z","avatar_url":"https://github.com/oramasearch.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SeqProto\n\n[![Nodejs](https://github.com/oramasearch/seqproto/actions/workflows/nodejs.yml/badge.svg)](https://github.com/oramasearch/seqproto/actions/workflows/nodejs.yml)\n![npm package minimized gzipped size (select exports)](https://img.shields.io/bundlejs/size/seqproto)\n\nThis library provides a simple way to serialize and deserialize objects in binary format.\n\n## Why another serialization library?\n\nWhile I have been writing this library, I have in mind the following main goals:\n- **Runtime independent** - I want to have a library that runs in every javascript runtime, from Node.JS, through browsers, to CloudFlare.\n- **Performance** - I want to have a library that is fast and easy to use.\n- **Small size** - I want to have a library that is small and easy to use.\n- **Customizable** - Due to the JavaScript nature, the data structures are limited.\n- **TypeScript support** - I want to have a library that is easy to use in TypeScript.\n\n## Installation\n\nSeqproto works in any JavaScript environment. You can install it via npm:\n\n```sh\nnpm install seqproto\n```\n\nOr via CDN:\n\n```js\nimport { createSer, createDes } from 'https://unpkg.com/seqproto@latest/dist/esm/index.js'\n```\n\n## Usage\n\n### Examples\n\nFor more examples, see the [examples](https://github.com/oramasearch/seqproto/tree/main/examples) directory.\n\n```typescript\nimport { createSer, createDes } from 'seqproto'\n\n// Create a serializer\nconst ser = createSer()\n\n// Serialize some data\nser.serializeBoolean(true)\nser.serializeUInt32(42)\nser.serializeFloat32(-0.5)\nser.serializeString('hello world')\nser.serializeArray([1, 2, 3], (ser, n) =\u003e ser.serializeUInt32(n))\n\n// Get ArrayBuffer with serialized data\nconst buffer = ser.getBuffer()\n\n// Create a deserializer\nconst des = createDes(buffer)\n\n// Deserialize data\nconst b = des.deserializeBoolean()\nconst i = des.deserializeUInt32()\nconst f = des.deserializeFloat32()\nconst s = des.deserializeString()\nconst a = des.deserializeArray((des) =\u003e des.deserializeUInt32())\n\nconsole.log({ b, i, f, s, a })\n```\n\n### Object\n\n```typescript\nimport type { Ser, Des } from 'seqproto'\nimport { createSer, createDes } from 'seqproto'\n\ninterface Todo {\n  id: number\n  userId: number\n  title: string\n  completed: boolean\n}\n\nfunction serializeTodo (ser: Ser, todo: Todo) {\n  ser.serializeUInt32(todo.id)\n  ser.serializeUInt32(todo.userId)\n  ser.serializeString(todo.title)\n  ser.serializeBoolean(todo.completed)\n}\n\nfunction deserializeTodo (des: Des): Todo {\n  const id = des.deserializeUInt32()\n  const userId = des.deserializeUInt32()\n  const title = des.deserializeString()\n  const completed = des.deserializeBoolean()\n  return { id, userId, title, completed }\n}\n\nconst ser: Ser = createSer()\n\nserializeTodo(ser, {\n  id: 1,\n  userId: 1,\n  title: 'hello',\n  completed: false,\n})\n\nconst buffer = ser.getBuffer()\n\nconst des: Des = createDes(buffer)\nconst todo = deserializeTodo(des)\n\nconsole.log(JSON.stringify(todo, null, 2))\n```\n\n### Array of object\n\n```typescript\nimport type { Ser, Des } from 'seqproto'\nimport { createSer, createDes } from 'seqproto'\n\nlet buffer\n\n// Serialize\nconst todos = [\n  { userId: 1, id: 1, completed: false, title: \"delectus aut autem\" },\n  { userId: 1, id: 2, completed: true, title: \"quis ut nam facilis et officia qui\" }\n]\n\nconst ser: Ser = createSer()\n\nser.serializeArray(todos, (ser, todo) =\u003e {\n  ser.serializeUInt32(todo.id)\n  ser.serializeUInt32(todo.userId)\n  ser.serializeString(todo.title)\n  ser.serializeBoolean(todo.completed)\n})\n\nbuffer = ser.getBuffer()\n\n// Deserialize\nconst des: Des = createDes(buffer)\n\nconst deserializedTodos = des.deserializeArray((des) =\u003e {\n  const id = des.deserializeUInt32()\n  const userId = des.deserializeUInt32()\n  const title = des.deserializeString()\n  const completed = des.deserializeBoolean()\n  return { id, userId, title, completed }\n})\n\nconsole.log(deserializedTodos)\n```\n\n## API\n\nThis library exports the following functions:\n\n- `createSer()`/`createSer({ bufferSize: number })`: creates a new serializer.\n- `createDes(buffer)`: creates a new deserializer.\n- `ser.reset()`: reset the serializer.\n- `ser.serializeBoolean(b)`: serializes a boolean value.\n- `des.deserializeBoolean()`: deserializes a boolean value.\n- `ser.serializeUInt32(uint32)`: serializes a 32-bit unsigned integer.\n- `des.deserializeUInt32(uint32)`: deserializes a 32-bit unsigned integer.\n- `ser.serializeNumber(n)`: serializes a 32-bit unsigned integer or signed integer or float.\n- `des.deserializeNumber(n)`: deserializes a 32-bit unsigned integer or signed integer or float.\n- `ser.serializeString(string)`: serializes a string.\n- `des.deserializeString()`: deserializes a string.\n- `ser.serializeArray(array, (ser, item) =\u003e { ... })`: serializes an array.\n- `des.deserializeArray((des) =\u003e { ... })`: deserializes an array.\n- `ser.serializeIterable(iterable, (ser, item) =\u003e { ... })`: serializes an iterable.\n- `des.deserializeIterable((des) =\u003e { ... })`: deserializes an iterable.\n- `ser.serializeFloat32(float32)`: serializes float 32bit.\n- `des.deserializeFloat32()`: deserializes float 32bit.\n- `ser.getBuffer()`: returns the serialized buffer.\n\n## Benchmarks\n\nWe created 3 different benchmarks to compare the performance of SeqProto with JSON and Avro.\n\n### Isolated benchmark\n\nYou can run the benchmarks with the following command:\n```sh\nnpm run benchmark:serdes\n```\n\nSerialization / Deserialization:\n| name | ops | margin | percentSlower |\n| -------- | ------- | -------- | ------- |\n| seqproto | 29764 | 0.72 | 0 |\n| protobuf | 13698 | 0.19 | 53.98 |\n| avro | 24204 | 0.14 | 18.68 |\n| cbor | 803 | 0.22 | 97.3 |\n| cborx | 9707 | 0.32 | 67.39 |\n| msgpack | 6857 | 0.06 | 76.96 |\n| msgpackr | 10449 | 0.27 | 64.89 |\n| JSON | 14434 | 0.07 | 51.51 |\n\n### Http benchmark\n\nYou can run the benchmarks using 2 shells.\n\n1.\n```sh\ncd bechmarks/e2e\npnpm install\npnpm start\n```\n2.\n```sh\ncd bechmarks/e2e\npnpm run autocannon:json\npnpm run autocannon:seqproto\npnpm run autocannon:avro\n```\n\n| type     | req (in 10s) | Avg req/sec | Avg Bytes/Sec | Avg Latency (ms) |\n| -------- | ---- | --------- | ---- | ---- |\n| JSON     | 164k | 14892 | 275 | 0.11 |\n| SeqProto | 269k | 26865.6 | 321 | 0.01 |\n| Avro     | 197k | 17926.55 | 169 | 0.04 |\n\n### e2e benchmark\n\nYou can run the benchmarks with the following command:\n```sh\ncd bechmarks/e2e\npnpm install\npnpm start\n```\nAnd go to http://localhost:3000/public/index.html.\n\n| iteration    | parallelism | type | ms |\n| -------- | ------- | -------- | ------- |\n| 10 | 1 | JSON | 30.69999998807907 |\n| 10 | 1 | SeqProto | 25.600000023841858 |\n| 10 | 1 | Avro | 30.399999976158142 |\n| 100 | 1 | JSON | 108.80000001192093 |\n| 100 | 1 | SeqProto | 96.80000001192093 |\n| 100 | 1 | Avro | 96 |\n| 100 | 3 | JSON | 162.10000002384186 |\n| 100 | 3 | SeqProto | 152.4000000357628 |\n| 100 | 3 | Avro | 167.5 |\n| 100 | 6 | JSON | 277.19999998807907 |\n| 100 | 6 | SeqProto | 263.30000001192093 |\n| 100 | 6 | Avro | 308.19999998807907 |\n\n## Contributing\n\nContributions are welcome! Please open an issue if you have any ideas for improvement or found a bug.\n\n## License\n\nApache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foramasearch%2Fseqproto","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foramasearch%2Fseqproto","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foramasearch%2Fseqproto/lists"}