{"id":20770496,"url":"https://github.com/ably/delta-codec-js","last_synced_at":"2025-04-30T13:49:45.474Z","repository":{"id":39763110,"uuid":"210660558","full_name":"ably/delta-codec-js","owner":"ably","description":null,"archived":false,"fork":false,"pushed_at":"2023-07-08T10:09:11.000Z","size":502,"stargazers_count":3,"open_issues_count":18,"forks_count":3,"subscribers_count":23,"default_branch":"main","last_synced_at":"2025-04-20T10:41:04.253Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ably.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-24T17:30:03.000Z","updated_at":"2024-01-10T00:51:50.000Z","dependencies_parsed_at":"2024-11-10T02:03:37.711Z","dependency_job_id":null,"html_url":"https://github.com/ably/delta-codec-js","commit_stats":{"total_commits":52,"total_committers":6,"mean_commits":8.666666666666666,"dds":0.6538461538461539,"last_synced_commit":"a38542525b03a6caa9782c648d17836cf2d88157"},"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably%2Fdelta-codec-js","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably%2Fdelta-codec-js/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably%2Fdelta-codec-js/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably%2Fdelta-codec-js/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ably","download_url":"https://codeload.github.com/ably/delta-codec-js/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251714664,"owners_count":21631769,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-17T12:09:54.120Z","updated_at":"2025-04-30T13:49:45.433Z","avatar_url":"https://github.com/ably.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Vcdiff Codec Library for JavaScript\n\n[![Build Status](https://travis-ci.org/ably/delta-codec-js.svg?branch=main)](https://travis-ci.org/ably/delta-codec-js)\n[![npm version](https://badge.fury.io/js/%40ably%2Fdelta-codec.svg)](https://badge.fury.io/js/%40ably%2Fvcdiff-decoder)\n\nA Vcdiff codec library supporting the Vcdiff delta format, as defined by\n[RFC 3284](https://tools.ietf.org/html/rfc3284).\nThis library supports developers who need to consume delta streams from Ably without using the official JavaScript client library (e.g. for\n[MQTT](https://www.ably.io/concepts/mqtt)\nand\n[SSE](https://www.ably.io/concepts/server-sent-events)\napplications).\n\nThroughout this documentation, and within the code itself, we refer to a Vcdiff payload as a 'delta'.\nElsewhere such delta payloads may be referred to as patches or diffs, but for consistency within this repository we stick to the terms 'delta' and 'deltas'.\n\n## Installation from npm for Node.js\n\n    npm install @ably/delta-codec\n\nand require as:\n\n```js\nvar deltaCodec = require('@ably/delta-codec');\n```\n\n## Script include for Web Browsers\n\nInclude the library in your HTML from our CDN:\n\n```html\n\u003cscript src=\"https://cdn.ably.io/lib/delta-codec.min-1.js\"\u003e\u003c/script\u003e\n```\n\nWe follow [Semantic Versioning](http://semver.org/). To lock into a major or minor version of the client library, you can specify a specific version number - for example:\n\n* `https://cdn.ably.io/lib/delta-codec.min-1.js` for latest `1.*` version\n* `https://cdn.ably.io/lib/delta-codec.min-1.0.js` for latest `v1.0.*` version\n* `https://cdn.ably.io/lib/delta-codec.min-1.0.2.js` for version `1.0.2` explicitly\n\nYou can load the non-minified version by omitting `min-` from the URL, for example `https://cdn.ably.io/lib/delta-codec-1.js`.\n\nSee [tagged releases](https://github.com/ably/delta-codec-js/releases) for available versions.\n\n## Basic Stream Decoder\n\nThe `VcdiffDecoder` constructor provides the most basic entry point to the public API. It provides a stateful way of applying a stream of Vcdiff deltas, producing a new value after each delta has been applied to the previous value.\n\nFirst provide the base value, upon which the first delta will be applied using the 'setBase' method:\n\n```js\nlet decoder = new deltaCodec.VcdiffDecoder();\ndecoder.setBase(value);\n```\n\nOnce the decoder has been initialized like this, then each subsequent delta is applied using the `applyDelta` method:\n\n```js\nlet result = decoder.applyDelta(delta);\n// TODO call method on result to get the value format you require\n```\n\nThe `decoder` retains the current value, with the result of this method encapsulating that new value now that `delta` has been applied to the previous value.\nThe `result` of this method offers methods to allow you to access the new current value in the format you require:\n\n- `asUint8Array()`: a `Uint8Array`, being the new current value, as received (i.e. 'raw' bytes)\n- `asUtf8String()`: a `string`, decoded from data using UTF-8\n- `asObject()`: a JavaScript object, decoded from data using `JSON.parse`\n\n### `setBase(value)` Instance Method\n\nAccepts a single `value` argument, the 'base', which may be `ArrayBuffer`, `Uint8Array`, `Buffer` or `string`.\nIf a `string` is supplied then it will be UTF-8 encoded by the library before being stored as the current value.\nThis is freeform data, as specified by the application.\n\nThe decoder also exposes an alternative method, `setBase64Base(value)`, where the single `value` argument must be `string` and is Base64 decoded by the library before being stored as the current value.\n\n### `applyDelta(delta)` Instance Method\n\nAccepts a single `delta` argument which may be `ArrayBuffer`, `Uint8Array` or `Buffer`.\nThis is a Vcdiff format delta.\n\nThe decoder also exposes an alternative method, `applyBase64Delta(delta)`, where the single `delta` argument must be `string` and is Base64 decoded by the library before being applied as a Vcdiff format delta to the current value.\n\n### `isDelta(data)` Static Method\n\nAccepts a single `data` argument which may be `ArrayBuffer`, `Uint8Array` or `Buffer`.\nReturns `true` only if `data` has a Vcdiff delta header.\n\nThis method can be used on receipt of a binary payload to detect whether it should be interpreted as an absolute value or as a delta to be applied to the previous value.\nSuch 'sniffing' should be avoided where there is metadata available alongside received payloads to indicate whether they are deltas or not, as is the case when receiving a stream of enveloped data from Ably over SSE (see example).\n\n## Checked Stream Decoder\n\nThe `CheckedVcdiffDecoder` is a variant of `VcdiffDecoder` that can be used when the values and the deltas applied to them have unique identifiers. The 'set' and 'apply' methods on the checked decoder have the same names but require additional arguments for these identifiers:\n\n- `applyDelta(delta, deltaId, baseId)`\n- `applyBase64Delta(delta, deltaId, baseId)`\n- `setBase(value, baseId)`\n- `setBase64Base(value, baseId)`\n\nThe `baseId` argument supplied to the 'set' methods and the `deltaId` arguments supplied to the 'apply' methods are stored alongside the current value and then compared to the `baseId` argument supplied in subsequent 'apply' calls.\nAn `Error` is thrown if there is a mismatch.\n\n## Example Use Cases\n\n### Node.js: Text stream from Ably via SSE (enveloped)\n\nBy default the event data received from Ably is enveloped in JSON format.\nWe decode this data and inspect the Ably formatted message contents in order to establish whether the data in this message is an absolute value or a delta to be applied to the previously received value.\n\n```js\nconst deltaCodec = require('@ably/delta-codec');\nconst EventSource = require('eventsource');\n\nconst prefix = '[?delta=vcdiff]';\nconst url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}\u0026v=1.2\u0026key=${APP_KEY}`;\nconst eventSource = new EventSource(url);\nconst decoder = new deltaCodec.CheckedVcdiffDecoder();\neventSource.onmessage = function onEventSourceMessage(event) {\n  const message = JSON.parse(event.data);\n  let value;\n  const deltaExtras = (message.extras \u0026\u0026 message.extras.delta) ? message.extras.delta : null;\n  if (deltaExtras) {\n    if (deltaExtras.format !== 'vcdiff') {\n      throw new Error(`Delta format ${deltaExtras.format} not understood.`);\n    }\n    value = decoder.applyBase64Delta(message.data, message.id, deltaExtras.from).asUtf8String();\n  } else {\n    value = message.data;\n    decoder.setBase(value, message.id);\n  }\n  console.log(`received: ${value}`);\n};\neventSource.onerror = function onEventSourceError(event) {\n  console.log(`error: ${event.data}`);\n};\n```\n\n### Node.js: Text stream from Ably via SSE (not enveloped)\n\nFor this example we have subscribed to Ably as our event source and specified that we do not want the inbound event data to be JSON enveloped.\nWithout envelopes the events will be smaller, taking up less transmission bandwidth, however this then means we need 'sniff' each inbound event's data to identify whether it is an absolute value or a delta to be applied to the previously received value.\n\nAbsolute values are sent to us as strings, ready to use. Deltas are sent to us as Base64 encoded binary.\n\n```js\nconst deltaCodec = require('@ably/delta-codec');\nconst EventSource = require('eventsource');\n\nconst prefix = '[?delta=vcdiff]';\nconst url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}\u0026v=1.2\u0026key=${APP_KEY}\u0026enveloped=false`;\nconst eventSource = new EventSource(url);\nconst decoder = new deltaCodec.VcdiffDecoder();\neventSource.onmessage = function onEventSourceMessage(event) {\n  const stringData = event.data;\n  let value;\n  if (deltaCodec.VcdiffDecoder.isBase64Delta(stringData)) {\n    value = decoder.applyBase64Delta(stringData).asUtf8String();\n  } else {\n    value = stringData;\n    decoder.setBase(value);\n  }\n  console.log(`received: ${value}`);\n};\neventSource.onerror = function onEventSourceError(event) {\n  console.log(`error: ${event.data}`);\n};\n```\n\n### Node.js: Binary stream from Ably via MQTT\n\nThe raw binary data received over MQTT has no encoding or other form of envelope encapsulating it.\nWe need to 'sniff' each inbound payload to identify whether it is an absolute value or a delta to be applied to the previously received value.\nIn this example we are transporting UTF-8 encoded strings.\n\n```js\nconst deltaCodec = require('@ably/delta-codec');\nconst mqtt = require('mqtt');\n\nconst brokerUrl = `mqtts://mqtt.ably.io`;\nconst options = {\n  username: APP_KEY_NAME,\n  password: APP_KEY_SECRET,\n};\nconst prefix = '[?delta=vcdiff]';\nconst client = mqtt.connect(brokerUrl, options);\nclient.on('connect', () =\u003e {\n  client.subscribe(`${prefix}${CHANNEL_NAME}`);\n});\nconst decoder = new deltaCodec.VcdiffDecoder();\nclient.on('message', (topic, message) =\u003e {\n  let value;\n  if (deltaCodec.VcdiffDecoder.isDelta(message)) {\n    value = decoder.applyDelta(message).asUtf8String();\n  } else {\n    decoder.setBase(message);\n    value = message.toString();\n  }\n  console.log(`received: ${value}`);\n});\n```\n\n## Contributing\n\n### Building\n\nYou can trigger a build using Webpack with:\n\n    npm run grunt -- build\n\nwhich creates `delta-codec.js` and `delta-codec.min.js` in the `dist` folder.\n\n### Testing\n\nTo run tests in all runtimes (Node and browsers):\n\n    npm test\n\nTo run tests on a single runtime:\n\n- Node (very quick): `npm run grunt -- test:node`\n- Local browser (Firefox): `npm run grunt -- test:browser:local`\n- Remote browsers (Safari, Firefox, Chrome, IE, Edge, Chrome Mobile and Mobile Safari): `npm run grunt -- test:browser:remote`\n\n**Known Issue:**\nWhen testing in a local browser either using `npm run grunt -- test:browser:local` or indirectly by using `npm test` on macOS, you may see a \"segmentation fault\". Launch the Console app to find the associated crash reports, of which there will be too. More information in [issue #7](https://github.com/ably/delta-codec-js/issues/7).\n\nRemote browser testing supported by\n\n[\u003cimg src=\"./resources/Browserstack-logo@2x.png\" width=\"200px\"\u003e\u003c/img\u003e](https://www.browserstack.com/)\n\nfor which you will need to configure environment variables for `BROWSERSTACK_USERNAME` and `BROWSERSTACK_ACCESSKEY`.\n\n## Release Procedure\n\nOn the `main` branch:\n\n1. Increment the version, regenerate from source (a.k.a. build / bundle) and make a tagged commit which includes the built output from the `/dist` folder by running `npm run grunt -- release:patch` (or \"major\", \"minor\" or \"prepatch\" as appropriate - see [grunt-bump Usage Examples](https://github.com/vojtajina/grunt-bump#usage-examples))\n2. Release the tagged commit to Github using `git push origin main --follow-tags`\n3. Release to NPM using `npm publish . --access public` ([this package](https://www.npmjs.com/package/@ably/delta-codec) is configured to require that [2FA](https://docs.npmjs.com/configuring-two-factor-authentication) is used by publishers)\n4. Release to Ably's CDN using `npm run grunt -- publish-cdn` (operable by Ably staff only)\n5. Visit [tags](https://github.com/ably/delta-codec-js/tags) and draft new release for the newly created tag\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fably%2Fdelta-codec-js","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fably%2Fdelta-codec-js","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fably%2Fdelta-codec-js/lists"}