{"id":13508425,"url":"https://github.com/JoshData/jot","last_synced_at":"2025-03-30T11:31:50.774Z","repository":{"id":9902875,"uuid":"11910387","full_name":"JoshData/jot","owner":"JoshData","description":"JSON Operational Transformation (JOT)","archived":false,"fork":false,"pushed_at":"2022-12-10T18:17:58.000Z","size":737,"stargazers_count":347,"open_issues_count":10,"forks_count":35,"subscribers_count":11,"default_branch":"primary","last_synced_at":"2024-04-13T21:57:38.418Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JoshData.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-08-05T22:32:43.000Z","updated_at":"2024-02-27T20:17:29.000Z","dependencies_parsed_at":"2023-01-13T15:37:47.389Z","dependency_job_id":null,"html_url":"https://github.com/JoshData/jot","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JoshData%2Fjot","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JoshData%2Fjot/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JoshData%2Fjot/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JoshData%2Fjot/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JoshData","download_url":"https://codeload.github.com/JoshData/jot/tar.gz/refs/heads/primary","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246296619,"owners_count":20754635,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T02:00:52.891Z","updated_at":"2025-03-30T11:31:50.746Z","avatar_url":"https://github.com/JoshData.png","language":"JavaScript","funding_links":[],"categories":["JavaScript"],"sub_categories":[],"readme":"JSON Operational Transformation (JOT)\n=====================================\n\nBy Joshua Tauberer \u003chttps://razor.occams.info\u003e.\n\nAugust 2013.\n\nLicense: GPL v3 \u003chttp://choosealicense.com/licenses/gpl-v3/\u003e\n\nThis module implements operational transformation (OT) on a JSON data model,\nwritten in JavaScript for use either in node.js or browsers.\n\nWhile most collaborative editing models operate on plain text documents with\noperations like insert and delete on strings, the document model in JOT is JSON\n--- i.e. the value space of null, booleans, numbers, strings, arrays, and\nobjects (key-value pairs with string keys). JOT includes the basic insert/delete\noperations on strings but adds many other operations that make JOT useful\nfor tracking changes to any sort of data that can be encoded in JSON.\n\nBasically, this is the core of real time simultaneous editing, like Etherpad,\nbut for structured data rather than just plain text. Since everything can\nbe represented in JSON, this provides plain text collaboration functionality\nand much more.\n\nThis is a work in progress. There is no UI or collaboration framework here.\n\nWhy JOT?\n--------\n\n### Introduction\n\nThe core problem addressed by operational transformation libraries like JOT\nis merging edits made simultaneously, i.e. asynchronously, by two or more\nusers, and the handling of potential conflicts that arise when multiple\nusers edit the same part of the document.\n\nTo illustrate the problem, imagine two users open the following JSON document:\n\n\t{ \"title\": \"Hello World!\", \"count\": 10 }\n\nEach user now has a copy of this document in their local memory. The first user\nmodfies their copy by changing the title and incrementing the count:\n\n \t{ \"title\": \"It's a Small World!\", \"count\": 20 }\n\nAt the same time, the second user changes their copy of the document by changing\n`Hello World!` to `Hello, Small World!` also incrementing the count by 5, yielding:\n\n \t{ \"title\": \"Hello, Small World!\", \"count\": 15 }\n\n### Structured representation of changes\n\nIn order to merge these changes, there needs to be a structured representation of the\nchanges. In the flat land of plain text, you are probably used to diffs and patches\nas structured representations of changes --- e.g. at lines 5 through 10, replace with\nnew content. In JOT, it is up to the library user to form structured representations\nof changes using JOT's classes. The changes in the example above are constructed as:\n\n\tvar user1 = new jot.LIST([\n\t\tnew jot.APPLY(\"title\", new jot.SPLICE(0, 5, \"It's a Small\")),\n\t\tnew jot.APPLY(\"count\", new jot.MATH(\"add\", 10))\n\t]);\n\n\tvar user2 = new jot.LIST([\n\t\tnew jot.APPLY(\"title\", new jot.SPLICE(5, 1, \", Small \")),\n\t\tnew jot.APPLY(\"count\", new jot.MATH('add', 5))\n\t]);\n\nIn other words, user 1 makes a change to the `title` property by replacing the 5 characters\nstarting at position 0 with `It's a Small` and increments the `count` property by 10.\nUser 2 makes a change to the `title` property by replacing the 1 character at position 5,\ni.e. the first space, with `, Small ` and increments the `count` property by 5.\n\nThese changes cannot yet be combined. If they were applied in order we would get a corrupted\ndocument because the character positions that user 2's operation referred to are shifted once\nuser 1's changes are applied. After applying user 1's changes, we have the document:\n\n\t{ title: \"It's a Small World!\", count: 20 }\n\nBut then if we apply user 2's changes, which say to replace the character at position 5, we\nwould get:\n\n\t{ title: \"It's , Small  Small World!\", count: 25 }\n\nThat's not what user 2 intended. **The second user's changes must be \"transformed\" to take into\naccount the changes to the document made by the first user before they can be applied.**\n\n### Transformation\n\nJOT provides an algorithm to transform the structured representation of changes so \nthat simultaneous changes can be combined sequentially.\n\nContinuing the example, we desire to transform the second user's changes so that\nthey can be applied in sequence after the first user's changes.\n\nInstead of\n\n\t... new jot.APPLY(\"title\", new jot.SPLICE(5, 1, \", Small \"))  ...\n\nwe want the second user's changes to look like\n\n\t... new jot.APPLY(\"title\", new jot.SPLICE(12, 1, \", Small \"))  ...\n\nNote how the character index has changed. These changes now _can_ be applied after the first\nuser's changes and achieve the _intent_ of user 2's change.\n\nJOT provides a `rebase` function on operation objects that can make this\ntransformation. (The transformation is named after [git's rebase](https://git-scm.com/book/en/v2/Git-Branching-Rebasing).) The `rebase` function transforms the operation and yields a new operation that should be applied instead, taking as an argument the operations executed by another user concurrently that have already applied to the document:\n\n\tuser2 = user2.rebase(user1)\n\nThese changes can now be merged using `compose`:\n\n\tvar all_changes = user1.compose(user2);\n\nand then applied to the base document:\n\n\tdocument = all_changes.apply(document)\n\nafter which the base document will include both user's changes:\n\n\t{ title: \"It's a Small, Small World!\", count: 25 }\n\nIt would also have been possible to rebase `user1` and then compose the operations in the other order, for the exact same result.\n\nSee [example.js](example.js) for the complete example.\n\n### Compared to other OT libraries\n\nOperational transformation libraries often operate only over strings. JOT has\nthose operations too. For instance, start with the document:\n\n\tHello world!\n\nTwo simultaneous changes might be:\n\n\tUser 1: REPLACE CHARS 0-4 WITH \"Brave new\"\n\n\tUser 2: REPLACE CHARS 11-11 WITH \".\"\n\nTo merge these changes, the second user's changes must be rebased to:\n\n\tUser 2: REPLACE CHARS 15-15 WITH \".\"\n\nJOT's rebase algorithm can handle this case too:\n\n\t// Construct operations\n\tvar document = \"Hello world!\";\n\tvar user1 = new jot.SPLICE(0, 5, \"Brave new\");\n\tvar user2 = new jot.SPLICE(11, 1, \".\");\n\n\t// Rebase user 2\n\tuser2 = user2.rebase(user1, { document: document })\n\n\t// user2 now holds:\n\t// new jot.SPLICE(15, 1, \".\")\n\n\t// Merge\n\tuser1.compose(user2).apply(document);\n\t\u003e 'Brave new world.'\n\nUnlike most collaborative editing models where there are only operations like insert and delete that apply to strings, the document model in JOT is JSON --- i.e. the value space of null, booleans, numbers, strings, arrays, and objects (key-value pairs with string keys). Operations are provided that manipulate all of these data types. This makes JOT useful when tracking changes to data, rather than simply to plain text.\n\n\nInstallation\n------------\n\nThe code is written for the node.js platform and can also be used client-side in modern browsers.\n\nFirst install node, then install this package:\n\n\tnpm install git+https://github.com/joshdata/jot.git\n\nIn a node script, import the library:\n\n\tvar jot = require(\"jot\");\n\nTo build the library for browsers, run:\n\n\tnpm install -g browserify\n\tbrowserify browser_example/browserfy_root.js -d -o dist/jot.js\n\nThen use the library in your HTML page (see [the example](browser_example/example.html) for details):\n\n\t\u003chtml\u003e\n\t\t\u003cbody\u003e\n\t\t\t\u003cscript src=\"jot.js\"\u003e\u003c/script\u003e\n\t\t\t\u003cscript\u003e\n\t\t\t\t// see the example below, but skip the 'require' line\n\t\t\t\u003c/script\u003e\n\t\t\u003c/body\u003e\n\t\u003c/html\u003e\n\n\nOperations\n----------\n\nThe operations in JOT are instantiated as `new jot.OPERATION(arguments)`. The available operations are...\n\n### General operations\n\n* `SET(new_value)`: Replaces any value with any other JSON-able value. `new_value` is the new value after the operation applies.\n* `LIST([op1, op2, op3, ...])`: Executes a series of operations in order. `op1`, `op2`, `op3`, ... are other JOT operations. Equivalent to `op1.compose(op2).compose(op3)...`.\n\n### Operations on booleans and numbers\n\n* `MATH(op, value)`: Applies an arithmetic or boolean operation to a value. `op` is one of \"add\", \"mult\" (multiply), \"rot\" (increment w/ modulus), \"and\" (boolean or bitwise and), \"or\" (boolean or bitwise or), \"xor\" (boolean or bitwise exclusive-or), \"not\" (boolean or bitwise negation). For `rot`, `value` is given as an array of `[increment, modulus]`. For `not`, `value` is ignored and should be `null`. `add` and `mult` apply to any number, `rot` applies to integers only, and the boolean/bitwise operations only apply to integers and booleans. Because of rounding, operations on floating-point numbers or with floating-point operands could result in inconsistent state depending on the order of execution of the operations.\n\n### Operations on strings and arrays\n\nThe same operation is used for both strings and arrays:\n\n* `SPLICE(index, length, new_value)`: Replaces text in a string or array elements in an array at the given index and length in the original. To delete, `new_value` should be an empty string or zero-length array. To insert, `length` should be zero.\n* `ATINDEX(index, operation)`: Apply any operation to a particular array element at `index`. `operation` is any operation. Operations at multiple indexes can be applied simultaneously using `ATINDEX({ index1: op1, index2: op2, ... })`.\n* `MAP(operation)`: Apply any operation to all elements of an array (or all characters in a string). `operation` is any operation created by these constructors.\n\n`SPLICE` is the only operation you need for basic plain text concurrent\nediting. JOT includes the entire text editing model in the `SPLICE`\noperations plus it adds new operations for non-string data structures!\n\n(Note that internally `SPLICE` and `ATINDEX` are sub-cases of an internal PATCH operation that maintains an ordered list of edits to a string or array.)\n\n### Operations on objects\n\n* `PUT(key, value)`: Adds a new property to an object. `key` is any valid JSON key (a string) and `value` is any valid JSON object. Equivalent to `APPLY(key, SET(value))`.\n* `REM(key)`: Remove a property from an object. Equivalent to `APPLY(key, SET(~))` where `~` is a special internal value.\n* `APPLY(key, operation)`: Apply any operation to a particular property named `key`. `operation` is any operation. The operation can also take a mapping from keys to operations, as `APPLY({key: operation, ...})`.\n\n### Operations that affect document structure\n\n* `COPY([ [source1, target1], [source2, target2], ... ])`: Copies a value from one location in the document to another. The source and target parameters are [JSON Pointer](https://tools.ietf.org/html/rfc6901) strings (but `/-` is not allowed). Use in combination with other operations to move parts of the document, e.g. a `COPY` plus a `REM` can be used to rename an object property.\n\nMethods\n-------\n\n### Instance Methods\n\nEach operation object provides the following instance methods:\n\n* `op.inspect()` returns a human-readable string representation of the operation. (A helper method so you can do `console.log(op)`.)\n* `op.isNoOp()` returns a boolean indicating whether the operation does nothing.\n* `op.apply(document)` applies the operation to the document and returns the new value of the document. Does not modify `document`.\n* `op.simplify()` attempts to simplify complex operations. Returns a new operation or the operation unchanged. Useful primarily for `LIST`s.\n* `op.drilldown(index_or_key)` looks inside an operation on a string, array, or object and returns the operation that represents the effect of this operation on a particular index or key.\n* `op.inverse(document)` returns the inverse operation, given the document value *before* the operation applied.\n* `op.compose(other)` composes two operations into a single operation instance, sometimes a `LIST` operation.\n* `op.rebase(other)` rebases an operation. Returns null if the operations conflict, otherwise a new operation instance.\n* `op.rebase(other, { document: ... })` rebases an operation in conflictless mode. The document value provided is the value of the document *before* either operation applied. Returns a new operation instance. See further documentation below.\n* `op.toJSON()` turns the operation into a JSON-able data structure (made up of objects, arrays, strings, etc). See `jot.opFromJSON()`. (A helper method so you can do `JSON.stringify(op)`.)\n* `op.serialize()` serializes the operation to a string. See `jot.deserialize()`.\n\n### Global Methods\n\nThe `jot` library itself offers several global methods:\n\n* `jot.diff(a, b, options)` compares two documents, `a` and `b`, and returns a JOT operation that when applied to `a` gives `b`.  `options`, if given, is an object that controls how the diff is performed. Any data type that can be a JOT document (i.e. any JSON-like data type) can be compared, and the comparison follows the document structure recursively. If the keys `words`, `lines`, or `sentences` is set to a truthy value, then strings are compared word-by-word, line-by-line, or sentence-by-sentence, instead of character-by-character. There is no general purpose structured diff algorithm that works well on all documents --- this one probably works fine on relatively small structured data.\n* `jot.opFromJSON(opdata)` is the inverse of `op.toJSON()`.\n* `jot.deserialize(string)` is the inverse of `op.serialize()`.\n\nConflictless Rebase\n-------------------\n\nWhat makes JOT useful is that each operation knows how to \"rebase\" itself against\nevery other operation. This is the \"transformation\" part of operational transformation,\nand it's what you do when you have two concurrent edits that need to be merged.\n\nThe rebase operation guarantees that any two operations can be combined in any order\nand result in the same document. In other words, rebase satisfies the constraints\n`A ○ (B/A) == B ○ (A/B)` and `C / (A○B) == (C/A) / B`, where `○` is `compose`\nand `/` is rebase.\n\n### Rebase conflicts\n\nIn general, not all rebases are possible in a way that preserves the logical intent\nof each change. This is what results in a merge conflict in source code control\nsoftware like git. The conflict indicates where two operations could not be merged\nwithout losing the logical intent of the changes and intervention by a human is\nnecessary. `rebase` will return `null` in these cases.\n\nFor example, two `MATH` operations with different operators will conflict because\nthe order that these operations apply is significant:\n\n\t\u003e new jot.MATH(\"add\", 1)\n\t    .rebase( new jot.MATH(\"mult\", 2) )\n\tnull\n\n(10 + 1) * 2 = 22 but (10 * 2) + 1 == 21. A vanilla `rebase` will return `null` in this case\nto signal that human intervention is needed to choose which operation should apply\nfirst.\n\n### Using conflictless rebase\n\nHowever, JOT provides a way to guarantee that `rebase` will return *some* operation,\nso that a merge conflict cannot occur. We call this \"conflictless\" rebase. The result\nof a conflictless rebase comes *close* to preserving the logical intent of the\noperations by choosing one operation over the other *or* choosing an order that\nthe operations will apply in.\n\nTo get a conflictless rebase, pass a second options argument to `rebase` with the\n`document` option set to the content of the document prior to both operations applying:\n\n\t\u003e new jot.MATH(\"add\", 1)\n\t    .rebase( new jot.MATH(\"mult\", 2),\n\t             { document: 10 } )\n\t\u003cvalues.SET 22\u003e\n\nThe rebase returns a valid operation now, in this case telling us that to add 1 *after the multiplication has applied*, we should simply set the\nresult to 22 instead of adding 1. In other words, the rebase has chosen the order\nwhere multiplication goes second.\n\nRebasing the other way around yields a consistent operation:\n\n\t\u003e new jot.MATH(\"mult\", 2)\n\t    .rebase( new jot.MATH(\"add\", 1),\n\t             { document: 10 } )\n\t\u003cvalues.MATH mult:2\u003e\n\nIn other words, if we're multing by 2 *after the addition has applied*, we should\ncontinue to multiply by 2. That's the same order as rebase chose above.\n\nDevelopment and Testing\n-----------------------\n\nRun code coverage tests with `npm test` or:\n\n\tnpm test -- --coverage-report=html\n\nNotes\n-----\n\nThanks to @konklone for some inspiration and the first pull request.\n\nSimilar work: [ShareDB](https://github.com/share/sharedb), [ottypes/json0](https://github.com/ottypes/json0), [Apache Wave](http://incubator.apache.org/wave/) (formerly Google Wave), [Substance Operator](https://github.com/substance/operator) (defunct).","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJoshData%2Fjot","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FJoshData%2Fjot","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FJoshData%2Fjot/lists"}