{"id":19732402,"url":"https://github.com/terminusdb/diffpatchjson","last_synced_at":"2025-04-04T22:11:54.285Z","repository":{"id":101654883,"uuid":"459266524","full_name":"terminusdb/diffpatchjson","owner":"terminusdb","description":"API to Efficiently Diff and Patch JSON ","archived":false,"fork":false,"pushed_at":"2022-02-16T13:37:37.000Z","size":22,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-02-10T06:41:26.457Z","etag":null,"topics":["collaboration","diff","json","jsondiffpatch","patch","revision-control","terminusdb"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/terminusdb.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-02-14T17:50:17.000Z","updated_at":"2025-01-06T12:51:50.000Z","dependencies_parsed_at":"2023-05-03T23:31:12.198Z","dependency_job_id":null,"html_url":"https://github.com/terminusdb/diffpatchjson","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/terminusdb%2Fdiffpatchjson","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/terminusdb%2Fdiffpatchjson/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/terminusdb%2Fdiffpatchjson/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/terminusdb%2Fdiffpatchjson/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/terminusdb","download_url":"https://codeload.github.com/terminusdb/diffpatchjson/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247256116,"owners_count":20909240,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["collaboration","diff","json","jsondiffpatch","patch","revision-control","terminusdb"],"created_at":"2024-11-12T00:26:14.663Z","updated_at":"2025-04-04T22:11:54.267Z","avatar_url":"https://github.com/terminusdb.png","language":null,"readme":"# [JSON Diff and Patch](https://terminusdb.com/docs/index/json-diff-and-patch)\n---\n\n### Introduction to JSON Diff and Patch (JSON-DP)\n---\n\n## Key topics\n\n[Use JSON-DP with a TerminusDB client](json-diff-and-patch.md#use-json-dp-with-a-terminusdb-client)\n\n[JSON-DP operations](json-diff-and-patch.md#json-dp-operations)\n\n[JSON-DP examples using curl](json-diff-and-patch.md#diff-examples-using-curl)\n\n## The uses of JSON-DP\n\nJSON objects are a common way of representing data for software development. The serialization of JSON is simple and facilitates communication via networks and storage in databases. Almost all modern programming languages support JSON objects natively.\n\nWhen objects are modified in distributed systems, it is useful to compare versions of an object to see what has changed. This is where **diff** and **patch** come in.\n\n### Diff\n\nA **diff** takes two JSON objects and presents any differences between them. Diff has several uses. A key use is displaying a clear summary of differences between large objects, enhancing the visibility of changes. This enables manual or user-interface assisted action to resolve changes. Actions include retaining the original object, changing to the new (or latest) version of the object, or creating an entirely new version of the object.\n\n### Patch\n\nA **patch** applies a diff to two objects to obtain a new object with any differences highlighted. A patch is applied individually or in bulk to a patch endpoint that will apply the patch to the specified data product.\n\n## Use JSON-DP with a TerminusDB client\n\nUse JSON-DP with a TerminusDB JavaScript or Python client to find and handle changes in TerminusDB schemas and documents, JSON schemas, and other document databases such as MongoDB. See [JSON-DP client tutorials](json-diff-and-patch.md#json-dp-client-tutorials) for use cases, including connectivity with MongoDB.\n\n### Prerequisites\n\n[Install TerminusDB JavaScript client](../readme/terminusx/install/install-javascript-client.md)\n\n[Install TerminusDB Python client](../readme/terminusx/install/install-python-client.md)\n\n### Get started\n\nGet started with the simple steps below.\n\nIf using **TerminusX with Python**, connect to your TerminusX cloud instance first - see [Connect with WOQLClient](../readme/terminusx/quick-start/start-with-client.md#connect-with-woqlclient) for instructions if required. \n\n1\\. [Create an endpoint](json-diff-and-patch.md#create-an-endpoint)\n\n2\\. [Apply a diff to obtain a patch](json-diff-and-patch.md#apply-a-diff-to-obtain-a-patch)\n\n3\\. [Review the patch](json-diff-and-patch.md#review-the-patch)\n\n4\\. [Apply the patch](json-diff-and-patch.md#apply-the-patch)\n\n#### Create an endpoint\n\nCreate a client endpoint with `WOQLClient`.\n\n\n**JavaScript**\n```javascript\nconst TerminusClient = require(\"@terminusdb/terminusdb-client\");\n\nvar client = new TerminusClient.WOQLClient(\"http://127.0.0.1:6363\")\n```\n\n**Python**\n```python\nfrom terminusdb_client import WOQLClient\n\nclient = WOQLClient(\"http://localhost:6363/\")\n```\n\n#### Apply a diff to obtain a patch\n\nGet the difference/s between two hypothetical documents - `Doc1` and `Doc2`.\n\n**JavaScript**\nUse `getDiff`\n\n\n\n```javascript\nlet result_patch = await client.getDiff(Doc1, Doc2)\n```\n\n\n**Python**\nUse`diff`\n\n\n\n```python\nresult_patch = client.diff(Doc1, Doc2)\n```\n\n\n#### Review the patch\n\nPrint the contents of a patch.\n\n**JavaScript**\n```javascript\nconsole.log(result_patch)\n```\n\n\n**Python**\nExample uses`pprint` (`from pprint import pprint`)\n\n\n\n```python\npprint(result_patch.content)\n```\n\n#### Apply the patch\n\nApply the patch to `Doc1`.\n\n**JavaScript**\n```javascript\nlet after_patch = await client.patch(Doc1, result_patch);\n```\n\n\n**Python**\n```python\nafter_patch = client.patch(Doc1, result_patch)\n```\n\n## JSON-DP operations \u0026#x20;\n\nThe available JSON-DP operations with some examples using `curl`.\n\n### Copy Diff\n\nCopy is implicit. All properties which are not specifically mentioned will be considered part of an implicit copy. This will make patches more compressed and easier to specify by hand.\n\n### Mandatory Diff\n\n`@before`/`@after` instructions contain objects specified as tightly as required to obtain ids, or as ids.\n\n```jsx\n{ '@id' : \"Person/jim\",\n  'date_of_birth' : { '@op' : 'SwapValue',\n                      '@before' : \"1928-03-05\",\n                      '@after' : \"1938-03-05\"\n                    }}\n```\n\n### Optional Diff\n\nOptional diffs also contain `@before`/`@after` designations, but potentially `null` fields to describe missing elements.\n\n```jsx\n{ '@id' : \"Object/my_object\",\n  'name' : { '@op' : 'SwapValue',\n             '@before' : null,\n             '@after' : \"Jim\" }}\n```\n\n### Set Diff / Cardinality Diff\n\nSet requires the ability to explicitly remove or add elements - we can do this by maintaining a `@before`/`@after` with a list of those which exist _only_ on the left, and _only_ on the right.\n\n### List Diff\n\nThe list diff requires swaps at a position. We use, `@copy`, `@swap` and `@keep`.\n\n#### Copy List\n\nCopy the previous list from `From_Position` to `To_Position`.\n\n```jsx\n{ \"@op\" : \"CopyList\",\n  \"@to\" : To_Position,\n  \"@rest\" : Diff }\n```\n\n#### Swap List\n\nSwap out the list starting from the current point from `Previous` to `Next`. This can be used to extend, or drop elements as well as do full replacement.\n\n```jsx\n{ \"@op\" : \"SwapList\",\n  \"@before\" : Previous,\n  \"@after\" : Next,\n  \"@rest\" : Diff }\n```\n\n#### Patch List\n\nPatch the list starting from the current point with the patch list in `\"@patch\"`. The patch must be less than or equal to the length of the list.\n\n```jsx\n{ \"@op\" : \"PatchList\",\n  \"@patch\" : Patch,\n  \"@rest\" : Diff }\n```\n\n#### Example:\n\n```jsx\nvar Patch =\n{ '@id' : \"TaskList/my_tasks\",\n  'tasks' : { '@op' : \"CopyList\",                      % Replace List\n              '@to' : 2,\n              '@rest' : { '@op' : \"PatchList\",\n                          '@patch' : [{ '@op' : \"SwapValue\",\n                                        '@before' : \"Task/shopping\",\n                                        '@after' : \"Task/climbing\"},\n                                      { '@op' : \"SwapValue\",\n                                        '@before' : \"Task/cleaning\",\n                                        '@after' : \"Task/dining\"},\n                                      { '@op' : \"SwapValue\",\n                                        '@before' : \"Task/fishing\",\n                                        '@after' : \"Task/travelling\"}],\n                          '@rest' : { '@op' : \"KeepList\" } } }}\nvar Before =\n{ '@id' : \"TaskList/my_tasks\",\n  'tasks' : [\"Task/driving\", \"Task/reading\", \"Task/shopping\",\n             \"Task/cleaning\",\"Task/fishing\", \"Task/arguing\"] }\nvar After =\n{ '@id' : \"TaskList/my_tasks\",\n  'tasks' : [\"Task/driving\", \"Task/reading\", \"Task/climbing\",\n             \"Task/dining\", \"Task/travelling\", \"Task/arguing\"] }\n```\n\n### Array Diff\n\nArrays will allow index swapping or \"shrink\" and \"grow\".\n\n### Force Diff\n\nA \"Force Diff\" will set the value of a location regardless of the current read-state. This is a potentially unsafe operation as there is no guarantee we are seeing the object state version we think we are.\n\n```jsx\n{ '@id' : \"Employee/012\" ,\n  'name' : { '@op' : 'ForceValue',\n             '@after' : \"Jake\" }}\n```\n\n### Table Diff\n\nA Table diff specifies the differences and similarities between the two tables. These tables _need not_ have the same dimensions. In order to describe these differences, we use a `ModifyTable` patch. The `ModifyTable` patch is comprised of `copies`, `deletes`, `inserts` and `moves`.\n\n`copies` give the sections of the table which can be copied verbatim. `deletes` gives all segments which are to be removed from the original. `inserts` gives all segments which are to be inserted into the new table.\n\n`moves` specifies segments that are the same in both tables, but have moved location. This is particularly useful as moving rows and columns is a typical operation in a table (such as a CSV or Excel document).\n\n#### Example Table\n\nGiven the following table:\n\n```javascript\n[['Job Title','Company','Location','Company Size','Company Industry'],\n ['Sr. Mgt.','Boeing','USA','Large','Aerospace'],\n ['Data Architect','Airbus','France','Large','Aerospace'],\n ['Founder','Ellie Tech','Sweden','Startup','AI'],\n ['Platform Engineer','Adidas','Germany','Large','Apparel']]\n```\n\nAnd a sorted version of the same (sorting on the first column):\n\n```javascript\n[['Job Title','Company','Location','Company Size','Company Industry'],\n ['Data Architect','Airbus','France','Large','Aerospace'],\n ['Founder','Ellie Tech','Sweden','Startup','AI'],\n ['Platform Engineer','Adidas','Germany','Large','Apparel'],\n ['Sr. Mgt.','Boeing','USA','Large','Aerospace']]\n```\n\nWe have the following patch resulting from the diff:\n\n```\n{'@op':\"ModifyTable\",\n dimensions:{'@after':[5,5],'@before':[5,5]},\n deletes:[],\n inserts:[],\n copies:[{'@at':{'@height':1,'@width':5,'@x':0,'@y':0},'@value':[['Job Title','Company','Location','Company Size','Company Industry']]}],\n moves:[{'@from':{'@height':1,'@width':5,'@x':0,'@y':1},\n         '@to':{'@height':1,'@width':5,'@x':0,'@y':4},\n         '@value':[['Sr. Mgt.','Boeing','USA','Large','Aerospace']]},\n        {'@from':{'@height':1,'@width':5,'@x':0,'@y':2},\n         '@to':{'@height':1,'@width':5,'@x':0,'@y':1},\n         '@value':[['Data Architect','Airbus','France','Large','Aerospace']]},\n        {'@from':{'@height':1,'@width':5,'@x':0,'@y':3},\n         '@to':{'@height':1,'@width':5,'@x':0,'@y':2},\n         '@value':[['Founder','Ellie Tech','Sweden','Startup','AI']]},\n        {'@from':{'@height':1,'@width':5,'@x':0,'@y':4},\n         '@to':{'@height':1,'@width':5,'@x':0,'@y':3},\n         '@value':[['Platform Engineer','Adidas','Germany','Large','Apparel']]}]}\n```\n\n## Diff and Patch Endpoints\n\nThe Patch and Diff endpoints expose endpoints for clients to obtain diffs or patches of data.\n\n### Diff\n\nThe diff endpoint takes a POST of two JSON documents, _before_, and _after_. This endpoint then returns a 200 and a patch which takes _before_ to _after_ if applied using the patch interface. The payload is structured as a JSON document, with two keys, `\"before\"` and `\"after\"`, pointing to the documents you would like to diff.\n\nAn example of the payload:\n\n```json\n{ \"before\" : { \"@id\" : \"Person/Jane\", \"@type\" : \"Person\", \"name\" : \"Jane\"},\n  \"after\" :  { \"@id\" : \"Person/Jane\", \"@type\" : \"Person\", \"name\" : \"Janine\"}}\n```\n\nWhich would result in the following patch:\n\n```json\n{ \"name\" : { \"@op\" : \"SwapValue\", \"@before\" : \"Jane\", \"@after\": \"Janine\" }}\n```\n\n#### Diff examples using curl\n\n```shell\n$ curl -X POST -H \"Content-Type: application/json\" 'http://127.0.0.1:6363/api/diff' -d \\\n  '{ \"before\" : [{ \"asdf\" : \"foo\"}], \"after\" : [{ \"asdf\" : \"bar\"}]}'\n# Output: [ {\"asdf\": {\"@after\":\"bar\", \"@before\":\"foo\", \"@op\":\"SwapValue\"}} ]\n```\n\n```bash\n$ curl -X POST -H \"Content-Type: application/json\" 'http://127.0.0.1:6363/api/diff' -d \\\n  '{ \"before\" : [0,1,2], \"after\" : [0,1,2,3]}'\n\n# Output:\n{\n  \"@op\":\"CopyList\",\n  \"@rest\": {\n    \"@after\": [3 ],\n    \"@before\": [],\n    \"@op\":\"SwapList\",\n    \"@rest\": {\"@op\":\"KeepList\"}\n  },\n  \"@to\":3\n}\n```\n\n```bash\n$ curl -X POST -H \"Content-Type: application/json\" 'http://127.0.0.1:6363/api/diff' -d \\\n  '{ \"before\" : { \"asdf\" : { \"fdsa\" : \"quux\"}}, \"after\" : { \"asdf\" : { \"fdsa\" : \"quuz\" }}}'\n\n# Output:\n{\n  \"asdf\": {\"fdsa\": {\"@after\":\"quuz\", \"@before\":\"quux\", \"@op\":\"SwapValue\"}}\n}\n```\n\n### Patch\n\nPatch takes a POST with a _before_ document and a _patch_ and produces an _after_ document.\n\n```json\n{ \"before\" : { \"@id\" : \"Person/Jane\", \"@type\" : \"Person\", \"name\" : \"Jane\"}\n  \"patch\" : {\"name\" : { \"@op\" : \"ValueSwap\", \"@before\" : \"Jane\", \"@after\": \"Janine\" }}}\n```\n\nResulting in the following document:\n\n```json\n{ \"@id\" : \"Person/Jane\", \"@type\" : \"bahPerson\", \"name\" : \"Janine\"}\n```\n\n#### Patch examples using curl\n\n```shell\n$ curl -X POST -H \"Content-Type: application/json\" 'http://127.0.0.1:6363/api/patch' -d \\\n   '{ \"before\" : { \"alpha\" : 1, \"asdf\" : { \"fdsa\" : \"quux\"}}, \"patch\" : {\n      \"asdf\": {\"fdsa\": {\"@after\":\"quuz\", \"@before\":\"quux\", \"@op\":\"SwapValue\"}}\n}}'\n# Output: {\"alpha\":1, \"asdf\": {\"fdsa\":\"quuz\"}}\n```\n\n```bash\n$ curl -X POST -H \"Content-Type: application/json\" 'http://127.0.0.1:6363/api/patch' -d '\n{ \"before\" : [0,1,2], \"patch\" : {\n  \"@op\":\"CopyList\",\n  \"@rest\": {\n    \"@after\": [3 ],\n    \"@before\": [],\n    \"@op\":\"SwapList\",\n    \"@rest\": {\"@op\":\"KeepList\"}\n  },\n  \"@to\":3\n}}'\n#Output: [0, 1, 2, 3 ]\n```\n\n## See Also\n\n### JSON-DP client tutorials\u0026#x20;\n\nTutorials for using JSON-DP with a TerminusDB [JavaScript or Python client](https://github.com/terminusdb/terminusdb-tutorials/tree/diff\\_patch/diff\\_patch), including the use of JSON-DiffPatch with MongoDB.\n\n### JSON-DP JavaScript client reference guide\n\nJSON-DP functions in the [JavaScript client reference guide](https://terminusdb.github.io/terminusdb-client-js/#/api/woqlClient.js?id=getdiff).\n\n### JSON-DP Python client reference guide\n\nJSON-DP functions in the [Python client reference guide](https://terminusdb.github.io/terminusdb-client-python/woqlClient.html?highlight=diff#terminusdb\\_client.WOQLClient.diff).\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fterminusdb%2Fdiffpatchjson","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fterminusdb%2Fdiffpatchjson","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fterminusdb%2Fdiffpatchjson/lists"}