{"id":26477516,"url":"https://github.com/slashdotted/libmelda-tools","last_synced_at":"2026-02-15T12:32:26.313Z","repository":{"id":62444725,"uuid":"476692801","full_name":"slashdotted/libmelda-tools","owner":"slashdotted","description":"Tools for Melda","archived":false,"fork":false,"pushed_at":"2025-03-18T09:13:50.000Z","size":61,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-18T09:35:29.217Z","etag":null,"topics":["cli","conflict-free-replicated-datatype","crdt","delta-state","json","rust","shell","tool"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/slashdotted.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-04-01T11:29:11.000Z","updated_at":"2025-03-18T09:13:54.000Z","dependencies_parsed_at":"2025-03-18T09:28:30.888Z","dependency_job_id":"d5b8a826-a4d5-45a8-9e21-68ace2c0ca4a","html_url":"https://github.com/slashdotted/libmelda-tools","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slashdotted%2Flibmelda-tools","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slashdotted%2Flibmelda-tools/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slashdotted%2Flibmelda-tools/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/slashdotted%2Flibmelda-tools/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/slashdotted","download_url":"https://codeload.github.com/slashdotted/libmelda-tools/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244227455,"owners_count":20419253,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cli","conflict-free-replicated-datatype","crdt","delta-state","json","rust","shell","tool"],"created_at":"2025-03-20T00:47:29.272Z","updated_at":"2026-02-15T12:32:21.263Z","avatar_url":"https://github.com/slashdotted.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# What are Melda Tools?\n\nMelda Tools are command line utilities to work with [libmelda](https://github.com/slashdotted/libmelda/).\n\n# What is Melda?\n\nMelda is a Delta-State JSON CRDT. CRDTs, which stand for Conflict-free Replicated Data Types, are data structures which can be replicated (copied) across multiple computers in a network. Each replica can be individually and concurrently updated without the need for central coordination or synchronization. Updates made on each replica can be merged at any time.\n\nThere exist different types of CRDTs: operation-based CRDTs (which generate and exchange update operations between replicas), state-based CRDTS (which exchange and merge the full state of each replica) and delta-state CRDT, such as Melda, (which exchange only the differences between versions, or states, of the data type).\n\nMelda natively supports the JSON data format and provides a way to synchronize changes made to arbitrary JSON documents.\n\n# How do I use Melda Tools?\n\nYou need to clone this repository and then compile the CLI tool executable:\n```\ngit clone https://github.com/slashdotted/libmelda-tools\ncd libmelda-tools\ncargo build\n```\nTo understand how to use Melda using the CLI tools, we consider the following situation, where a shared JSON document used by a fictitious activity planning software (i.e. a todo management software) is concurrently updated by multiple parties. The provided JSON is generated by the application (by serializing its data model). We assume that user **Alice** creates the first version of the shared JSON document, which will be named **v1_alice.json**. This first version contains the following data:\n\n```json\n{\n\t\"software\" : \"MeldaDo\",\n\t\"version\" : \"1.0.0\",\n\t\"items♭\" : []\n}\n\n```\nThe **root** object contains three fields: a **software** field which defines the name of the application, a **version** field, which sets the version of the software, and an **items♭** field, which maps to an array of JSON objects (one for each todo). Since this is the first version, the array of items is empty. The **♭** suffix is used to ask Melda to *flatten* the contents of the array, by extracting the contained JSON objects in order to keep track of their changes individually.\n\nTo better understand the purpose of the *flattening* procedure, consider how Melda processes the following two JSON files. The first one, named **v2_alice_noflat.json** contains:\n\n```json\n{\n\t\"software\" : \"MeldaDo\",\n\t\"version\" : \"1.0.0\",\n\t\"items\" : [\n\t   {\"_id\" : \"alice_todo_01\", \"title\" : \"Buy milk\", \"description\" : \"Go to the grocery store\"}\n\t]\n}\n\n```\nIn this case, Melda will keep the root object as is, and the changes made to the items array by one user will not merge with changes made by other users. So, for example, if two users add an element to the array on their replica and later merge those replicas, only one of the elements will be visible. On the contrary, consider now another version of the document, named **v2_alice.json**, which contains:\n\n```json\n{\n\t\"software\" : \"MeldaDo\",\n\t\"version\" : \"1.0.0\",\n\t\"items♭\" : [\n\t   {\"_id\" : \"alice_todo_01\", \"title\" : \"Buy milk\", \"description\" : \"Go to the grocery store\"}\n\t]\n}\n\n```\nIn this case the object within the **items♭** array will be extracted and tracked individually. In particular, two JSON objects results from the above document:\n```json\n{\n\t\"_id\" : \"√\",\n\t\"software\" : \"MeldaDo\",\n\t\"version\" : \"1.0.0\",\n\t\"items♭\" : [\n\t  \"alice_todo_01\"\n\t]\n}\n\n```\nAnd the todo item itself:\n```json\n{\n\t\"_id\" : \"alice_todo_01\",\n\t\"title\" : \"Buy milk\",\n\t\"description\" : \"Go to the grocery store\"\n}\n\n```\nPlease notice that each object has its own unique identifier stored in the **_id** field. If an identifier is not provided by the client application, Melda will auto-generate one. The root object is always identified by **√** (this identifier cannot be changed by the client application). Since each object of the **items♭** array is tracked individually, if an user adds an element to the array and later merges his/her replica with another user all changes will be preserved.\n\nIf the collection of items becomes too large don't worry! Melda only stores the differences between the newest revision of the document and the previous one instead of the full array. Now let's see how we can interact with Melda in order to update the data structure.\n\n## Adapters\n\nMelda implements a modular design where the logic of the CRDT is separated from the data storage. Storing the data (in our case, delta states) is achieved by means of **Adapters**. Melda already provides different types of adapters, supporting in-memory storage (**MemoryAdapter**), filesystem storage (**FilesystemAdapter**) and Solid Pods (**SolidAdapter**). Furthermore, it is possible to use a meta-adapter to compress data using the Flate2 algorithm (**Flate2Adapter**): such an adapter can be composed with other adapters.\n\nWith Melda Tools we can choose an an adapter that will store data on the filesystem (in the **todolist** directory) by specifying a path like **file://$(pwd)/todolist** (where *$(pwd)* returns the absolute path of the current directory). If we want to used compression we would add the **Flate2Adapter** we would use **file+flate://$(pwd)/todolist**.\n\n\n## Creating the CRDT\n\nIn order to create the first state or update the state of the CRDT we use the **update** command. Suppose that the first version of the document is stored in file **v1.json** and contains:\n```json\n{ \"software\" : \"MeldaDo\", \"version\" : \"1.0.0\", \"items♭\" : []}\n```\nAlice can create / update the CRDT inside the **todolist** directory with:\n```bash\n./target/debug/libmelda-tools update -a \"Alice\" -d \"First commit\" -j v1.json -t file://$(pwd)/todolist\n```\nPlease note that we assume that **Melda Tools** have been compiled in debug mode and that the executable is **./target/debug/libmelda-tools**.\n\nUpdates made to the CRDT are committed to disk. We can pass an optional **author** (*-a* option) and **description** (*-d* option) with additional information that will be stored along with the updates.\n\nThe result of the **update** is either an error message or the identifier of the committed block.\n\nUpon success, on disk (in the **todolist** directory) the following content should have been created:\n```\ntodolist/\n├── 49\n│   └── 49ccea4d5797250208edf9bc5d0b89edf23c30a61f5cb3fafb87069f07276a62.delta\n└── b4\n    └── b4e50e445542c4737f4cfd7a9193ffd3be3794049d361d114a44f36434257cb3.pack\n```\n\nThe **.delta** file is called **delta block**, and contains the versioning information of each object in the CRDT, wherease the **.pack** file is the **data pack** which stores the actual JSON content of each object. Each commit produces a new delta block (with a different name, which corresponds to the hash digest of its content) and possibly a data pack (if new JSON values are produced). The directory structure of the **todolist** directory organizes files into sub-directories according to their prefix. \n\nAlice can perform another update using (again) the **update** method. First, the contents of a new version are stored in **v2.json**:\n```json\n{ \"software\" : \"MeldaDo\", \"version\" : \"1.0.0\", \"items♭\" : [\n       {\"_id\" : \"alice_todo_01\", \"title\" : \"Buy milk\", \"description\" : \"Go to the grocery store\"}\n    ]\n    }\n```\nThen the CRDT is updated and changes are committed:\n```bash\n./target/debug/libmelda-tools update -a \"Alice\" -d \"Add buy milk\" -j v2.json -t file://$(pwd)/todolist\n```\n\nThe changes will reflect on disk (with new packs and blocks created in the corresponding directories):\n```\ntodolist/\n├── 2b\n│   └── 2b0a463fcba92d5cf7dae531a5c40b67aaa0f45ab351c15613534fb5bba28564.pack\n├── 49\n│   └── 49ccea4d5797250208edf9bc5d0b89edf23c30a61f5cb3fafb87069f07276a62.delta\n├── b4\n│   └── b4e50e445542c4737f4cfd7a9193ffd3be3794049d361d114a44f36434257cb3.pack\n└── b6\n    └── b6297035f06f13186160577099759dea843addcd1fbd05d24da87d9ac071da3b.delta\n```\n## Reading the data\n\nAt any time it is possible to read the state of the CRDT back into a JSON document using the **read** command:\n```bash\n./target/debug/libmelda-tools read -s file://$(pwd)/todolist\n```\n\nThis will print the following on the terminal:\n```json\n{\"_id\":\"√\",\"items♭\":[{\"_id\":\"alice_todo_01\",\"description\":\"Go to the grocery store\",\"title\":\"Buy milk\"}],\"software\":\"MeldaDo\",\"version\":\"1.0.0\"}\n```\n\nEach object managed by Melda will contain the **_id** field with the corresponding unique identifier.\n\n## Sharing data\n\nWe now suppose that Alice shares the current state of the  **todolist** directory with Bob (she can simply zip the contents and send the compressed file by e-mail to Bob). We assume that Bob saves the contents in the **todolist_bob** directory. Bob can perform some updates (which we assume are stored in **v3_bob.json**):\n```json\n{ \"software\" : \"MeldaDo\", \"version\" : \"1.0.0\", \"items♭\" : [\n       {\"_id\" : \"alice_todo_01\", \"title\" : \"Buy milk\", \"description\" : \"Go to the grocery store\"},\n       {\"_id\" : \"bob_todo_01\", \"title\" : \"Pay bills\", \"description\" : \"Withdraw 500 to pay bill\"},\n       {\"_id\" : \"bob_todo_02\", \"title\" : \"Call mom\", \"description\" : \"Call mom to schedule dinner\"}\n    ]\n    }\n```\n\nBob updates his own replica with:\n```bash\n./target/debug/libmelda-tools update -a \"Bob\" -d \"Add some todos\" -j v3_bob.json -t file://$(pwd)/todolist_bob\n```\n\nAs you might notice, two new items have been added by Bob. In the meantime, Alice continues to work on her replica, by removing one item (**alice_todo_01**) and adding a new item (**alice_todo_02**). The file used by Alice is called **v3_alice.json** and contains the following:\n```json\n{ \"software\" : \"MeldaDo\", \"version\" : \"1.0.0\", \"items♭\" : [\n        {\"_id\" : \"alice_todo_02\", \"title\" : \"Take picture of our dog\", \"description\" : \"It must be a nice one\"}\n     ]\n     }\n```\n\nTo update her own copy, Alice uses the following command line:\n```bash\n./target/debug/libmelda-tools update -a \"Alice\" -d \"Some more stuff to do\" -j v3_alice.json -t file://$(pwd)/todolist\n```\n\nFinally, Bob shares his own copy with Alice: now Alice simply needs to merge the content of the directory (as received from Bob) with the local directory (using something like **cp -r todolist_bob/* todolist/**). Alternatively, suppose that the data modified by Bob is in the **todolist_bob** directory on Alice's computer. To merge changes back into the **todolist** directory, Alice can use the **meld** method:\n```bash\n./target/debug/libmelda-tools meld -t file://$(pwd)/todolist -s file://$(pwd)/todolist_bob\n```\nAlice can then read the new state of the CRDT with:\n```bash\n./target/debug/libmelda-tools read -s file://$(pwd)/todolist\n```\n\nThe result, printed on the terminal should look like:\n```json\n{\"_id\":\"√\",\"items♭\":[{\"_id\":\"bob_todo_01\",\"description\":\"Withdraw 500 to pay bill\",\"title\":\"Pay bills\"},{\"_id\":\"bob_todo_02\",\"description\":\"Call mom to schedule dinner\",\"title\":\"Call mom\"},{\"_id\":\"alice_todo_02\",\"description\":\"It must be a nice one\",\"title\":\"Take picture of our dog\"}],\"software\":\"MeldaDo\",\"version\":\"1.0.0\"}\n```\n\nAs you can see, there is only one todo from Alice, as well as the two todos added by Bob.\n\nBoth Alice and Bob can see the history of changes made to their replica using the **log** command:\n```bash\n./target/debug/libmelda-tools log -s file://$(pwd)/todolist\n```\nFor Alice the result will look like:\n```\n(A) Block: d0d23eeaf013b216a32386e708fb37489743cb2c9c8153082fc8e944a91eedf6\n\t\tInformation: {\"author\":\"Bob\",\"description\":\"Add some todos\"}\n\t\tPacks: [\"515ebf5ebd96fe8210945856d09b53fa673434291a598c893db76bed117b243e\"]\n\t\tParents: [\"460b4dd46257efbb018201d9c1ada3e165174241b8ef9a30f8f0f0b77a551283\"]\n(A) Block: ec11159e3497a89d1f0cb23db2600239535c70cc35a4f4b5a96e1d561d2bead3\n\t\tInformation: {\"author\":\"Alice\",\"description\":\"Some more stuff to do\"}\n\t\tPacks: [\"967e769c2b65c0a30a9aeed1350ed78c46e98073c61a23421e8a7c4b721e61d0\"]\n\t\tParents: [\"460b4dd46257efbb018201d9c1ada3e165174241b8ef9a30f8f0f0b77a551283\"]\n(-) Block: 460b4dd46257efbb018201d9c1ada3e165174241b8ef9a30f8f0f0b77a551283\n\t\tInformation: {\"author\":\"Alice\",\"description\":\"Add buy milk\"}\n\t\tPacks: [\"2b0a463fcba92d5cf7dae531a5c40b67aaa0f45ab351c15613534fb5bba28564\"]\n\t\tParents: [\"49ccea4d5797250208edf9bc5d0b89edf23c30a61f5cb3fafb87069f07276a62\"]\n(O) Block: 49ccea4d5797250208edf9bc5d0b89edf23c30a61f5cb3fafb87069f07276a62\n\t\tInformation: {\"author\":\"Alice\",\"description\":\"First commit\"}\n\t\tPacks: [\"b4e50e445542c4737f4cfd7a9193ffd3be3794049d361d114a44f36434257cb3\"]\n```\nThe list of delta blocks contains **origin** blocks (**(O)**) and **anchor** blocks (**(A)**). Origin blocks are the first one that have been created: in our scenario there is only one origin, since the CRDT was created on one replica only (by Alice). There are however two **anchor** blocks, namely *d0d23eeaf013b216a32386e708fb37489743cb2c9c8153082fc8e944a91eedf6* (created by Bob) and *ec11159e3497a89d1f0cb23db2600239535c70cc35a4f4b5a96e1d561d2bead3* (created by Alice). This is due to the merge/meld operation that was performed by Alice. Multiple anchors will be referenced by the next commit.\n\nConcurrent modifications made by Alice and Bob also resulted in a conflict. By default this is automatically hidden, since Melda can cope with this situation without problems. We can nonetheless show the conflicting information using the **conflicts** command:\n```bash\n./target/debug/libmelda-tools conflicts -s file://$(pwd)/todolist\n```\nThis will show that the root document (√) has a conflict, and the conflicting versions will be shown (the one with 🏆 is the version currently chosen by Melda as the *winner*, conflicts are shown with 🗲):\n```\n√:\n\t🏆 3-8f147f811da66dccc212b3147a185c7c68d365e02ae84614e6533b7857d4744a_6258b20: {\"items♭\":[\"alice_todo_01\",\"bob_todo_01\",\"bob_todo_02\",\"alice_todo_02\"],\"software\":\"MeldaDo\",\"version\":\"1.0.0\"}\n\t🗲 3-5bf6651423be2df90bf3a7250a5b8d7e457da397ab7a31bd24f96c099c183711_6258b20: {\"items♭\":[\"alice_todo_02\",\"alice_todo_01\",\"bob_todo_01\",\"bob_todo_02\"],\"software\":\"MeldaDo\",\"version\":\"1.0.0\"}\n\n```\nFurther updates will always consider the *winner*. We can however resolve the conflict (and make it disappear from the conflict view) using the **resolve** command:\n```bash\n./target/debug/libmelda-tools resolve -t file://$(pwd)/todolist\n```\nThis command by default resolves all conflicts in all objects using the current *winner*. Different strategies can be chosen, in order to promote a different *winner*. The **conflicts** command will confirm that there are no conflicts.\n\n# Publications\n\n\n## 2022\nAmos Brocco \"Melda: A General Purpose Delta State JSON CRDT\". 9th Workshop on Principles and Practice of Consistency for Distributed Data (PaPoC'22). April 2022. [PDF](https://amosbrocco.ch/pubs/PaPoC_2022_Submission.pdf) [BibTex](https://amosbrocco.ch/pubs/bib/PaPoC_2022_Submission.bib)\n\n## 2021\nAmos Brocco \"Delta-State JSON CRDT: Putting Collaboration on Solid Ground\". (Brief announcement). 23rd International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS 2021). November 2021. [PDF](https://amosbrocco.ch/pubs/sss_2021_submission_13.pdf) [BibTex](https://amosbrocco.ch/pubs/bib/sss_2021_submission_13.bib)\n\n# License\n(c)2021-2022 Amos Brocco,\nGPL v3 (for now... but I will evaluate a change of license - to something like BSD3/MIT/... in the near future)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslashdotted%2Flibmelda-tools","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fslashdotted%2Flibmelda-tools","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fslashdotted%2Flibmelda-tools/lists"}