https://github.com/jackyzha0/bft-json-crdt

🏰 the first JSON-like Byzantine Fault Tolerant CRDT
https://github.com/jackyzha0/bft-json-crdt

bft crdt crdt-implementations json json-crdt

Last synced: about 21 hours ago
JSON representation

🏰 the first JSON-like Byzantine Fault Tolerant CRDT

• Host: GitHub
• URL: https://github.com/jackyzha0/bft-json-crdt
• Owner: jackyzha0
• License: mit
• Created: 2022-09-03T00:24:15.000Z (over 2 years ago)
• Default Branch: main
• Last Pushed: 2024-04-16T00:58:21.000Z (9 months ago)
• Last Synced: 2024-12-13T12:11:38.683Z (14 days ago)
• Topics: bft, crdt, crdt-implementations, json, json-crdt
• Language: Rust
• Homepage:
• Size: 3.16 MB
• Stars: 207
• Watchers: 7
• Forks: 11
• Open Issues: 1
• Metadata Files:
  • Readme: README.md
  • License: LICENSE.txt

Awesome Lists containing this project

README

        
# Byzantine Fault Tolerant CRDTs

This work is mainly inspired by implementing Martin Kleppmann's 2022 paper on *Making CRDTs Byzantine Fault Tolerant*[^2] 

on top of a simplified [Automerge](https://automerge.org/) implementation.

The goal is to show a working prototype that demonstrated in simple code the ideas behind

1. An Automerge-like CRDT

2. How a primitive list CRDT can be composed to create complex CRDTs like JSON

2. How to add Byzantine Fault Tolerance to arbitrary CRDTs

Unlike most other CRDT implementations, I leave out many performance optimizations that would make the basic algorithm harder to understand.

Check out the [accompanying blog post for this project!](https://jzhao.xyz/posts/bft-json-crdt)

## Benchmarks

Altough this implementation does not optimize for performance, it still nonetheless performs quite well.

Benchmarking happened on a 2019 Macbook Pro with a 2.6GHz i7.

Numbers are compared to Automerge which report their performance benchmarks [here](https://github.com/automerge/automerge-perf)

| # Ops | Raw String (JS) | Ours (basic) | Ours (BFT) | Automerge (JS) | Automerge (Rust) |

|--|--|--|--|--|--|

|10k       | n/a     | 0.081s   | 1.793s   | 1.6s         | 0.047s  |

|100k      | n/a     | 9.321s   | 38.842s  | 43.0s        | 0.597s  |

|All (259k)| 0.61s   | 88.610s  | 334.960s | Out of Memory| 1.780s  |

|Memory    | 0.1MB   | 27.6MB   | 59.5MB   | 880MB        | 232.5MB |

## Flamegraph

To get some flamegraphs of the time graph on MacOS, run:

```bash

sudo cargo flamegraph --dev --root --bench speed

```

## Further Work 

This is mostly a learning/instructional project but there are a few places where performance improvements are obvious:

1. This is backed by `std::Vec` which isn't great for random insert. Replace with a B-tree or something that provides better insert and find performance

    1. [Diamond Types](https://github.com/josephg/diamond-types) and [Automerge (Rust)](https://github.com/automerge/automerge-rs) use a B-tree

    2. Yjs is backed by a doubly linked-list and caches last ~5-10 accessed locations (assumes that most edits happen sequentially; seeks are rare)

    3. (funnily enough, main peformance hit is dominated by find and not insert, see [this flamegraph](./flamegraphs/flamegraph_unoptimized.svg))

2. Avoid calling `find` so many times. A few Automerge optimizations that were not implemented

    1. Use an index hint (especially for local inserts)

    2. Skipping the second `find` operation in `integrate` if sequence number is already larger

3. Improve storage requirement. As of now, a single `Op` weighs in at *over* 168 bytes. This doesn't even fit in a single cache line!

4. Implement 'transactions' for a group of changes that should be considered atomic.

    1. This would also speed up Ed25519 signature verification time by batching.

    2. For example, a peer might create an atomic 'transaction' that contains a bunch of changes.

5. Currently, each character is a single op. Similar to Yjs, we can combine runs of characters into larger entities like what André, Luc, et al.[^1] suggest

6. Implement proper persistence using SQLLite or something similar

7. Compile the project to WASM and implement a transport layer so it can be used in browser. Something similar to [Yjs' WebRTC Connector](https://github.com/yjs/y-webrtc) could work.

[^1]: André, Luc, et al. "Supporting adaptable granularity of changes for massive-scale collaborative editing." 9th IEEE International Conference on Collaborative Computing: Networking, Applications and Worksharing. IEEE, 2013. 

[^2]: Kleppmann, Martin. "Making CRDTs Byzantine Fault Tolerant." Proceedings of the 9th Workshop on Principles and Practice of Consistency for Distributed Data. 2022.

## Acknowledgements

Thank you to [Nalin Bhardwaj](https://nibnalin.me/) for helping me with my cryptography questions and [Martin Kleppmann](https://martin.kleppmann.com/)

for his teaching materials and lectures which taught me a significant portion of what I've learned about distributed systems and CRDTs.