{"id":15111551,"url":"https://github.com/despreston/go-craq","last_synced_at":"2025-10-23T04:31:22.814Z","repository":{"id":48054568,"uuid":"330836212","full_name":"despreston/go-craq","owner":"despreston","description":"CRAQ (Chain Replication with Apportioned Queries) in Go","archived":false,"fork":false,"pushed_at":"2022-04-18T13:28:29.000Z","size":130,"stargazers_count":123,"open_issues_count":0,"forks_count":15,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-01-30T17:27:50.660Z","etag":null,"topics":["chain-replication","craq","databases","distributed-systems","golang","replication"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/despreston.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-01-19T02:00:30.000Z","updated_at":"2024-09-08T19:23:38.000Z","dependencies_parsed_at":"2022-08-12T17:40:11.656Z","dependency_job_id":null,"html_url":"https://github.com/despreston/go-craq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/despreston%2Fgo-craq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/despreston%2Fgo-craq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/despreston%2Fgo-craq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/despreston%2Fgo-craq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/despreston","download_url":"https://codeload.github.com/despreston/go-craq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":237780154,"owners_count":19365139,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["chain-replication","craq","databases","distributed-systems","golang","replication"],"created_at":"2024-09-26T00:21:06.968Z","updated_at":"2025-10-23T04:31:22.449Z","avatar_url":"https://github.com/despreston.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-craq [![Test Status](https://github.com/despreston/go-craq/workflows/Test/badge.svg)](https://github.com/despreston/go-craq/actions)  [![Go Reference](https://pkg.go.dev/badge/github.com/despreston/go-craq.svg)](https://pkg.go.dev/github.com/despreston/go-craq)\n\nPackage `go-craq` implements CRAQ (Chain Replication with Apportioned Queries)\nas described in [the CRAQ paper](https://pdos.csail.mit.edu/6.824/papers/craq.pdf). MIT Licensed.\n\nCRAQ is a replication protocol that allows reads from any replica while still\nmaintaining strong consistency. CRAQ _should_ provide better read throughput\nthan Raft and Paxos. Read performance grows linearly with the number of nodes\nadded to the system. Network chatter is significantly lower compared to Raft and\nPaxos.\n\n### Learn more about CRAQ\n[CRAQ Paper](https://pdos.csail.mit.edu/6.824/papers/craq.pdf)\n\n[Chain Replication: How to Build an Effective KV Storage](https://medium.com/coinmonks/chain-replication-how-to-build-an-effective-kv-storage-part-1-2-b0ce10d5afc3)\n\n[MIT 6.824 Distributed Systems Lecture on CRAQ (80mins)](http://nil.csail.mit.edu/6.824/2020/video/9.html)\n\n```\n            +------------------+\n            |                  |\n      +-----+   Coordinator    |\n      |     |                  |\nWrite |     +------------------+\n      |\n      v\n  +---+----+     +--------+     +--------+\n  |        +----\u003e+        +----\u003e+        |\n  |  Node  |     |  Node  |     |  Node  |\n  |        +\u003c----+        +\u003c----+        |\n  +---+-+--+     +---+-+--+     +---+-+--+\n      ^ |            ^ |            ^ |\n Read | |       Read | |       Read | |\n      | |            | |            | |\n      + v            + v            + v\n```\n\n## Processes\nThere are 3 packages that should be started to run the chain. The default node\nimplementation in [cmd/node](cmd/node) uses the Go net/rpc package and\n[bbolt](go.etcd.io/bbolt) for storage.\n\n### Coordinator\nFacilitates new writes to the chain; allows nodes to announce themselves to the\nchain; manages the order of the nodes of the chain. One Coordinator should be\nrun for each chain. For better resiliency, you _could_ run a cluster of\nCoordinators and use something like Raft or Paxos for leader election, but\nthat's outside the scope of this project.\n\n#### Run Flags\n```sh\n-a # Local address to listen on. Default: :1234\n```\n\n### Node\nRepresents a single node in the chain. Responsible for storing writes, serving\nreads, and forwarding messages along the chain. In practice, you would probably\nhave a single Node process running on a machine. Each Node should have it's own\nstorage unit.\n\n#### Run Flags\n```sh\n-a # Local address to listen on. Default: :1235\n-p # Public address reachable by coordinator and the other nodes. Default: :1235\n-c # Coordinator address. Default: :1234\n-f # Bolt DB database file. Default: craq.db\n```\n\n### Client\nBasic CLI tool for interacting with the chain. Allows writes and reads. The one\nincluded in this project uses the net/rpc package as the transport layer and\nbbolt as the storage layer.\n\n#### Run Flags\n```sh\n-c # Address of coordinator. Default: :1234\n-n # Address of node to send reads to. Default: :1235\n```\n\n#### Usage\n```sh\n./client write hello \"world\" # Write a new entry for key 'hello'\n./client read hello # read the latest committed version of key 'hello'\n```\n\n## Communication\n_go-craq_ processes communicate via RPC. The project is designed to be used with\nwhatever RPC system shall be desired. The basic default client included in the\ngo-craq package uses the net/rpc package from Go's stdlib; an easy-to-work-with\npackage with a great API.\n\n### Adding a New Transport Implementation\nPull requests for additional transport implementations are very welcome. Some\ncommon ones that would be great to have are gRPC and HTTP. Start by reading\nthrough [transport/transport.go](transport/transport.go). Use\n[transport/netrpc](transport/netrpc) as an example.\n\n## Storage\n_go-craq_ is designed to make it easy to swap the persistance layer. CRAQ is\nflexible and any storage unit that implements the `Storer` interface in\n[store/store.go](store/store.go) can be used. Some implementations for common\nstorage projects can be found in the `store` package. [store/kv](store/kv)\npackage is a _very simple_ in-memory key/value store that is included as an\nexample to work off of when adding new storage implementations.\n\n### Adding a New Storage Implementation\nPull requests for additional storage implementations are very welcome. Start by\nreading through the comments in [store/store.go](store/store.go). Use the\nstore/kv package as an example. CRAQ should work well with volatile and\nnon-volatile storage but mixing should be avoided or else you may end up seeing\nlong startup times due to data propagation. Mixing persistent storage mechanisms\nis an interesting idea I've been playing with myself. For example, one node\nstoring items in the cloud and another storing items locally.\n\n[store/storetest](store/storetest) should be used for testing new storage\nimplementations. Run the test suite like this:\n```go\nfunc TestStorer(t *testing.T) {\n\tstoretest.Run(t, func(name string, test storetest.Test) {\n    // New() is your store's constructor function.\n\t\ttest(t, New())\n\t})\n}\n```\n\n## Reading the Code\nThere are several places to start that'll give you a great understanding of how\nthings work.\n\n`connectToCoordinator` method in [node/node.go](node/node.go). This is the\nmethod the Node must run during startup to connect to the Coordinator and\nannounce itself to the chain. The Coordinator responds with some metadata about\nwhere in the chain the node is now located. The node uses this info to connect\nto the predecessor in the chain.\n\n`Update` method in [node/node.go](node/node.go). This is the method the\nCoordinator uses to update the node's metadata. New data is sent to the node if\nthe node's predecessor or successor changes, and if the address of the tail node\nchanges.\n\n`ClientWrite` method in [node/node.go](node/node.go). This is the method the\nCoordinator uses to send writes to the head node. This is where the chain begins\nthe process of propagation.\n\n## Q/A\n### What happens during a write?\nA write request containing the key and value are sent to the Coordinator via the\nCoordinator's `Write` RPC method. If the chain is not empty, the Coordinator\nwill pass the write request to the head node via the node's `ClientWrite`\nmethod.\n\nThe head node receives the key and value. The node looks up the key in it's\nstore to determine what the latest version should be. If the key already exists,\nthe version of the new write will be the existing version incremented by one.\nThe new value is written to the store. If the node is not connected to another\nnode in the chain, it commits the version. If the node does have a successor,\nthe key, value, and version are forwarded to the next node in the chain via the\nsuccessor's `Write` RPC method.\n\nThe key, value, and version are passed along the chain one-by-one. Each node\nadds the item to the store and sends a message to the successor in the chain.\nWhen the write reaches the tail node, the tail marks the item as committed. The\ntail sends a `Commit` RPC method to it's predecessor. The tail's predecessor\ncommits that version of the item, then continues to forward the `Commit` message\nbackwards through the chain, one node at a time, until every node has committed\nthe version.\n\n### What happens when a new node joins the chain?\nWhen the `node.Start` method is run, the Node will backfill it's list of latest\nversions for all committed items in it's store, then it'll connect to the\ncoordinator. Each Node stores the latest version of each committed item in it's\nstore in-memory. This is done so that if the node is or becomes the tail node,\nother nodes can query it for the latest committed version of an item. The\nresources at the top of the readme provide some info on why this is important,\nbut basically it helps ensure strong consistency.\n\nAfter backfilling the map of latest versions the Node connects to the\nCoordinator. The Coordinator adds the new Node to the list of Nodes in the\nchain, connects to the Node, responds with some metadata for the new Node, then\nsends updated metadata to the rest of the nodes in the chain to let it know\nthere's a new tail.\n\nThe metadata that the Coordinator sends back to the new Node includes info to\nlet the Node know if it's the head, the tail, and who it's predecessor in the\nchain is. The Node uses this info to connect to the predecessor in the chain.\n\nAfter connecting to the predecessor, the Node asks the predecessor for all items\nit has that a) the Node has no record of, or b) the Node has older versions of.\nThis ensures that the new Node is caught up with the chain. Because the new node\nis now the tail, any uncommitted items sent during propagation are immediately\ncommitted.\n\nOnce the Node is connected to it's neighbor and the coordinator, it starts\nlistening for RPCs. The RPC server is setup and started in [cmd/node](cmd/node).\n\n### Why store the latest committed versions in-memory?\nIt's worth mentioning that CRAQ works best with read-heavy workloads. One of\nit's best \"features\" is being able to read from any node in the chain. If a node\nreceives a read request for key Y and the latest version of key Y in the store\nis not committed, the node will send a request to the tail to ask for the latest\ncommitted version of key Y; this helps ensure that all reads to any node returns\nthe same value. In other words, it asserts strong consistency. As the chain\ngrows, it takes longer for an item to be committed and the probability of\nneeding to ask the tail for the latest version rises. Asking the tail for the\nlatest version can quickly become a bottleneck. Therefore, storing these latest\nversions in-memory affords us higher throughput from the tail for requests for\nthe latest version at the expense of slower startup times because the tail needs\nto backfill it's map of latest versions.\n\nIn the future, it may be beneficial to let the operator of the node signify\nwhether they'd like to backfill at startup or serve 'latest version' requests\ndirectly from the store.\n\n## Backlog\n- [ ] Benchmarks based off the tests in the paper, as close as reasonably possible.\n- [ ] gRPC transporter\n- [ ] HTTP transporter\n- [ ] Allow nodes to join at any location in the chain.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdespreston%2Fgo-craq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdespreston%2Fgo-craq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdespreston%2Fgo-craq/lists"}