{"id":16819802,"url":"https://github.com/nvanbenschoten/epaxos","last_synced_at":"2025-03-22T03:31:39.964Z","repository":{"id":56582791,"uuid":"88163031","full_name":"nvanbenschoten/epaxos","owner":"nvanbenschoten","description":"A pluggable implementation of the Egalitarian Paxos Consensus Protocol","archived":false,"fork":false,"pushed_at":"2020-10-30T08:26:36.000Z","size":48572,"stargazers_count":61,"open_issues_count":1,"forks_count":11,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-03-18T07:43:15.246Z","etag":null,"topics":["consensus","distributed-systems","egalitarian","paxos","replication"],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nvanbenschoten.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-13T12:34:15.000Z","updated_at":"2025-02-03T12:17:32.000Z","dependencies_parsed_at":"2022-08-15T21:20:53.230Z","dependency_job_id":null,"html_url":"https://github.com/nvanbenschoten/epaxos","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvanbenschoten%2Fepaxos","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvanbenschoten%2Fepaxos/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvanbenschoten%2Fepaxos/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nvanbenschoten%2Fepaxos/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nvanbenschoten","download_url":"https://codeload.github.com/nvanbenschoten/epaxos/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244902929,"owners_count":20529114,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consensus","distributed-systems","egalitarian","paxos","replication"],"created_at":"2024-10-13T10:54:40.164Z","updated_at":"2025-03-22T03:31:34.955Z","avatar_url":"https://github.com/nvanbenschoten.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Egalitarian Paxos\n\n_A pluggable implementation of the Egalitarian Paxos Consensus Protocol_\n\nPaxos is a protocol for solving consensus through state machine replication in\nan asynchronous environment with unreliable processes. It can tolerate up to F\nconcurrent replica failures with 2F+1 total replicas. This consensus protocol is\nthen extended with a stable leader optimization to a replication protocol\n(commonly referred to as Multi-Paxos) to assign global, persistent, total order\nto a sequence of client updates. The protocol works by having multiple replicas\nwork in parallel to maintain the same state. This state is updated on each\nrequest from a client by each replica, allowing it to be automatically\nreplicated and preserved even in the case of failures. The basic algorithm was\nfamously described by Leslie Lamport in his 1998 paper, [The Part-Time\nParliament](https://www.microsoft.com/en-us/research/publication/part-time-parliament/).\nIt was later clarified in his follow-up paper from 2001, [Paxos Made\nSimple](https://www.microsoft.com/en-us/research/publication/paxos-made-simple/).\n\nEgalitarian Paxos is an efficient, leaderless variation of this protocol,\nproposed by Iulian Moraru in his 2013 paper, [There Is More Consensus in\nEgalitarian\nParliaments](https://www.cs.cmu.edu/~dga/papers/epaxos-sosp2013.pdf).\nIt provides strong consistency with optimal wide-area latency, perfect\nload-balancing across replicas (both in the local and the wide area), and\nconstant availability for up to F failures. Concretely, it provides the\nfollowing properties:\n\n- High throughput, low latency\n- Constant availability\n- Load distributed evenly across all replicas (no leader)\n- Limited by fastest replicas, not slowest\n- Can always use closest replicas (low latency)\n- 1 round-trip fast path\n\nIt does so by breaking the global command slot space into subspaces, each owned\nby a single replica. Replicas then attach ordering constraints to each command\nwhile voting on them to allow for proper ordering during command execution. For\nmore intuition on how this works, check out the [presentation given at SOSP\n'13](https://www.youtube.com/watch?v=KxoWlUZNKn8), and for a full technical\nreport and proof of correctness of the protocol, check out [A Proof of\nCorrectness for Egalitarian\nPaxos](http://www.pdl.cmu.edu/PDL-FTP/associated/CMU-PDL-13-111.pdf).\n\nThis library is implemented with a minimalistic philosophy. The main `epaxos`\npackage implements only the core EPaxos algorithm, with storage handling,\nnetwork transport, and physical clocks left to the clients of the library. This\nminimalism buys flexibility, determinism, and performance. The design was\nheavily inspired by [CoreOS's raft\nlibrary](https://github.com/coreos/etcd/tree/master/raft).\n\n\n## Features\n\nThe epaxos implementation is a full implementation of the Egalitarian Paxos\nreplication protocol. Features include:\n\n- Command replication\n- Command compaction\n- Persistence\n- Failure Recovery\n\nFeatures not yet implemented:\n\n- Explicit Prepare Phase\n- Optimized Egalitarian Paxos (smaller fast path quorum)\n- Membership changes\n- Batched commands\n- Thrifty operation (see paper)\n- [Quorum leases](https://www.cs.cmu.edu/~dga/papers/leases-socc2014.pdf)\n- Snapshots\n\n\n## Building\n\nRun `make` or `make test` to run all tests against the library \n\nRun `make clean` to clean all build artifacts\n\nRun `make check` to perform linting and static analysis\n\n\n## Testing\n\nThe project comes with an automated test suite which contains both direct unit\ntests to test pieces of functionality within the EPaxos state machine, and larger\nnetwork tests that test a network of EPaxos nodes. The unit tests are scattered\nthroughout the `epaxos/*_test.go` files, while the network tests are located in\nthe `epaxos/epaxos_test.go` file.\n\nTo run all tests, run the command `make test`\n\n\n## Library Interface\n\nThe library is designed around the the `epaxos` type, which is a single-threaded\nstate machine implementing the Egalitarian Paxos consensus protocol. The state\nmachine can be interacted with only through a `Node` instance, which is a\nthread-safe handle to a `epaxos` state machine.\n\nBecause the library pushes tasks like storage handling and network transport up\nto the users of the library, these users have a few responsibilities. In a loop,\nthe user should read from the `Node.Ready` channel and process the updates it\ncontains. These `Ready` struct will contain any updates to the persistent state\nof the node that should be synced to disk, and messages that need to be\ndelivered to other nodes, and any commands that have been successfully committed\nand that are ready to be executed. The user should also periodically call\n`Node.Tick` in regular interval (probably via a `time.Ticker`).\n\nTogether, the state machine handling loop will look something like:\n\n```\nfor {\n    select {\n    case \u003c-ticker.C:\n        node.Tick()\n    case rd := \u003c-node.Ready():\n        for _, msg := range rd.Messages {\n            send(msg)\n        }\n        for _, cmd := range rd.ExecutableCommands {\n            execute(cmd)\n        }\n    case \u003c-ctx.Done():\n        return\n    }\n}\n```\n\nTo propose a change to the state machine, first construct a `pb.Command`\nmessage. The `pb.Command` message contains both an arbitrary byte slice to hold\nclient updates and additional metadata fields to related to command\ninterference. Use of these metadata fields in `pb.Command` is the mechanism in\nwhich clients of the library express application-specific command interference\nsemantics. `pb.Commands` operate in a virtual keyspace, and each command\noperates over a subset of this keyspace, which is expressed in the `Span` field.\n`pb.Commands` can also be specified as reads or writes using the `Writing`\nfield. Interference between commands is then defined as two commands whose\n`Spans` overlap, where at-least one of the commands is `Writing`.\n\nAfter a `pb.Command` is constructed with the desired update and the necessary\ninterference constrains, call:\n\n```\nnode.Propose(ctx, command)\n```\n\nOnce executable, the `pb.Command` will appear in the `rd.ExecutableCommands` slice.\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvanbenschoten%2Fepaxos","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnvanbenschoten%2Fepaxos","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnvanbenschoten%2Fepaxos/lists"}