{"id":22288290,"url":"https://github.com/jabolina/go-mcast","last_synced_at":"2025-07-28T22:32:29.852Z","repository":{"id":44816367,"uuid":"268154230","full_name":"jabolina/go-mcast","owner":"jabolina","description":"Golang based implementation of the Generic Multicast protocol.","archived":false,"fork":false,"pushed_at":"2022-08-27T22:29:56.000Z","size":1819,"stargazers_count":14,"open_issues_count":0,"forks_count":4,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-06-20T13:42:43.445Z","etag":null,"topics":["atomic-multicast","fault-tolerance","generic-multicast","multicast","state-machine-replication"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jabolina.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-05-30T20:25:56.000Z","updated_at":"2024-03-07T10:07:43.000Z","dependencies_parsed_at":"2022-08-25T12:01:18.051Z","dependency_job_id":null,"html_url":"https://github.com/jabolina/go-mcast","commit_stats":null,"previous_names":[],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jabolina%2Fgo-mcast","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jabolina%2Fgo-mcast/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jabolina%2Fgo-mcast/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jabolina%2Fgo-mcast/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jabolina","download_url":"https://codeload.github.com/jabolina/go-mcast/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":227961830,"owners_count":17847841,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["atomic-multicast","fault-tolerance","generic-multicast","multicast","state-machine-replication"],"created_at":"2024-12-03T17:03:33.558Z","updated_at":"2024-12-03T17:03:36.024Z","avatar_url":"https://github.com/jabolina.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-mcast\n![Go](https://github.com/jabolina/go-mcast/workflows/Go/badge.svg?branch=master)\n![MCast](https://circleci.com/gh/jabolina/go-mcast.svg?style=shield)\n\nGolang based implementation of the Generic Multicast protocol\n\n\n# Introduction\n\nTo build fault-tolerant application, one of the possible approach is the use of state machine replication, where \nan application is designed like a state machine and replicated to be executed across *N* replicas. Any system that can \nbe structured in terms of procedures and calls, can be structured using state machines and clients. \n\nTo create a fault tolerant application, is now possible to spreading the state machines across different processors. \nAll replicas are executed on a non-faulty processor starts at the state and all replicas must receive the same sequence \nof commands in the same order. To guarantee this is needed a replication coordination, that contains two requirements:\n\n- **Agreement:** *every* non faulty component receives *every* request\n- **Order:** *every* non faulty component receives *every* request in the same relative order\n\nThese requirements can be guaranteed when using an Atomic Broadcast implementation, a communication primitive \nensures that all *atomic broadcast* message will be delivered to all processes in the same order. Some well-known \nimplementations of Atomic Broadcast algorithms are *Raft* and *Paxos.* Using this protocol the requests will be \nordered across all system.\n\nThis approach will guarantee that the system is fault-tolerant, but not scale well, since all replicas must receive \nall requests. To tackle this problem, services shards their states thus only needing to have a partial ordering of \nrequests. Reliably delivering requests in partial order can be solved using an Atomic Multicast implementation, \nwhere only a subset of the system nodes receive the needed messages.\n\n### Why atomic multicast?\n\nWhen architecting an application is all about trade-offs, facing the decision about *consistency vs. scalability*, and some\napplications can choose between one or another. For an application that prioritize scalability, can be used an eventual\nconsistency protocol, that given a sharded data is consistent within partition but eventually consistent across partitions\nthat this data is sharded. This approach can lead to concurrent requests using stale information. \n\nSome applications have as requirements that the data must be always consistent, even across partitions. Most used protocols\ntotally order all requests, since the protocol enforces that **all** replicas must receive all requests regardless of the\nreplica content.\n\nUsing atomic broadcast, each request can be sent only to the replicas that are somehow involved in the request process.\nIn a place where the information is sharded across multiple replicas, this will be more efficient than ordering *every* \nrequest for *every* replica (why to shard if everyone is receiving requests?). Even though only some replicas are being \ninvolved in each request, all requests become ordered within and across partitions.\n\n\n# Atomic Multicast\n\nAn atomic multicast protocol, that can be executed on failure-free environment is the Skeen Algorithm, that uses logical \nclocks to give a sequence number for each message before delivering. Having a coordinator that sends the message to each \ndestination processes, then gather all timestamp that each process proposed and define a final timestamp. Using the final \ntimestamp the messages are sorted and delivered.\n\nA fault-tolerant version of the Skeen's algorithm, proposed by Fritzke, uses group of processes instead of single \nprocesses on the destination end. When the coordinator sends the message to the destination, is not a single destination \nprocess, is now a group of processes that works like a unity. Using this approach, even on the face of failures the \nprotocol can be considered reliable if exist a quorum of correct processes on the unity destination. On this protocol, \nfirst each group will agree on the group timestamp, then the groups will exchange the timestamp between them and after \nthis the maximum value is used as the final timestamp, and like the Skeen's algorithm, the messages are sorted and \ndelivered accordingly with the final timestamp.\n\nSome improvements were proposed by Schiper and Pedone on the Fritzke algorithm, about some practical aspects of the \natomic multicast algorithm and some impossibilities. It is well-known in a system with two or more processes in which \none process can crash is impossible to solve the atomic multicast using a genuine implementation without a perfect \nfailure detector. From this impossibility exists a corollary that atomic multicast is strictly harder than atomic broadcast.\n\n# Generic Multicast\n\nThe atomic multicast, even with the proposed improvements still can be an expensive primitive communication to build \nreliable and fault-tolerant applications, since all messages will be ordered, when a better approach would be to order \nonly messages that need to be ordered. Using this method for relaxing the messages delivery order is the Generic Multicast, \nthis can be interesting since is possible that only conflicting messages can be ordered and is possible to declare this \nrelationship at application level, suiting best what is going to be built. Generic Multicast is defined by the primitives \n*GM-Cast* and *GM-Deliver* following properties:\n\n- *Validity*: If a correct process *GM-Cast* a message *m* to *Dest\u003cm\u003e*, then some process in *Dest\u003cm\u003e* eventually *GM-Deliver m;*\n- *Agreement*: If a correct process *GM-Deliver* a message *m,* then all correct processes in *Dest\u003cm\u003e* eventually *GM-Deliver\u003cm\u003e;*\n- *Integrity*: For any message *m*, every correct process in *Dest\u003cm\u003e* *GM-Deliver m* at most once, and only if *m* was previously *GM-Cast* by some process;\n- *Partial Order*: If correct processes *p* and *q* both *GM-Deliver* messages *m* and *m'* and *p* and *q* belongs to *Dest\u003cm\u003e* united with *Dest\u003cm'\u003e* and *m* and *m'* conflicts, then *p GM-Deliver m* before *m'* iff *q* *GM-Deliver m* before *m'.*\n\nThe current project aims to implement a Generic Atomic Multicast that can be used as a primitive to build fault-tolerant \napplications. The algorithm here was proposed by Douglas Antunes and Lasaro Camargos at Universidade Federal de Uberlândia.\n\n### Model and primitives\n\nThis work was done considering a set of processes that can fail by crashing or not failing at all, hence processes do \nnot behave maliciously. Communication is done using message on transport channels that do not corrupt nor duplicate \nmessages and which is quasi-reliable. The system is asynchronous with unreliable failure detectors.\n\nSince communication is done using message passing, some communication primitives are used. The first and more simple one is:\n\n- *Send\u003cm, p\u003e*: send message *m* to process *p;*\n- *Recv\u003cm\u003e*: happens at the destination when a new message is received;\n\nBoth of the above primitives are unreliable and can be used the existent primitives from the transport layer for \nprocess communication. This is used to enable unicast process communication, when a message needs to be sent to a \nsingle process and there is no need for reliability.\n\nAnother primitive needed and used for group communication are:\n\n- *GB-Cast\u003cm\u003e* and *GB-Deliver\u003cm\u003e*\n\nThis primitive will be used for group communication inside a process unity. This primitive is *Generic Broadcast,* \nthat work similar as primitives like *Atomic Broadcast* but prevent the ordering of message when not necessary, taking \ninto account the semantics of the messages to establish a partial order on message delivery. This primitive is specified \nby a conflict relationship *C* and the following conditions:\n\n- *Validity*: If a correct process *GB-Cast* a message *m*, then eventually *GB-Deliver* m;\n- *Agreement*: If a correct process *GB-Deliver* a message *m*, then all correct processes eventually *GB-Deliver* *m;*\n- *Integrity*: For any message *m*, every correct process *GB-Deliver m* at most once, and only if *m* was previously *GB-Cast* by some process;\n- *Partial Order*: If the correct processes *p* and *q* both *GB-Deliver* messages *m* and *m'*, and *m* and *m'* conflict, then *p GB-Deliver m* before *m'* iff *q GB-Deliver m* before *m'*;\n\nUsing a Total Order Broadcast is possible to solve the Generic Broadcast problem, but is not the ideal solution.\n\n# References\n\nPEDONE, F.; SCHIPER, A. Generic broadcast. In: SPRINGER. International Symposium on Distributed Computing. [S.l.], 1999. p. 94–106.\n\nGUERRAOUI, R.; SCHIPER, A. Genuine atomic multicast in asynchronous distributed systems. 2001. Theoretical Computer Science, 254(1–2), 297–316.\n\nAHMED-NACER, T.; SUTRA, P.; CONAN, D. The convoy effect in atomic multicast. In: IEEE. 2016 IEEE 35th Symposium on \nReliable Distributed Systems Workshops (SRDSW). [S.l.], 2016. p. 67–72.\n\nANTUNES, D. A Fault-Tolerant Generic Multicast Algorithm for Wide Area Networks. 2019.\n\nBENZ, S.; MARANDI, P. J.; PEDONE, F.; GARBINATO, B. Building global and scalable systems with atomic multicast. 2014. In Middleware \nConference 2014 - Proceedings of the Posters and Demos Session (pp. 11–12). Association for Computing Machinery, Inc.\n ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjabolina%2Fgo-mcast","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjabolina%2Fgo-mcast","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjabolina%2Fgo-mcast/lists"}