{"id":15914318,"url":"https://github.com/bbengfort/zmqnet","last_synced_at":"2025-04-03T03:26:15.884Z","repository":{"id":69222882,"uuid":"97057902","full_name":"bbengfort/zmqnet","owner":"bbengfort","description":"A fully connected ZMQ topology","archived":false,"fork":false,"pushed_at":"2017-08-01T16:51:54.000Z","size":1599,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-02-08T17:28:16.383Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Jupyter Notebook","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bbengfort.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-12T22:43:34.000Z","updated_at":"2018-05-12T21:32:04.000Z","dependencies_parsed_at":"2023-04-09T03:15:53.995Z","dependency_job_id":null,"html_url":"https://github.com/bbengfort/zmqnet","commit_stats":{"total_commits":13,"total_committers":1,"mean_commits":13.0,"dds":0.0,"last_synced_commit":"2ac9fb504cd9158a6f11cf5027a60dc8568cf93c"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fzmqnet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fzmqnet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fzmqnet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bbengfort%2Fzmqnet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bbengfort","download_url":"https://codeload.github.com/bbengfort/zmqnet/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246930035,"owners_count":20856526,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-06T17:01:42.924Z","updated_at":"2025-04-03T03:26:15.857Z","avatar_url":"https://github.com/bbengfort.png","language":"Jupyter Notebook","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ZMQ Net\n\n**Tests for a fully connected ZMQ topology**\n\n![Topologies](fixtures/zmqnet.png)\n\nTrying to figure out all the patterns for REQ/REP/ROUTER,DEALER, etc. in order to get a fully connected network topology where all replicas respond to all other replicas and can make a request of any/all other replicas. My original idea was that I could create a REP on each node and a REQ for each other node (requiring full awareness of the topology). This would give us push based messaging. Alternatively perhaps we could create a PUB for each node and have all other replicas SUB to each other; more a push based messaging system.\n\nBecause Raft/HC requires RPC calls (e.g. request/response) - I went with the push based messaging model to begin with (the figure on the left).\n\n## Current Status\n\nRight now the network is the primary object. Each process creates a network object such that it has a `local` identification for the server process (REP) and `remotes` to connect to other process (REQ). When the network is run, it instantiates all of these connections.\n\nAt this point a `client` can send a message (REQ) to any of the processes in the network. When a client message is received (REP) it is broadcast (bounced) to all other processes in the network (REQ), who acknowledge receipt of the message. Once all processes have responded a REP is sent to the client.\n\n![Topologies](fixtures/zmqnet-broadcast.png)\n\n\u003cs\u003eThe current state of this network is not fault tolerant, ZMQ will block until all messages are received and sent. I need to implement request timeouts so\nthat any node that is down in the network does not block the client or server processes.\u003c/s\u003e Currently, I've implemented [Lazy Pirate](http://zguide.zeromq.org/go:lpclient) with a request timeout and limited number of retries. The connection is just reset in a brute force way if there is no response from the server. If the server comes back online during retries then the message is handled.\n\nThis all seems to be working well, except in Broadcast (no retries). For some reason, I'm occasionally getting `\u003cNONE\u003e` for the `sock.Events` variable using `PollAll` and I'm not sure why.\n\nIt would be nice if there was a ROUTER/DEALER pattern with no in the middle broker ...\n\n## Getting Started\n\nTo run the network code, first get and install the package:\n\n    $ go get github.com/bbengfort/zmqnet/...\n\nThen create a peers.json file to define the network as follows:\n\n```json\n{\n  \"info\": {\n    \"num_replicas\": 3,\n    \"updated\": \"2017-07-13T13:36:53.158Z\"\n  },\n  \"replicas\": [\n    {\n      \"pid\": 1000,\n      \"name\": \"alpha\",\n      \"address\": \"localhost:3264\",\n      \"host\": \"apollo\",\n      \"ipaddr\": \"127.0.0.1\",\n      \"port\": 3264\n    },\n    {\n      \"pid\": 1001,\n      \"name\": \"bravo\",\n      \"address\": \"localhost:3265\",\n      \"host\": \"apollo\",\n      \"ipaddr\": \"127.0.0.1\",\n      \"port\": 3265\n    },\n    {\n      \"pid\": 1002,\n      \"name\": \"charlie\",\n      \"address\": \"localhost:3266\",\n      \"host\": \"apollo\",\n      \"ipaddr\": \"127.0.0.1\",\n      \"port\": 3266\n    }\n  ]\n}\n```\n\nYou can save this file anywhere and either pass its location to the network or expose the path using the `$PEERS_PATH` environment variable.\n\nIn separate terminals, start each individual process by name:\n\n    $ zmqnet -p peers.json serve -n alpha\n\n    $ zmqnet -p peers.json serve -n bravo\n\n    $ zmqnet -p peers.json serve -n charlie  \n\n\nThen you can use the client in yet a fourth terminal to create messages for them to broadcast:\n\n    $ zmqnet -p peers.json send -n bravo \"this is a test message\"\n\nBoth the `serve` and `send` commands require a specification of which host to use.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbengfort%2Fzmqnet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbbengfort%2Fzmqnet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbbengfort%2Fzmqnet/lists"}