{"id":31841907,"url":"https://github.com/robertobatts/distributed-counter","last_synced_at":"2026-07-03T16:33:37.888Z","repository":{"id":119190782,"uuid":"210691040","full_name":"robertobatts/distributed-counter","owner":"robertobatts","description":"Prototype of a distributed system using primary/backup replication, exposed as an API that allows you to store objects and to count them","archived":false,"fork":false,"pushed_at":"2019-09-26T21:07:25.000Z","size":27,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-10-19T14:58:26.827Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/robertobatts.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-24T20:23:07.000Z","updated_at":"2019-10-13T12:03:13.000Z","dependencies_parsed_at":null,"dependency_job_id":"29c8e7b5-d322-4535-9c4d-04fdb62310df","html_url":"https://github.com/robertobatts/distributed-counter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/robertobatts/distributed-counter","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robertobatts%2Fdistributed-counter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robertobatts%2Fdistributed-counter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robertobatts%2Fdistributed-counter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robertobatts%2Fdistributed-counter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/robertobatts","download_url":"https://codeload.github.com/robertobatts/distributed-counter/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/robertobatts%2Fdistributed-counter/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":35094063,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-07-03T02:00:05.635Z","response_time":110,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-12T05:55:12.120Z","updated_at":"2026-07-03T16:33:37.865Z","avatar_url":"https://github.com/robertobatts.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# distributed-counter\n\n## Run \n\n```\ngo build coordinator.go\n\n./coordinator\n```\n\nThis is how you have to send messages to save items into the nodes:\n\n```bash\nPOST http://localhost:8085/items\n[\n    {\n        \"ID\": 1,\n        \"tenant\": \"hello\"\n    },\n    {\n        \"ID\": 2,\n        \"tenant\": \"publicsonar\"\n    }\n]\n```\n\nThis is how you retrieve the counter of the number of items grouped by tenant:\n\n```bash\nGET http://localhost:8085/items/{tenant}/count\n```\n\n\n\n## How it works\n\n### Start\n\nWhen the program starts, the Coordinator initializes 4 nodes in a safe way (if a port is busy, it tries automatically with another). All the nodes listen on a TCP port and one of them is the Master\n\n### Update/Insert (POST /items)\n\nThe Coordinator distributes the items among the nodes, then it call the nodes parallelly so that they can save the items in their memory.  After saving the items, the nodes become again available to be called by the Coordinator while they're sending their local memory to the Master. This is an optimization, in fact the user doesn't have to wait for the data replication to receive his http response, although the data are replicated at every request.\n\n#### Load Balancing\n\nThe items are distributed equally among the nodes. For example, if I have 4 nodes and 11 items the Coordinator distributes them in this way: \n\n```\nNode1: 3,    Node2: 3,        Node3: 3,       Node4: 2\n```\n\n\n\n#### Error handling\n\n- If a node is already busy when the Coordinator calls it, it start looking for an available node to call, until it found one\n- If a slave node crashes, there is no data loss because the items are saved in the Master after every request\n- If the items sent by the user are not valid, the Coordinator sends an http error\n\n #### Storage handling\n\nThe items are collected into a map (key=ID, value=item). At the moment of the storage the *LastUpdateDt* is saved in the item as an additional field, so that if multiple nodes send the same ID simultaneously with different tenants, the Master node can understand which is the last updated value to save in its memory.\n\n### Read (GET /items/{tenant}/count)\n\nThe Coordinator calls the Master node to received the number of items grouped by tenant. The Master is the only node to be query-able because the items are not replicated in the slave nodes during the POST operations\n\n#### Error handling\n\n- If the master node crashes because of network failure, it starts listening automatically to another port (see func Run()). When the Coordinator tries to call it and receives an error, it detects that the port is been changed and it tries calling the new port. In this way the system is still query-able if the Master crashed during a GET operation \n\n\n\n## What to improve\n\n- Data replication: the data should be replicated to every node. In the current system the data are safe from network failures in the master node, but if it crashes for unexpected reasons the system is no more query-able. This could be fixed by implementing something similar to the functions that I used to merge the Master memory from the slaves. It's just a little more complex because we have to deal with merging all the nodes ","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobertobatts%2Fdistributed-counter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frobertobatts%2Fdistributed-counter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frobertobatts%2Fdistributed-counter/lists"}