{"id":24164481,"url":"https://github.com/paulja/go-work","last_synced_at":"2026-05-13T06:02:08.982Z","repository":{"id":271228036,"uuid":"907485400","full_name":"paulja/go-work","owner":"paulja","description":"Go project that schedules workers and uses a heartbeat all over gRPC","archived":false,"fork":false,"pushed_at":"2025-02-12T13:12:44.000Z","size":101,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-12T14:27:31.718Z","etag":null,"topics":["grpc"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":false,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/paulja.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-23T17:29:23.000Z","updated_at":"2025-02-12T13:12:05.000Z","dependencies_parsed_at":"2025-01-06T12:31:34.940Z","dependency_job_id":"1ead4710-4fe4-449a-8f4d-f51a60656db0","html_url":"https://github.com/paulja/go-work","commit_stats":null,"previous_names":["paulja/go-work"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulja%2Fgo-work","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulja%2Fgo-work/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulja%2Fgo-work/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/paulja%2Fgo-work/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/paulja","download_url":"https://codeload.github.com/paulja/go-work/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241470363,"owners_count":19968041,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["grpc"],"created_at":"2025-01-12T19:17:26.076Z","updated_at":"2026-05-13T06:02:03.939Z","avatar_url":"https://github.com/paulja.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# go-work\n\nThis a repository to demonstrate how to have cluster of workers in go.\n### Design\n\nThe Leader performs two main functions: heartbeat management i.e. the health of the workers in the cluster, and scheduling new work based the workers available.\n\n```mermaid\nflowchart LR\n\nsubgraph leader\n\tc[cluster]\n\ts[scheduler]\nend\n\nw1[\"worker..n\"]\n\nw1-- health --\u003ec\nw1-- join / leave --\u003ec\nw1-- work complete --\u003es\n\ns-- start/cancel work --\u003ew1\nc-- members --\u003es\n```\n\nThere are 3 core contracts:\n\n- cluster - the cluster manages joining and leaving the cluster and the health of workers.\n- scheduler - the scheduler manages the receiving of work request and the dishing out to the workers.\n- worker - the worker receives commands to start (or stop) work, and then reports health and completion of work to the cluster and schedule respectively.\n\nAll the communication is over gRPC (non-TLS in this example) using client and server RPC calls (non streaming).\n\n### Testing\n\nThe test coverage in the codebase is relatively high, however, there are a few gaps that can be closed, the plan is to close these in the not too distant future.\n\n### Future\n\n- [x] Write a CLI to get a list of members, tasks and schedule work\n  - [x] The system hangs after work is scheduled and you try to list tasks\n  - [x] Only one worker is scheduled to work no matter if more are free\n- [ ] Put the comms over mTLS (outside of the implementation code, i.e. NGINX or Envoy)\n- [x] Covert the heartbeat comms between the cluster leader and the members (workers) over a stream\n- [ ] Create a consumer library to enable custom code to become a worker\n- [ ] Get it working in ECS\n- [ ] Implement RAFT and have multiple leaders\n- [ ] The current task list is an in-memory linked list, make more resilient\n- [ ] There are some TODOs in the code, work to remove them (mainly tests)\n\n\u003e [!note]\n\u003e I implemented streaming of the heartbeat and the code for the client was unpleasant, therefore, I decided to not merge the changes. I have left the branch intact in `dev/streaming` for the curious.\n\n### Useful Commands\n\nList all the operations available on the Leader server.\n\n```sh\ngrpcurl -plaintext :50051 describe cluster.v1.LeaderService\n```\n\nList all the operations available on the Scheduler server.\n\n```sh\ngrpcurl -plaintext :50052 describe scheduler.v1.SchedulerService\n```\n\nGet the list of worker members in the cluster.\n\n```sh\ngrpcurl -plaintext :50051 cluster.v1.LeaderService.Members\n```\n\nGet the list of tasks pushed through the leader.\n\n```sh\ngrpcurl -plaintext :50052 scheduler.v1.SchedulerService.GetTasks\n```\n\nAdd a task to the scheduler.\n\n```sh\ngrpcurl -plaintext -d '{\"task\": {\"id\":\"1\",\"payload\":\"do some work\"}}' :50052 scheduler.v1.SchedulerService.ScheduleTask\n```\n\nCancel a task from the scheduler.\n\n```sh\ngrpcurl -plaintext -d '{\"id\":\"1\"}' :50052 scheduler.v1.SchedulerService.CancelTask\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulja%2Fgo-work","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpaulja%2Fgo-work","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpaulja%2Fgo-work/lists"}