https://github.com/dlemel8/consistent_hashing_workers

compare consistent hashing implementations to solve distributed continues jobs processing problem
https://github.com/dlemel8/consistent_hashing_workers

consistent-hashing golang rabbitmq zeromq

Last synced: 13 days ago
JSON representation

compare consistent hashing implementations to solve distributed continues jobs processing problem

• Host: GitHub
• URL: https://github.com/dlemel8/consistent_hashing_workers
• Owner: dlemel8
• Created: 2020-09-12T16:55:45.000Z (over 5 years ago)
• Default Branch: master
• Last Pushed: 2020-12-10T06:10:50.000Z (over 5 years ago)
• Last Synced: 2025-02-23T18:45:40.792Z (about 1 year ago)
• Topics: consistent-hashing, golang, rabbitmq, zeromq
• Language: Go
• Homepage:
• Size: 506 KB
• Stars: 0
• Watchers: 2
• Forks: 1
• Open Issues: 0
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# consistent_hashing_workers

![Test](https://github.com/dlemel8/consistent_hashing_workers/workflows/Verify%20Strategies/badge.svg)

This repo contains 3 services:

* Generator is generating jobs with an id and data

* Workers are processing the jobs and send their results to reporter

* Reporter is aggregating jobs results and then saves a report

In steady state, requirements are:

* Each job id is processed by the same worker

* Workload is evenly distributed among the workers

## Abstract Design

![Abstract](design/abstract.jpg?raw=true "Abstract")

## Implementations

### RabbitMQ (Centralized Message Broker)

![RabbitMQ](design/rabbitmq.jpg?raw=true "RabbitMQ")

### ZeroMQ (Distributed Messaging Sockets)

![ZeroMQ](design/zeromq.jpg?raw=true "ZeroMQ")