https://github.com/gasparian/worker-pool-go
Implementation of worker pool with a variable balancing strategy
https://github.com/gasparian/worker-pool-go
go go-concurrency go-workerpool go-workers-pool golang worker-pool worker-pool-go
Last synced: 4 months ago
JSON representation
Implementation of worker pool with a variable balancing strategy
- Host: GitHub
- URL: https://github.com/gasparian/worker-pool-go
- Owner: gasparian
- License: mit
- Created: 2021-05-29T09:39:01.000Z (about 5 years ago)
- Default Branch: main
- Last Pushed: 2022-01-30T21:09:18.000Z (over 4 years ago)
- Last Synced: 2025-08-09T21:38:24.466Z (10 months ago)
- Topics: go, go-concurrency, go-workerpool, go-workers-pool, golang, worker-pool, worker-pool-go
- Language: Go
- Homepage:
- Size: 27.3 KB
- Stars: 1
- Watchers: 1
- Forks: 1
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# worker-pool-go
Implementation of worker pool with extendable functionality.
By default, I've implemented "round-robin" balancing to evenly distribute jobs across workers.
You need to implement the `BalancingStrategy` interface in order to use other balancing strategies:
```go
type BalancingStrategy interface {
NextWorkerId(workersStats []Stats) int
}
```
API:
```go
// worker pool struct private api
(wp *WorkerPool) getCurrentJobsNumber() int64
(wp *WorkerPool) getWorkerStats(workerId int) (Stats, error)
(wp *WorkerPool) terminateWorker(workerId int) error
(wp *WorkerPool) reloadWorker(workerId int) error
// worker pool public methods
// New creates new instance of workers pool
// params defining it's size and max jobs per worker (without blocking)
NewWorkerPool(nWorkers, maxJobs uint, balancer BalancingStrategy) *WorkerPool
// ScheduleJob puts new job in some worker queue
(wp *WorkerPool) ScheduleJob(f JobFunc) (chan Result, error)
// NewRoundRobin creates new instance of the "load balancer"
NewRoundRobin() *RoundRobin
// NextWorkerId returns worker id selected by some
// predefined policy
(rr *RoundRobin) NextWorkerId(workerStats []Stats) int
```
Install:
```
go get -u github.com/gasparian/worker-pool-go
```
Usage example:
```go
package main
import (
"fmt"
wp "github.com/gasparian/worker-pool-go"
)
// Use closure to create job function
func exampleJob(inp int) wp.JobFunc {
return func() wp.Result {
res := inp * inp
return wp.Result{
Data: res,
Err: nil,
}
}
}
func main() {
pool := NewWorkerPool(
3,
10,
NewRoundRobin(),
)
// NOTE: if number of jobs per woker will be > MaxJobs,
// ScheduleJob func will block
nJobs := 50
results := make([]chan wp.Result, 0)
for i := 0; i < nJobs; i++ {
ch := pool.ScheduleJob(exampleJob(i))
results = append(results, ch)
}
for _, r := range results {
res := <-r
fmt.Println(res)
}
}
```
Run tests:
```
make test
```