https://github.com/luca-moser/pool
worker pool implementation for Go.
https://github.com/luca-moser/pool
concurrency go golang parallelization worker-pool
Last synced: 3 months ago
JSON representation
worker pool implementation for Go.
- Host: GitHub
- URL: https://github.com/luca-moser/pool
- Owner: luca-moser
- License: mit
- Created: 2017-06-19T14:59:08.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2017-06-28T19:19:39.000Z (almost 8 years ago)
- Last Synced: 2025-01-10T18:21:59.488Z (5 months ago)
- Topics: concurrency, go, golang, parallelization, worker-pool
- Language: Go
- Homepage:
- Size: 22.5 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Worker Pool [](https://travis-ci.org/luca-moser/pool)
A simple worker pool which receives jobs and distributes them to workers.
Workers can send errors and results back to the pool creator through channels or handler functions.
## Usage
```go
// new pool with 8 worker goroutines
workerPool, err := pool.NewWorkerPool(8)
if err != nil {
...
}
const iterationCount = 100
var processed int
// called when a result was produced by a worker
resultHandler := func(result interface{}) {
// no mutex needed here as the resultHandler is not called concurrently
processed += result.(int)
}
// called when an error was produced by a worker
errorHandler := func(err error) {
t.Fatal(err)
}
// define the handler functions on the pool
workerPool.AddHandlers(resultHandler, errorHandler)
// OR discard any results and errors (not to be used with AddHandlers simultaneously(!))
// workerPool.Discard()
// add jobs to the pool
for i := 0; i < iterationCount; i++ {
// add a function to the pool for execution which returns a result or error
workerPool.AddFuncWithResult(f)
// OR a function which does not return anything, except maybe an error
// workerPool.AddFunc(func() error {})
// OR a job
// workerPool.AddJob(Job{
// Function: func(args ...interface{}) (interface{}, error) {
// return nil, nil
// },
// Arguments: []interface{}{1,2,3,4},
// })
}
// wait for the amount of jobs to complete
workerPool.Wait(iterationCount)
// free up goroutines and close channels
workerPool.Stop()
// returns information about each worker's processed jobs
workerPool.Stats()
```
### Job results and errors
It's important that the error and result channels are being actively used or discarded.
This can be achieved by using the `Errors()` and `Results()` functions which give you channels
to listen for or `DiscardErrors()` and `DiscardResults()` or `Discard()` (errors and results combined).
By using the `AddHandlers` function handlers can be defined which are executed as soon as results and errors are
produced by workers.
### Other
If a worker panics during the execution of the job, an error will be generated, the worker will then resume normally.