https://github.com/mavolin/worker
👪 Worker Pool and Concurrency Limiter with error and result collection.
https://github.com/mavolin/worker
Last synced: 2 months ago
JSON representation
👪 Worker Pool and Concurrency Limiter with error and result collection.
- Host: GitHub
- URL: https://github.com/mavolin/worker
- Owner: mavolin
- License: mit
- Created: 2023-03-12T23:51:59.000Z (about 2 years ago)
- Default Branch: v1
- Last Pushed: 2023-06-18T18:16:18.000Z (almost 2 years ago)
- Last Synced: 2025-01-10T02:47:53.867Z (4 months ago)
- Language: Go
- Homepage:
- Size: 33.2 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
worker
[](https://github.com/mavolin/worker/actions)
[](https://codecov.io/gh/mavolin/worker)
[](https://goreportcard.com/report/github.com/mavolin/worker)
[](https://github.com/mavolin/worker/blob/develop/LICENSE)
---
## About
`worker` is a small library that provides a concurrency limiter and a worker pool.
## Main Features
**`Limiter`**
> Limits the amount of times a task is executed at once.
> * ⤵ Tasks are blockingly executed in the same goroutine as the caller
> * 👮 Ensures has a task is only executed n-times at the same time
> * 🔄 Tasks are executed in the order they are added in
> * 🚚 Result collection and error handling is done by the caller
> * ⭐ Support for `context.Context`
**`Pool` and `ResultPool`**
> Execute multiple tasks at the same time.
> * ⤴ Task are executed in a separate goroutine, with max. n+1 goroutines running
> * 👮 Optionally limit the amount of concurrently running workers
> * 🔄 Tasks are executed in the order they are added in
> * 🚫 Tasks can abort execution of other tasks
> * 🚚 Result collection and error handling is done by the pool
> * ⭐ Support for `context.Context`
> * ⌚ `Wait()` to await the completion of all tasks
## Examples
First impressions matter, so here are examples of a limiter and a pool:
```go
var l = worker.NewLimiter(5)
// veryExpensiveComputation does a very expensive computation and returns with
// the result.
// At any given time only up to 5 computations will be made simultaneously.
// If there are more than 5 calls to veryExpensiveComputations at once,
// veryExpensiveComputation will queue up computations until another is
// finished.
//
// If ctx is cancelled before the computation starts, veryExpensiveComputation
// will return with ctx.Err.
func veryExpensiveComputation(ctx context.Context) (res int, err error) {
ran := l.DoContext(ctx, func() {
// assign something to res and err
})
if !ran {
return 0, ctx.Err()
}
return res, err
}
```
```go
// veryExpensiveComputation does a very expensive computation.
//
// It splits the computation into 123 smaller computations.
// These computations are executed concurrently, 5 at a time.
//
// If ctx is cancelled before the computation concludes, ctx.Err() is returned.
//
// Otherwise, the result or an error is returned.
func veryExpensiveComputation(ctx context.Context) (res int, err error) {
partials := make([]worker.Result[int], 0, 123)
p := worker.NewResultPool(ctx, 5, worker.ToSlice(&partials))
for i := 0; i < 123; i++ {
p.Go(func(ctx context.Context) (result int, err error, cancel bool) {
// return the result of the computation
})
}
if ctxErr := p.Wait(); ctxErr != nil {
return 0, ctxErr
}
for _, partial := range partials {
if partial.Err != nil {
return 0, err
}
// compute res from partial result
}
return res, nil
}
```
## License
Built with ❤ by [Maximilian von Lindern](https://github.com/mavolin).
Available under the [MIT License](./LICENSE).