https://github.com/fireflycons/workerpool
Package workerpool provides a wrapper around github.com/cilium/workerpool to facilitate processing items from a concurrentqueue.Queue
https://github.com/fireflycons/workerpool
Last synced: 6 months ago
JSON representation
Package workerpool provides a wrapper around github.com/cilium/workerpool to facilitate processing items from a concurrentqueue.Queue
- Host: GitHub
- URL: https://github.com/fireflycons/workerpool
- Owner: fireflycons
- License: mit
- Created: 2026-01-09T06:29:02.000Z (6 months ago)
- Default Branch: master
- Last Pushed: 2026-01-12T12:43:23.000Z (6 months ago)
- Last Synced: 2026-01-12T18:39:55.238Z (6 months ago)
- Language: Go
- Size: 15.6 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# workerpool
[](https://pkg.go.dev/github.com/fireflycons/workerpool) [](https://github.com/fireflycons/workerpool/actions/workflows/build.yml) [](https://goreportcard.com/report/github.com/fireflycons/workerpool)
Package workerpool provides a wrapper around [github.com/cilium/workerpool](https://pkg.go.dev/github.com/cilium/workerpool) to facilitate processing items from a [concurrentqueue.Queue](https://pkg.go.dev/github.com/fireflycons/concurrentqueue) using a pool of workers.
## Features
* Generic
* Uses [concurrentqueue](https://pkg.go.dev/github.com/fireflycons/concurrentqueue) to feed workers and as optional dead letter queue
* Four run options
* Mutex-free
* Dead letter queue support
## Example
```go
package main
import (
"context"
"fmt"
"sync"
queue "github.com/fireflycons/concurrentqueue"
"github.com/fireflycons/workerpool"
)
type result struct {
input int
square int
}
func (r result) String() string {
return fmt.Sprintf("input: %d, square: %d", r.input, r.square)
}
func main() {
q := queue.New[int]()
for i := range 100 {
err := q.Enqueue(i)
if err != nil {
fmt.Printf("Enqueue error: %v\n", err)
}
}
q.Close()
resultsChan := make(chan result, 10)
wg := sync.WaitGroup{}
ctx := context.Background() // You would use a more meaningful context
wg.Go(func() {
_, err := workerpool.RunWorkersWithResults(
func(ctx context.Context, in <-chan int, out chan<- result, _ *queue.Queue[int]) error {
for v := range in {
select {
case <-ctx.Done():
return ctx.Err()
default:
// If a dead letter queue is provided (last argument), failed items would be sent there
out <- result{input: v, square: v * v}
}
}
return nil
},
q,
resultsChan,
workerpool.WithContext[int](ctx), // Default is context.Background() if this option is not provided.
workerpool.WithNumWorkers[int](4), // Default is runtime.NumCPU() if this option is not provided.
)
close(resultsChan)
if err != nil {
panic(fmt.Sprintf("Error starting workers: %v\n", err))
}
})
wg.Go(func() {
for r := range resultsChan {
fmt.Println(r)
}
})
wg.Wait()
}
```