Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/gammazero/workerpool
Concurrency limiting goroutine pool
https://github.com/gammazero/workerpool
concurrency worker-pool
Last synced: about 1 month ago
JSON representation
Concurrency limiting goroutine pool
- Host: GitHub
- URL: https://github.com/gammazero/workerpool
- Owner: gammazero
- License: mit
- Created: 2016-05-17T14:32:06.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2023-06-26T02:32:34.000Z (11 months ago)
- Last Synced: 2024-04-01T15:23:55.766Z (2 months ago)
- Topics: concurrency, worker-pool
- Language: Go
- Size: 85.9 KB
- Stars: 1,238
- Watchers: 20
- Forks: 132
- Open Issues: 16
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Lists
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go-extra - workerpool - 05-17T14:32:06Z|2022-08-17T01:04:44Z| (Goroutines / Advanced Console UIs)
- awesome-stars - gammazero/workerpool - Concurrency limiting goroutine pool (Go)
- awesome-stars - gammazero/workerpool - Concurrency limiting goroutine pool (Go)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- go-awesome - workerpool - 无阻塞队列任务池 (开源类库 / 协程/线程)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. Stars:`1.2K`. (Goroutines / Search and Analytic Databases)
- awesome-tools - workerpool - Concurrency limiting goroutine pool. (Programming / Go)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go-projects - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Standard CLI)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go - workerpool - | - | - | (Goroutines / Advanced Console UIs)
- awesome-go-with-framework - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go-cn - workerpool
- awesome-stars - workerpool - Concurrency limiting goroutine pool (Go)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go-stars - workerpool(stars: 1240) - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go - workerpool - 协程池,限制任务执行的并发性,而不是排队的任务数。 (<span id="协程-Coroutines">协程 Coroutines</span> / <span id="高级控制台用户界面-advanced-console-uis">高级控制台用户界面 Advanced Console UIs</span>)
- awesome-go. - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go-with-stars - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- repo-1316-awesome-go-cn - workerpool
- repo-1211-awesome-go-cn - workerpool
- awesome-Char - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-reader - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- Go-awesome - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- go-awesome-cn-star - workerpool
- awesome-go-cn - workerpool
- awesome-go-cn - workerpool
- awesome-go-zh - workerpool
- awesome-go-handwritten - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go2 - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. - :arrow_down:0 - :star:1 (Goroutines / Advanced Console UIs)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Advanced Console UIs)
- awesome-go-cn - workerpool
- awesome-go-cn - workerpool
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go - workerpool - Goroutine pool that limits the concurrency of task execution, not the number of tasks queued. (Goroutines / Search and Analytic Databases)
- awesome-go - workerpool - Concurrency limiting goroutine pool - ★ 51 (Goroutines)
README
# workerpool
[![GoDoc](https://pkg.go.dev/badge/github.com/gammazero/workerpool)](https://pkg.go.dev/github.com/gammazero/workerpool)
[![Build Status](https://github.com/gammazero/workerpool/actions/workflows/go.yml/badge.svg)](https://github.com/gammazero/workerpool/actions/workflows/go.yml)
[![Go Report Card](https://goreportcard.com/badge/github.com/gammazero/workerpool)](https://goreportcard.com/report/github.com/gammazero/workerpool)
[![codecov](https://codecov.io/gh/gammazero/workerpool/branch/master/graph/badge.svg)](https://codecov.io/gh/gammazero/workerpool)
[![License](https://img.shields.io/badge/License-MIT-blue.svg)](https://github.com/gammazero/workerpool/blob/master/LICENSE)Concurrency limiting goroutine pool. Limits the concurrency of task execution, not the number of tasks queued. Never blocks submitting tasks, no matter how many tasks are queued.
This implementation builds on ideas from the following:
- http://marcio.io/2015/07/handling-1-million-requests-per-minute-with-golang
- http://nesv.github.io/golang/2014/02/25/worker-queues-in-go.html## Installation
To install this package, you need to setup your Go workspace. The simplest way to install the library is to run:
```
$ go get github.com/gammazero/workerpool
```## Example
```go
package mainimport (
"fmt"
"github.com/gammazero/workerpool"
)func main() {
wp := workerpool.New(2)
requests := []string{"alpha", "beta", "gamma", "delta", "epsilon"}for _, r := range requests {
r := r
wp.Submit(func() {
fmt.Println("Handling request:", r)
})
}wp.StopWait()
}
```[Example wrapper function](https://go.dev/play/p/BWnRhJYarZ1) to show start and finish time of submitted function.
## Usage Note
There is no upper limit on the number of tasks queued, other than the limits of system resources. If the number of inbound tasks is too many to even queue for pending processing, then the solution is outside the scope of workerpool. It should be solved by distributing workload over multiple systems, and/or storing input for pending processing in intermediate storage such as a file system, distributed message queue, etc.