{"id":28541722,"url":"https://github.com/konstructio/workpool","last_synced_at":"2025-09-10T13:37:21.107Z","repository":{"id":263739521,"uuid":"891275094","full_name":"konstructio/workpool","owner":"konstructio","description":"A Go library to manage a pool of workers, useful for async tasks","archived":false,"fork":false,"pushed_at":"2024-11-20T21:02:36.000Z","size":10,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-07-07T15:45:33.596Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/konstructio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-11-20T03:22:40.000Z","updated_at":"2025-01-20T20:03:50.000Z","dependencies_parsed_at":"2024-11-20T07:34:56.099Z","dependency_job_id":"318774fb-5238-4199-8ef6-9b9242322e1c","html_url":"https://github.com/konstructio/workpool","commit_stats":null,"previous_names":["konstructio/workpool"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/konstructio/workpool","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konstructio%2Fworkpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konstructio%2Fworkpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konstructio%2Fworkpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konstructio%2Fworkpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/konstructio","download_url":"https://codeload.github.com/konstructio/workpool/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/konstructio%2Fworkpool/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274470870,"owners_count":25291611,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-10T02:00:12.551Z","response_time":83,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-06-09T20:09:43.925Z","updated_at":"2025-09-10T13:37:21.086Z","avatar_url":"https://github.com/konstructio.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Workpool\n\n[![GoDoc](https://godoc.org/github.com/konstructio/workpool?status.svg)](https://godoc.org/github.com/konstructio/workpool)\n\n`workpool` is a Go library that provides a simple and efficient worker pool implementation. It allows you to manage and execute tasks asynchronously with a controlled level of concurrency. The library handles task submission, worker management, and graceful shutdown with context cancellation.\n\n## Features\n\n- **Fixed Number of Workers**: Create a pool with a specified number of worker goroutines.\n- **Asynchronous Task Execution**: Submit tasks to be executed asynchronously.\n- **Synchronous Task Submission**: Submit tasks and wait for their completion.\n- **Graceful Shutdown**: Stop the pool gracefully, allowing workers to finish their current tasks.\n- **Context Propagation**: Each task receives a context to handle cancellations and timeouts.\n- **Configurable Queue Capacity**: Specify the capacity of the job queue to control memory usage and backpressure.\n\n## Installation\n\nTo install the `workpool` library, use `go get`:\n\n```bash\ngo get github.com/konstructio/workpool@latest\n```\n\n## Usage\n\nImport the package in your Go code:\n\n```go\nimport \"github.com/konstructio/workpool\"\n```\n\n### Creating a Pool\n\nCreate a new worker pool by specifying the context, number of workers, and job queue capacity:\n\n```go\nctx := context.Background()\nnumWorkers := 5\nqueueCapacity := 100\n\npool, err := workpool.Initialize(ctx, numWorkers, queueCapacity)\nif err != nil {\n    // Handle error\n}\n```\n\n### Submitting Tasks Asynchronously\n\nSubmit tasks to the pool to be executed asynchronously. This function returns an error if the pool is stopped, but otherwise doesn't return any other error, as the task is executed asynchronously. If you need the error message from the task, you can use `SubmitAndWait()` instead.\n\n```go\nerr := pool.Submit(func(ctx context.Context) error {\n    // Task logic here\n    fmt.Println(\"Executing async task\")\n    return nil\n})\nif err != nil {\n    // Handle error (e.g., pool is stopped)\n}\n```\n\n### Submitting Tasks and Waiting for Completion\n\nSubmit a task and wait for its completion using `SubmitAndWait`. This function returns an error if the pool is stopped. Contrary to `Submit()`, this function will also wait for the Job to be executed then return any potential error returned by the Job execution.\n\n```go\nerr := pool.SubmitAndWait(func(ctx context.Context) error {\n    // Task logic here\n    fmt.Println(\"Executing synchronous task\")\n    return nil\n})\nif err != nil {\n    // Handle error (e.g., pool is stopped)\n}\n\nerr := pool.SubmitAndWait(func(ctx context.Context) error {\n    // Task logic here\n    return fmt.Errorf(\"task failed\")\n})\nif err != nil {\n    fmt.Println(\"Task error:\", err)\n}\n```\n\n### Graceful Shutdown\n\nUse `Stop()` to stop the pool gracefully, allowing workers to finish their current tasks. The pool will not accept new tasks after calling `Stop`, and it will not forcefully stop the workers from completing their current tasks.\n\n```go\npool.Stop()\n```\n\nAlternatively, you can also cancel the context associated with the `Initialize()` function call to stop the pool:\n\n```go\npoolCtx, cancel := context.WithCancel(ctx)\n\npool, err := workpool.Initialize(poolCtx, numWorkers, queueCapacity)\nif err != nil { }\n\ngo func() {\n  pool.RunAndWait()\n}()\n\n// stop the pool\ncancel()\n```\n\n\n### Running the Pool and Waiting for Shutdown\n\nUse `RunAndWait` to run the pool and block until `Stop` is called or an OS interrupt signal is received:\n\n```go\ngo func() {\n  // Use a goroutine to prevent blocking the program\n  pool.RunAndWait()\n}()\n\n// Simulate work and then stop the pool after 5 seconds\ntime.Sleep(5 * time.Second)\npool.Stop()\n```\n\n### Example: Basic Usage\n\n```go\npackage main\n\nimport (\n    \"context\"\n    \"fmt\"\n    \"time\"\n\n    \"github.com/konstructio/workpool\"\n)\n\nfunc main() {\n    ctx := context.Background()\n    pool, err := workpool.Initialize(ctx, 5, 100)\n    if err != nil {\n        fmt.Println(\"Error initializing pool:\", err)\n        return\n    }\n\n    defer pool.Stop()\n\n    // Submit tasks asynchronously\n    for i := 0; i \u003c 10; i++ {\n        taskID := i\n        err := pool.Submit(func(ctx context.Context) error {\n            fmt.Printf(\"Async Task %d started\\n\", taskID)\n            time.Sleep(1 * time.Second)\n            fmt.Printf(\"Async Task %d completed\\n\", taskID)\n            return nil\n        })\n        if err != nil {\n            fmt.Printf(\"Error submitting task %d: %v\\n\", taskID, err)\n        }\n    }\n\n    // Submit a task and wait for its completion\n    err = pool.SubmitAndWait(func(ctx context.Context) error {\n        fmt.Println(\"Synchronous Task started\")\n        time.Sleep(2 * time.Second)\n        fmt.Println(\"Synchronous Task completed\")\n        return nil\n    })\n    if err != nil {\n        fmt.Println(\"Error executing synchronous task:\", err)\n    }\n\n    // Run the pool and wait for shutdown\n    go func() {\n        time.Sleep(5 * time.Second)\n        pool.Stop()\n    }()\n    pool.RunAndWait()\n}\n```\n\n### Example: Handling Context Cancellation in Tasks\n\nTasks receive a context that can be used to handle cancellations or timeouts:\n\n```go\nerr := pool.Submit(func(ctx context.Context) error {\n    select {\n    case \u003c-ctx.Done():\n        // Handle cancellation\n        fmt.Println(\"Task was cancelled\")\n        return ctx.Err()\n    case \u003c-time.After(3 * time.Second):\n        // Perform task\n        fmt.Println(\"Task completed\")\n    }\n    return nil\n})\nif err != nil {\n    // Handle error\n}\n```\n\n## API Reference\n\n### Functions\n\n- **Initialize(ctx context.Context, numWorkers int, queueCapacity int) (*Pool, error)**\n  - Creates a new worker pool.\n  - `ctx`: The context for the pool; can be used to set deadlines or cancellation.\n  - `numWorkers`: Number of worker goroutines.\n  - `queueCapacity`: Capacity of the job queue.\n  - Returns a pointer to the `Pool` and an error if the parameters are invalid.\n\n### Pool Methods\n\n- **(p *Pool) Submit(job Job) error**\n  - Submits a job to be executed asynchronously.\n  - Returns an error if the pool is stopped.\n\n- **(p *Pool) SubmitAndWait(job Job) error**\n  - Submits a job and waits for its completion.\n  - Returns any error returned by the job or if the pool is stopped.\n\n- **(p *Pool) Stop()**\n  - Stops the pool gracefully, allowing workers to finish current tasks.\n\n- **(p *Pool) RunAndWait()**\n  - Runs the pool and blocks until `Stop` is called or an OS interrupt signal is received.\n\n### Types\n\n- **Job**\n  - A function type `func(ctx context.Context) error`.\n  - Represents a unit of work to be executed by the pool.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkonstructio%2Fworkpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkonstructio%2Fworkpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkonstructio%2Fworkpool/lists"}