{"id":36585673,"url":"https://github.com/viney-shih/goroutines","last_synced_at":"2026-01-12T08:02:41.191Z","repository":{"id":37047140,"uuid":"301914295","full_name":"viney-shih/goroutines","owner":"viney-shih","description":"It is an efficient, flexible, and lightweight goroutine pool. It provides an easy way to deal with concurrent tasks with limited resource.","archived":false,"fork":false,"pushed_at":"2022-06-18T04:23:54.000Z","size":289,"stargazers_count":97,"open_issues_count":0,"forks_count":7,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-06-19T02:06:26.914Z","etag":null,"topics":["batch","batch-processing","go","golang","goroutine-pool","goroutines","pool"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/viney-shih.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":"SECURITY.md","support":null}},"created_at":"2020-10-07T03:12:05.000Z","updated_at":"2024-05-17T11:22:45.000Z","dependencies_parsed_at":"2022-06-25T04:55:04.246Z","dependency_job_id":null,"html_url":"https://github.com/viney-shih/goroutines","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"purl":"pkg:github/viney-shih/goroutines","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viney-shih%2Fgoroutines","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viney-shih%2Fgoroutines/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viney-shih%2Fgoroutines/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viney-shih%2Fgoroutines/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/viney-shih","download_url":"https://codeload.github.com/viney-shih/goroutines/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/viney-shih%2Fgoroutines/sbom","scorecard":{"id":922390,"data":{"date":"2025-08-11","repo":{"name":"github.com/viney-shih/goroutines","commit":"dee111bbce926cadb38d26d54c5945ec6ccfeb5d"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":5.1,"checks":[{"name":"Dangerous-Workflow","score":10,"reason":"no dangerous workflow patterns detected","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Pinned-Dependencies","score":0,"reason":"dependency not pinned by hash detected -- score normalized to 0","details":["Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:42: update your workflow using https://app.stepsecurity.io/secureworkflow/viney-shih/goroutines/codeql-analysis.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:46: update your workflow using https://app.stepsecurity.io/secureworkflow/viney-shih/goroutines/codeql-analysis.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:57: update your workflow using https://app.stepsecurity.io/secureworkflow/viney-shih/goroutines/codeql-analysis.yml/master?enable=pin","Warn: GitHub-owned GitHubAction not pinned by hash: .github/workflows/codeql-analysis.yml:71: update your workflow using https://app.stepsecurity.io/secureworkflow/viney-shih/goroutines/codeql-analysis.yml/master?enable=pin","Info:   0 out of   4 GitHub-owned GitHubAction dependencies pinned"],"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Code-Review","score":1,"reason":"Found 3/29 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":4,"reason":"security policy file detected","details":["Info: security policy file detected: SECURITY.md:1","Warn: no linked content found","Info: Found disclosure, vulnerability, and/or timelines in security policy: SECURITY.md:1","Info: Found text in security policy: SECURITY.md:1"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Token-Permissions","score":9,"reason":"detected GitHub workflow tokens with excessive permissions","details":["Info: jobLevel 'actions' permission set to 'read': .github/workflows/codeql-analysis.yml:28","Info: jobLevel 'contents' permission set to 'read': .github/workflows/codeql-analysis.yml:29","Warn: no topLevel permission defined: .github/workflows/codeql-analysis.yml:1","Info: no jobLevel write permissions found"],"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"Signed-Releases","score":-1,"reason":"no releases found","details":null,"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'master'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"SAST","score":7,"reason":"SAST tool detected but not run on all commits","details":["Info: SAST configuration detected: CodeQL","Warn: 0 commits out of 5 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":10,"reason":"0 existing vulnerabilities detected","details":null,"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-25T05:37:38.317Z","repository_id":37047140,"created_at":"2025-08-25T05:37:38.317Z","updated_at":"2025-08-25T05:37:38.317Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28336980,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-12T06:09:07.588Z","status":"ssl_error","status_checked_at":"2026-01-12T06:05:18.301Z","response_time":98,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.5:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch","batch-processing","go","golang","goroutine-pool","goroutines","pool"],"created_at":"2026-01-12T08:02:40.999Z","updated_at":"2026-01-12T08:02:41.181Z","avatar_url":"https://github.com/viney-shih.png","language":"Go","funding_links":[],"categories":["Go"],"sub_categories":[],"readme":"# goroutines\n\n[![GoDev](https://img.shields.io/badge/go.dev-doc-007d9c?style=flat-square\u0026logo=read-the-docs)](https://pkg.go.dev/github.com/viney-shih/goroutines?tab=doc)\n[![Build Status](https://travis-ci.com/viney-shih/goroutines.svg?branch=master)](https://travis-ci.com/github/viney-shih/goroutines)\n[![Go Report Card](https://goreportcard.com/badge/github.com/viney-shih/goroutines)](https://goreportcard.com/report/github.com/viney-shih/goroutines)\n[![codecov](https://codecov.io/gh/viney-shih/goroutines/branch/master/graph/badge.svg)](https://codecov.io/gh/viney-shih/goroutines)\n[![Coverage Status](https://coveralls.io/repos/github/viney-shih/goroutines/badge.svg?branch=master)](https://coveralls.io/github/viney-shih/goroutines?branch=master)\n[![License](http://img.shields.io/badge/License-Apache_2-red.svg?style=flat)](http://www.apache.org/licenses/LICENSE-2.0)\n[![Sourcegraph](https://sourcegraph.com/github.com/viney-shih/goroutines/-/badge.svg)](https://sourcegraph.com/github.com/viney-shih/goroutines?badge)\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fviney-shih%2Fgoroutines.svg?type=shield)](https://app.fossa.com/projects/git%2Bgithub.com%2Fviney-shih%2Fgoroutines?ref=badge_shield)\n\n\u003cp align=\"center\"\u003e\n  \u003cimg src=\"assets/logo.png\" title=\"Goroutines\" /\u003e\n\u003c/p\u003e\n\nPackage **goroutines** is an efficient, flexible, and lightweight goroutine pool written in Go. It provides a easy way to deal with several kinds of concurrent tasks with limited resource. \n\nInspired by [fastsocket](https://github.com/faceair/fastsocket), the implementation is based on **channel**. It adopts pubsub model for dispatching tasks, and holding surplus tasks in queue if submitted more than the capacity of pool.\n\n## Features\n- Spawning and managing arbitrary number of asynchronous goroutines as a worker pool.\n- Dispatch tasks to workers through pubsub model with specified queue size.\n- Adjust the worker numbers based on the usage periodically.\n- Easy to use when dealing with concurrent one-time batch jobs.\n- Monitor current status by metrics\n\n## Table of Contents\n\n* [Installation](#Installation)\n* [Get Started](#Get-Started)\n\t* [Basic usage of Pool (blocking mode)](#Basic-usage-of-Pool-in-blocking-mode)\n\t* [Basic usage of Pool (non-blocking mode)](#Basic-usage-of-Pool-in-nonblocking-mode)\n\t* [Advanced usage (one-time batch jobs)](#Advanced-usage-of-Batch-jobs)\n* [Options](#Options)\n\t* [PoolOption](#PoolOption)\n\t\t* [WithTaskQueueLength(length int)](#PoolOption)\n\t\t* [WithPreAllocWorkers(size int)](#PoolOption)\n\t\t* [WithWorkerAdjustPeriod(period time.Duration)](#PoolOption)\n\t* [BatchOption](#BatchOption)\n\t\t* [WithBatchSize(size int)](#BatchOption)\n* [References](#References)\n* [License](#License)\n\n## Installation\n\n```sh\ngo get github.com/viney-shih/goroutines\n```\n## Get Started\n### Basic usage of Pool in blocking mode\n\nBy calling `Schedule()`, it schedules the task executed by worker (goroutines) in the Pool.\nIt will be blocked until the workers accepting the request.\n\n```go\ntaskN := 7\nrets := make(chan int, taskN)\n\n// allocate a pool with 5 goroutines to deal with those tasks\np := goroutines.NewPool(5)\n// don't forget to release the pool in the end\ndefer p.Release()\n\n// assign tasks to asynchronous goroutine pool\nfor i := 0; i \u003c taskN; i++ {\n\tidx := i\n\tp.Schedule(func() {\n\t\t// sleep and return the index\n\t\ttime.Sleep(20 * time.Millisecond)\n\t\trets \u003c- idx\n\t})\n}\n\n// wait until all tasks done\nfor i := 0; i \u003c taskN; i++ {\n\tfmt.Println(\"index:\", \u003c-rets)\n}\n\n// Unordered output:\n// index: 3\n// index: 1\n// index: 2\n// index: 4\n// index: 5\n// index: 6\n// index: 0\n```\n\n### Basic usage of Pool in nonblocking mode\n\nBy calling `ScheduleWithTimeout()`, it schedules the task executed by worker (goroutines) in the Pool within the specified period.\nIf it exceeds the time and doesn't be accepted, it will return error `ErrScheduleTimeout`.\n\n```go\ntotalN, taskN := 5, 5\npause := make(chan struct{})\nrets := make(chan int, taskN)\n\n// allocate a pool with 5 goroutines to deal with those 5 tasks\np := goroutines.NewPool(totalN)\n// don't forget to release the pool in the end\ndefer p.Release()\n\n// full the workers which are stopped with the `pause`\nfor i := 0; i \u003c taskN; i++ {\n\tidx := i\n\tp.ScheduleWithTimeout(50*time.Millisecond, func() {\n\t\t\u003c-pause\n\t\trets \u003c- idx\n\t})\n}\n\n// no more chance to add any task in Pool, and return `ErrScheduleTimeout`\nif err := p.ScheduleWithTimeout(50*time.Millisecond, func() {\n\t\u003c-pause\n\trets \u003c- taskN\n}); err != nil {\n\tfmt.Println(err.Error())\n}\n\nclose(pause)\nfor i := 0; i \u003c taskN; i++ {\n\tfmt.Println(\"index:\", \u003c-rets)\n}\n\n// Unordered output:\n// schedule timeout\n// index: 0\n// index: 3\n// index: 2\n// index: 4\n// index: 1\n```\n### Advanced usage of Batch jobs\n\nTo deal with batch jobs and consider the performance, we need to run tasks concurrently. However, the use case usually happen once and need not maintain a Pool for reusing it. I wrap this patten and call it `Batch`. Here comes an example.\n\n```go\ntaskN := 11\n\n// allocate a one-time batch job with 3 goroutines to deal with those tasks.\n// no need to spawn extra goroutine by specifing the batch size consisting with the number of tasks.\nb := goroutines.NewBatch(3, goroutines.WithBatchSize(taskN))\n// don't forget to close batch job in the end\ndefer b.Close()\n\n// pull all tasks to this batch queue\nfor i := 0; i \u003c taskN; i++ {\n\tidx := i\n\tb.Queue(func() (interface{}, error) {\n\t\t// sleep and return the index\n\t\ttime.Sleep(10 * time.Millisecond)\n\t\treturn idx, nil\n\t})\n}\n\n// tell the batch that's all need to do\n// DO NOT FORGET THIS OR GOROUTINES WILL DEADLOCK\nb.QueueComplete()\n\nfor ret := range b.Results() {\n\tif ret.Error() != nil {\n\t\tpanic(\"not expected\")\n\t}\n\n\tfmt.Println(\"index:\", ret.Value().(int))\n}\n\n// Unordered output:\n// index: 3\n// index: 1\n// index: 2\n// index: 4\n// index: 5\n// index: 6\n// index: 10\n// index: 7\n// index: 9\n// index: 8\n// index: 0\n```\n\nSee the [examples](https://pkg.go.dev/github.com/viney-shih/goroutines#pkg-examples), [documentation](https://pkg.go.dev/github.com/viney-shih/goroutines) and [article](https://medium.com/17media-tech/%E9%82%A3%E4%BA%9B%E5%B9%B4%E6%88%91%E5%80%91%E8%BF%BD%E7%9A%84-goroutine-pool-e8d211757ee) for more details.\n\n## Options\n### PoolOption\nThe `PoolOption` interface is passed to `NewPool` when creating Pool.\n\n**• WithTaskQueueLength(** ***length*** `int` **)**\n\nIt sets up the length of task queue for buffering tasks before sending to goroutines. The default queue length is `0`.\n\n**• WithPreAllocWorkers(** ***size*** `int` **)**\n\nIt sets up the number of workers to spawn when initializing Pool. Without specifying this, It initialize all numbers of goroutines consisting with Pool size at the beginning.\n\n**• WithWorkerAdjustPeriod(** ***period*** `time.Duration` **)**\n\nIt sets up the duration to adjust the worker size, and needs to be used with `WithPreAllocWorkers` at the same time. By specifying both, it enables the mechanism to adjust the number of goroutines according to the usage dynamically.\n\n### BatchOption\nThe `BatchOption` interface is passed to `NewBatch` when creating Batch.\n\n**• WithBatchSize(** ***size*** `int` **)**\n\nIt specifies the batch size used to forward tasks.\nBy default, it needs to spawn an extra goroutine to prevent deadlocks.\nIt's helpful by specifing the batch size consisting with the number of tasks without an extra goroutine (see the [example](https://pkg.go.dev/github.com/viney-shih/goroutines#pkg-examples)). The default batch size is `10`.\n\n## References\n- https://github.com/faceair/fastsocket\n- https://github.com/valyala/fasthttp\n- https://github.com/Jeffail/tunny\n- https://github.com/go-playground/pool\n- https://github.com/panjf2000/ants\n\n## License\n[Apache-2.0](https://opensource.org/licenses/Apache-2.0)\n\n[![FOSSA Status](https://app.fossa.com/api/projects/git%2Bgithub.com%2Fviney-shih%2Fgoroutines.svg?type=large)](https://app.fossa.com/projects/git%2Bgithub.com%2Fviney-shih%2Fgoroutines?ref=badge_large)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fviney-shih%2Fgoroutines","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fviney-shih%2Fgoroutines","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fviney-shih%2Fgoroutines/lists"}