{"id":13413095,"url":"https://github.com/go-playground/pool","last_synced_at":"2025-05-15T02:05:20.113Z","repository":{"id":43057217,"uuid":"45126702","full_name":"go-playground/pool","owner":"go-playground","description":":speedboat: a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation","archived":false,"fork":false,"pushed_at":"2021-06-28T13:01:34.000Z","size":84,"stargazers_count":730,"open_issues_count":4,"forks_count":65,"subscribers_count":18,"default_branch":"v3","last_synced_at":"2025-04-11T14:57:00.545Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/go-playground.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-28T16:36:08.000Z","updated_at":"2025-04-11T00:07:32.000Z","dependencies_parsed_at":"2022-09-26T17:41:31.698Z","dependency_job_id":null,"html_url":"https://github.com/go-playground/pool","commit_stats":null,"previous_names":["joeybloggs/pool"],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-playground%2Fpool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-playground%2Fpool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-playground%2Fpool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/go-playground%2Fpool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/go-playground","download_url":"https://codeload.github.com/go-playground/pool/tar.gz/refs/heads/v3","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248790721,"owners_count":21162077,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-07-30T20:01:33.396Z","updated_at":"2025-04-13T22:33:35.216Z","avatar_url":"https://github.com/go-playground.png","language":"Go","readme":"Package pool\n============\n\n![Project status](https://img.shields.io/badge/version-3.1.1-green.svg)\n[![Build Status](https://semaphoreci.com/api/v1/joeybloggs/pool/branches/v3/badge.svg)](https://semaphoreci.com/joeybloggs/pool)\n[![Coverage Status](https://coveralls.io/repos/go-playground/pool/badge.svg?branch=v3\u0026service=github)](https://coveralls.io/github/go-playground/pool?branch=v3)\n[![Go Report Card](https://goreportcard.com/badge/gopkg.in/go-playground/pool.v3)](https://goreportcard.com/report/gopkg.in/go-playground/pool.v3)\n[![GoDoc](https://godoc.org/gopkg.in/go-playground/pool.v3?status.svg)](https://godoc.org/gopkg.in/go-playground/pool.v3)\n![License](https://img.shields.io/dub/l/vibe-d.svg)\n\nPackage pool implements a limited consumer goroutine or unlimited goroutine pool for easier goroutine handling and cancellation. \n\nFeatures:\n\n-    Dead simple to use and makes no assumptions about how you will use it.\n-    Automatic recovery from consumer goroutines which returns an error to the results\n\nPool v2 advantages over Pool v1:\n\n- Up to 300% faster due to lower contention ( BenchmarkSmallRun used to take 3 seconds, now 1 second )\n- Cancels are much faster\n- Easier to use, no longer need to know the # of Work Units to be processed.\n- Pool can now be used as a long running/globally defined pool if desired ( v1 Pool was only good for one run )\n- Supports single units of work as well as batching\n- Pool can easily be reset after a Close() or Cancel() for reuse.\n- Multiple Batches can be run and even cancelled on the same Pool.\n- Supports individual Work Unit cancellation.\n\nPool v3 advantages over Pool v2:\n\n- Objects are not interfaces allowing for less breaking changes going forward.\n- Now there are 2 Pool types, both completely interchangeable, a limited worker pool and unlimited pool.\n- Simpler usage of Work Units, instead of `\u003c-work.Done` now can do `work.Wait()`\n\nInstallation\n------------\n\nUse go get.\n\n\tgo get gopkg.in/go-playground/pool.v3\n\nThen import the pool package into your own code.\n\n\timport \"gopkg.in/go-playground/pool.v3\"\n\n\nImportant Information READ THIS!\n------\n\n- It is recommended that you cancel a pool or batch from the calling function and not inside of the Unit of Work, it will work fine, however because of the goroutine scheduler and context switching it may not cancel as soon as if called from outside.\n- When Batching DO NOT FORGET TO CALL batch.QueueComplete(), if you do the Batch WILL deadlock\n- It is your responsibility to call WorkUnit.IsCancelled() to check if it's cancelled after a blocking operation like waiting for a connection from a pool. (optional)\n\nUsage and documentation\n------\n\nPlease see http://godoc.org/gopkg.in/go-playground/pool.v3 for detailed usage docs.\n\n##### Examples:\n\nboth Limited Pool and Unlimited Pool have the same signatures and are completely interchangeable.\n\nPer Unit Work\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"gopkg.in/go-playground/pool.v3\"\n)\n\nfunc main() {\n\n\tp := pool.NewLimited(10)\n\tdefer p.Close()\n\n\tuser := p.Queue(getUser(13))\n\tother := p.Queue(getOtherInfo(13))\n\n\tuser.Wait()\n\tif err := user.Error(); err != nil {\n\t\t// handle error\n\t}\n\n\t// do stuff with user\n\tusername := user.Value().(string)\n\tfmt.Println(username)\n\n\tother.Wait()\n\tif err := other.Error(); err != nil {\n\t\t// handle error\n\t}\n\n\t// do stuff with other\n\totherInfo := other.Value().(string)\n\tfmt.Println(otherInfo)\n}\n\nfunc getUser(id int) pool.WorkFunc {\n\n\treturn func(wu pool.WorkUnit) (interface{}, error) {\n\n\t\t// simulate waiting for something, like TCP connection to be established\n\t\t// or connection from pool grabbed\n\t\ttime.Sleep(time.Second * 1)\n\n\t\tif wu.IsCancelled() {\n\t\t\t// return values not used\n\t\t\treturn nil, nil\n\t\t}\n\n\t\t// ready for processing...\n\n\t\treturn \"Joeybloggs\", nil\n\t}\n}\n\nfunc getOtherInfo(id int) pool.WorkFunc {\n\n\treturn func(wu pool.WorkUnit) (interface{}, error) {\n\n\t\t// simulate waiting for something, like TCP connection to be established\n\t\t// or connection from pool grabbed\n\t\ttime.Sleep(time.Second * 1)\n\n\t\tif wu.IsCancelled() {\n\t\t\t// return values not used\n\t\t\treturn nil, nil\n\t\t}\n\n\t\t// ready for processing...\n\n\t\treturn \"Other Info\", nil\n\t}\n}\n```\n\nBatch Work\n```go\npackage main\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\t\"gopkg.in/go-playground/pool.v3\"\n)\n\nfunc main() {\n\n\tp := pool.NewLimited(10)\n\tdefer p.Close()\n\n\tbatch := p.Batch()\n\n\t// for max speed Queue in another goroutine\n\t// but it is not required, just can't start reading results\n\t// until all items are Queued.\n\n\tgo func() {\n\t\tfor i := 0; i \u003c 10; i++ {\n\t\t\tbatch.Queue(sendEmail(\"email content\"))\n\t\t}\n\n\t\t// DO NOT FORGET THIS OR GOROUTINES WILL DEADLOCK\n\t\t// if calling Cancel() it calles QueueComplete() internally\n\t\tbatch.QueueComplete()\n\t}()\n\n\tfor email := range batch.Results() {\n\n\t\tif err := email.Error(); err != nil {\n\t\t\t// handle error\n\t\t\t// maybe call batch.Cancel()\n\t\t}\n\n\t\t// use return value\n\t\tfmt.Println(email.Value().(bool))\n\t}\n}\n\nfunc sendEmail(email string) pool.WorkFunc {\n\treturn func(wu pool.WorkUnit) (interface{}, error) {\n\n\t\t// simulate waiting for something, like TCP connection to be established\n\t\t// or connection from pool grabbed\n\t\ttime.Sleep(time.Second * 1)\n\n\t\tif wu.IsCancelled() {\n\t\t\t// return values not used\n\t\t\treturn nil, nil\n\t\t}\n\n\t\t// ready for processing...\n\n\t\treturn true, nil // everything ok, send nil, error if not\n\t}\n}\n```\n\nBenchmarks\n------\n###### Run on MacBook Pro (Retina, 15-inch, Late 2013) 2.6 GHz Intel Core i7 16 GB 1600 MHz DDR3 using Go 1.6.2\n\nrun with 1, 2, 4,8 and 16 cpu to show it scales well...16 is double the # of logical cores on this machine.\n\nNOTE: Cancellation times CAN vary depending how busy your system is and how the goroutine scheduler is but \nworse case I've seen is 1s to cancel instead of 0ns\n\n```go\ngo test -cpu=1,2,4,8,16 -bench=. -benchmem=true\nPASS\nBenchmarkLimitedSmallRun              \t       1\t1002492008 ns/op\t    3552 B/op\t      55 allocs/op\nBenchmarkLimitedSmallRun-2            \t       1\t1002347196 ns/op\t    3568 B/op\t      55 allocs/op\nBenchmarkLimitedSmallRun-4            \t       1\t1010533571 ns/op\t    4720 B/op\t      73 allocs/op\nBenchmarkLimitedSmallRun-8            \t       1\t1008883324 ns/op\t    4080 B/op\t      63 allocs/op\nBenchmarkLimitedSmallRun-16           \t       1\t1002317677 ns/op\t    3632 B/op\t      56 allocs/op\nBenchmarkLimitedSmallCancel           \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedSmallCancel-2         \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedSmallCancel-4         \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedSmallCancel-8         \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedSmallCancel-16        \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedLargeCancel           \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedLargeCancel-2         \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedLargeCancel-4         \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedLargeCancel-8         \t 1000000\t      1006 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedLargeCancel-16        \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkLimitedOverconsumeLargeRun   \t       1\t4027153081 ns/op\t   36176 B/op\t     572 allocs/op\nBenchmarkLimitedOverconsumeLargeRun-2 \t       1\t4003489261 ns/op\t   32336 B/op\t     512 allocs/op\nBenchmarkLimitedOverconsumeLargeRun-4 \t       1\t4005579847 ns/op\t   34128 B/op\t     540 allocs/op\nBenchmarkLimitedOverconsumeLargeRun-8 \t       1\t4004639857 ns/op\t   34992 B/op\t     553 allocs/op\nBenchmarkLimitedOverconsumeLargeRun-16\t       1\t4022695297 ns/op\t   36864 B/op\t     532 allocs/op\nBenchmarkLimitedBatchSmallRun         \t       1\t1000785511 ns/op\t    6336 B/op\t      94 allocs/op\nBenchmarkLimitedBatchSmallRun-2       \t       1\t1001459945 ns/op\t    4480 B/op\t      65 allocs/op\nBenchmarkLimitedBatchSmallRun-4       \t       1\t1002475371 ns/op\t    6672 B/op\t      99 allocs/op\nBenchmarkLimitedBatchSmallRun-8       \t       1\t1002498902 ns/op\t    4624 B/op\t      67 allocs/op\nBenchmarkLimitedBatchSmallRun-16      \t       1\t1002202273 ns/op\t    5344 B/op\t      78 allocs/op\nBenchmarkUnlimitedSmallRun            \t       1\t1002361538 ns/op\t    3696 B/op\t      59 allocs/op\nBenchmarkUnlimitedSmallRun-2          \t       1\t1002230293 ns/op\t    3776 B/op\t      60 allocs/op\nBenchmarkUnlimitedSmallRun-4          \t       1\t1002148953 ns/op\t    3776 B/op\t      60 allocs/op\nBenchmarkUnlimitedSmallRun-8          \t       1\t1002120679 ns/op\t    3584 B/op\t      57 allocs/op\nBenchmarkUnlimitedSmallRun-16         \t       1\t1001698519 ns/op\t    3968 B/op\t      63 allocs/op\nBenchmarkUnlimitedSmallCancel         \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedSmallCancel-2       \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedSmallCancel-4       \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedSmallCancel-8       \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedSmallCancel-16      \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedLargeCancel         \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedLargeCancel-2       \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedLargeCancel-4       \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedLargeCancel-8       \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedLargeCancel-16      \t2000000000\t         0.00 ns/op\t       0 B/op\t       0 allocs/op\nBenchmarkUnlimitedLargeRun            \t       1\t1001631711 ns/op\t   40352 B/op\t     603 allocs/op\nBenchmarkUnlimitedLargeRun-2          \t       1\t1002603908 ns/op\t   38304 B/op\t     586 allocs/op\nBenchmarkUnlimitedLargeRun-4          \t       1\t1001452975 ns/op\t   38192 B/op\t     584 allocs/op\nBenchmarkUnlimitedLargeRun-8          \t       1\t1005382882 ns/op\t   35200 B/op\t     537 allocs/op\nBenchmarkUnlimitedLargeRun-16         \t       1\t1001818482 ns/op\t   37056 B/op\t     566 allocs/op\nBenchmarkUnlimitedBatchSmallRun       \t       1\t1002391247 ns/op\t    4240 B/op\t      63 allocs/op\nBenchmarkUnlimitedBatchSmallRun-2     \t       1\t1010313222 ns/op\t    4688 B/op\t      70 allocs/op\nBenchmarkUnlimitedBatchSmallRun-4     \t       1\t1008364651 ns/op\t    4304 B/op\t      64 allocs/op\nBenchmarkUnlimitedBatchSmallRun-8     \t       1\t1001858192 ns/op\t    4448 B/op\t      66 allocs/op\nBenchmarkUnlimitedBatchSmallRun-16    \t       1\t1001228000 ns/op\t    4320 B/op\t      64 allocs/op\n```\nTo put some of these benchmarks in perspective:\n\n- BenchmarkLimitedSmallRun did 10 seconds worth of processing in 1.002492008s\n- BenchmarkLimitedSmallCancel ran 20 jobs, cancelled on job 6 and and ran in 0s\n- BenchmarkLimitedLargeCancel ran 1000 jobs, cancelled on job 6 and and ran in 0s\n- BenchmarkLimitedOverconsumeLargeRun ran 100 jobs using 25 workers in 4.027153081s\n\n\nLicense\n------\nDistributed under MIT License, please see license file in code for more details.\n","funding_links":[],"categories":["Goroutines","Relational Databases","\u003cspan id=\"协程-Coroutines\"\u003e协程 Coroutines\u003c/span\u003e","Goroutines `goroutines的管理和使用`"],"sub_categories":["Search and Analytic Databases","Advanced Console UIs","SQL 查询语句构建库","检索及分析资料库","\u003cspan id=\"高级控制台用户界面-advanced-console-uis\"\u003e高级控制台用户界面 Advanced Console UIs\u003c/span\u003e"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgo-playground%2Fpool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgo-playground%2Fpool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgo-playground%2Fpool/lists"}