{"id":13411735,"url":"https://github.com/tejzpr/ordered-concurrently","last_synced_at":"2025-06-16T14:12:06.775Z","repository":{"id":45986079,"uuid":"343174144","full_name":"tejzpr/ordered-concurrently","owner":"tejzpr","description":"Ordered-concurrently a library for concurrent processing with ordered output in Go. Process work concurrently and returns output in a channel in the order of input. It is useful in concurrently processing items in a queue, and get output in the order provided by the queue.","archived":false,"fork":false,"pushed_at":"2023-04-24T23:24:20.000Z","size":54,"stargazers_count":37,"open_issues_count":3,"forks_count":8,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-07-31T20:48:42.455Z","etag":null,"topics":["concurrent","concurrent-data-structure","data-pipeline","data-science","golang","golang-library","ordered","parallel","parallel-computing"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tejzpr.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2021-02-28T17:56:05.000Z","updated_at":"2024-04-03T21:55:30.000Z","dependencies_parsed_at":"2023-12-19T04:22:40.123Z","dependency_job_id":"30103e1b-0616-47c6-b45b-613257ce8c23","html_url":"https://github.com/tejzpr/ordered-concurrently","commit_stats":{"total_commits":73,"total_committers":3,"mean_commits":"24.333333333333332","dds":0.0273972602739726,"last_synced_commit":"51e061f8ddfb3f951f509fa8494be055c6db7d1d"},"previous_names":[],"tags_count":17,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tejzpr%2Fordered-concurrently","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tejzpr%2Fordered-concurrently/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tejzpr%2Fordered-concurrently/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tejzpr%2Fordered-concurrently/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tejzpr","download_url":"https://codeload.github.com/tejzpr/ordered-concurrently/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243618658,"owners_count":20320272,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["concurrent","concurrent-data-structure","data-pipeline","data-science","golang","golang-library","ordered","parallel","parallel-computing"],"created_at":"2024-07-30T20:01:16.329Z","updated_at":"2025-03-14T17:31:04.565Z","avatar_url":"https://github.com/tejzpr.png","language":"Go","readme":"\u003ca href=\"https://github.com/tejzpr/ordered-concurrently/actions/workflows/tests.yml\"\u003e\u003cimg src=\"https://github.com/tejzpr/ordered-concurrently/actions/workflows/tests.yml/badge.svg\" alt=\"Tests\"/\u003e\u003c/a\u003e\n[![codecov](https://codecov.io/gh/tejzpr/ordered-concurrently/branch/master/graph/badge.svg?token=6WIXWRO3EW)](https://codecov.io/gh/tejzpr/ordered-concurrently)\n[![Go Reference](https://pkg.go.dev/badge/github.com/tejzpr/ordered-concurrently.svg)](https://pkg.go.dev/github.com/tejzpr/ordered-concurrently)\n[![Gitpod ready-to-code](https://img.shields.io/badge/Gitpod-ready--to--code-blue?logo=gitpod)](https://gitpod.io/#https://github.com/tejzpr/ordered-concurrently)\n[![Go Report Card](https://goreportcard.com/badge/github.com/tejzpr/ordered-concurrently)](https://goreportcard.com/report/github.com/tejzpr/ordered-concurrently)\n\n# Ordered Concurrently\nA library for parallel processing with ordered output in Go. This module processes work concurrently / in parallel and returns output in a channel in the order of input. It is useful in concurrently / parallelly processing items in a queue, and get output in the order provided by the queue.\n\n# Usage \n## Get Module\n```go\ngo get github.com/tejzpr/ordered-concurrently/v3\n```\n## Import Module in your source code\n```go\nimport concurrently \"github.com/tejzpr/ordered-concurrently/v3\" \n```\n## Create a work function by implementing WorkFunction interface\n```go\n// Create a type based on your input to the work function\ntype loadWorker int\n\n// The work that needs to be performed\n// The input type should implement the WorkFunction interface\nfunc (w loadWorker) Run(ctx context.Context) interface{} {\n\ttime.Sleep(time.Millisecond * time.Duration(rand.Intn(100)))\n\treturn w * 2\n}\n```\n## Demo \n[Go Playground](https://go.dev/play/p/60b_x0YHzYu)\n\n## Run\n### Example - 1\n```go\nfunc main() {\n\tmax := 10\n\tinputChan := make(chan concurrently.WorkFunction)\n\tctx := context.Background()\n\toutput := concurrently.Process(ctx, inputChan, \u0026concurrently.Options{PoolSize: 10, OutChannelBuffer: 10})\n\tgo func() {\n\t\tfor work := 0; work \u003c max; work++ {\n\t\t\tinputChan \u003c- loadWorker(work)\n\t\t}\n\t\tclose(inputChan)\n\t}()\n\tfor out := range output {\n\t\tlog.Println(out.Value)\n\t}\n}\n```\n### Example - 2 - Process unknown number of inputs\n```go\nfunc main() {\n\tinputChan := make(chan concurrently.WorkFunction, 10)\n\tctx := context.Background()\n\toutput := concurrently.Process(ctx, inputChan, \u0026concurrently.Options{PoolSize: 10, OutChannelBuffer: 10})\n\n\tticker := time.NewTicker(100 * time.Millisecond)\n\tdone := make(chan bool)\n\twg := \u0026sync.WaitGroup{}\n\tgo func() {\n\t\tinput := 0\n\t\tfor {\n\t\t\tselect {\n\t\t\tcase \u003c-done:\n\t\t\t\treturn\n\t\t\tcase \u003c-ticker.C:\n\t\t\t\tinputChan \u003c- loadWorker(input)\n\t\t\t\twg.Add(1)\n\t\t\t\tinput++\n\t\t\tdefault:\n\t\t\t}\n\t\t}\n\t}()\n\n\tvar res []loadWorker\n\tgo func() {\n\t\tfor out := range output {\n\t\t\tres = append(res, out.Value.(loadWorker))\n\t\t\twg.Done()\n\t\t}\n\t}()\n\n\ttime.Sleep(1600 * time.Millisecond)\n\tticker.Stop()\n\tdone \u003c- true\n\tclose(inputChan)\n\twg.Wait()\n\n\t// Check if output is sorted\n\tisSorted := sort.SliceIsSorted(res, func(i, j int) bool {\n\t\treturn res[i] \u003c res[j]\n\t})\n\tif !isSorted {\n\t\tlog.Println(\"output is not sorted\")\n\t}\n}\n```\n# Credits\n1.  [u/justinisrael](https://www.reddit.com/user/justinisrael/) for inputs on improving resource usage.\n2.  [mh-cbon](https://github.com/mh-cbon) for identifying potential [deadlocks](https://github.com/tejzpr/ordered-concurrently/issues/2).\n","funding_links":[],"categories":["Data Structures and Algorithms","Data Structures","Data Integration Frameworks","数据结构与算法","Generators","Uncategorized"],"sub_categories":["Pipes","Advanced Console UIs","管道"],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftejzpr%2Fordered-concurrently","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftejzpr%2Fordered-concurrently","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftejzpr%2Fordered-concurrently/lists"}