{"id":19014529,"url":"https://github.com/deeptiman/go-batch","last_synced_at":"2025-07-03T01:32:17.495Z","repository":{"id":44774362,"uuid":"358899112","full_name":"Deeptiman/go-batch","owner":"Deeptiman","description":"A Simple Batch Processing library in Go","archived":false,"fork":false,"pushed_at":"2023-06-26T19:39:39.000Z","size":7899,"stargazers_count":23,"open_issues_count":1,"forks_count":4,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-23T00:44:01.853Z","etag":null,"topics":["batch-processing","batch-reader","concurrency","concurrent-programming","data-science","go-library","go-modules","golang","golang-channel","golang-concurrency","golang-library","golang-tools","parallel-computing","parallel-processing","parallel-programming","supply-chain-data-science","workers"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Deeptiman.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2021-04-17T14:24:50.000Z","updated_at":"2025-04-03T16:28:33.000Z","dependencies_parsed_at":"2022-08-28T08:22:36.011Z","dependency_job_id":"cac6a2ea-e8e1-4721-8ad2-6f84bc6b6182","html_url":"https://github.com/Deeptiman/go-batch","commit_stats":{"total_commits":35,"total_committers":2,"mean_commits":17.5,"dds":0.02857142857142858,"last_synced_commit":"e7a1457b0eb787620c357dec8eaa05bcd373fe58"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/Deeptiman/go-batch","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deeptiman%2Fgo-batch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deeptiman%2Fgo-batch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deeptiman%2Fgo-batch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deeptiman%2Fgo-batch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Deeptiman","download_url":"https://codeload.github.com/Deeptiman/go-batch/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Deeptiman%2Fgo-batch/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":263243888,"owners_count":23436388,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["batch-processing","batch-reader","concurrency","concurrent-programming","data-science","go-library","go-modules","golang","golang-channel","golang-concurrency","golang-library","golang-tools","parallel-computing","parallel-processing","parallel-programming","supply-chain-data-science","workers"],"created_at":"2024-11-08T19:29:51.393Z","updated_at":"2025-07-03T01:32:17.422Z","avatar_url":"https://github.com/Deeptiman.png","language":"Go","readme":"# go-batch\ngo-batch is a batch processing library written in Go. The process execution has multiple stages to release a Batch to the client.\n\n![GitHub top language](https://img.shields.io/github/languages/top/Deeptiman/go-batch) [![Go Report Card](https://goreportcard.com/badge/github.com/Deeptiman/go-batch)](https://goreportcard.com/report/github.com/Deeptiman/go-batch)\n\n## Features\n\n1. Client can use this library as an asynchronous batch processing for their application use case.\n2. There are no restrictions on applying batch processing matrices to the library. The client can define the maximum no of items for a batch using the \u003cb\u003eBatchOptions\u003c/b\u003e.\n3. The library has a Workerpool that will faster the batch processing in concurrent scenarios.\n\n## Demo\n\n   [![asciicast](https://asciinema.org/a/2vi5gAHjsuTrB3tCBTGeSW6hq.svg)](https://asciinema.org/a/2vi5gAHjsuTrB3tCBTGeSW6hq) \n\n## Stages\n\n 1. Batch Reader receives the resource payload from the client and marshals the payload item into the BatchItem object.\n\n    ``````````````````````````\n    type BatchItems struct {\n        Id      int\n        BatchNo int\n        Item    interface{}\n    } \n\n    ``````````````````````````\n2. BatchProducer has a \u003cb\u003eWatcher\u003c/b\u003e channel that receives the marshal payload from the Batch reader. Watcher marks each BatchItem with a BatchNo and adds it to the []BatchItems array. After the batch itemCounter++ increases to the MaxItems [DefaultMaxItems: 100], the Batch gets\nreleases to the Consumer callback function.\n\n3. BatchConsumer has a \u003cb\u003eConsumerFunc\u003c/b\u003e that gets invoke by BatchProducer as a callback function to send the prepared []BatchItems arrays. Then, the Consumer channel sends the \n[]BatchItems to the Worker channel.\n\n4. Workerline is the sync.WaitGroup synchronizes the workers to send the []BatchItems to the supply chain.\n\n5. BatchSupplyChannel works as a bidirectional channel that requests for the []BatchItems to the Workerline and gets in the response.\n\n6. ClientSupplyChannel is the delivery channel that works as a Supply line to sends the []BatchItems and the client receives by listening to the channel.\n\n\n## Go Docs\n\n   Documentation at \u003ca href=\"https://pkg.go.dev/github.com/Deeptiman/go-batch\"\u003epkg.go.dev\u003c/a\u003e\n\n## Installation\n\n    go get github.com/Deeptiman/go-batch\n\n## Example\n\n   ```````````````````````````````````````````````````` \n   b := batch.NewBatch(batch.WithMaxItems(100))\n   go b.StartBatchProcessing()\n\n   for i := 1; i \u003c= 1000; i++ {\n        b.Item \u003c- \u0026Resources{\n            id:   i,\n            name: fmt.Sprintf(\"%s%d\", \"R-\", i),\n            flag: false,\n        }\n    }\n   b.Close() \n  ````````````````````````````````````````````````````\n  \n## Note\n- In this version release, the library doesn't support starting concurrent BatchProcessing sessions. \n\n## License\n\nThis project is licensed under the \u003ca href=\"https://github.com/Deeptiman/go-batch/blob/main/LICENSE\"\u003eMIT License\u003c/a\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeptiman%2Fgo-batch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdeeptiman%2Fgo-batch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdeeptiman%2Fgo-batch/lists"}