https://github.com/practo/gobatch
Batch Processing
https://github.com/practo/gobatch
Last synced: 5 months ago
JSON representation
Batch Processing
- Host: GitHub
- URL: https://github.com/practo/gobatch
- Owner: practo
- License: apache-2.0
- Fork: true (herryg91/gobatch)
- Created: 2020-08-10T06:57:00.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2020-08-22T08:59:24.000Z (almost 6 years ago)
- Last Synced: 2024-06-20T05:15:58.200Z (almost 2 years ago)
- Language: Go
- Size: 1.05 MB
- Stars: 1
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# gobatch
Simple batch library for Golang.
## How to Use
Install:
```
go get github.com/herryg91/gobatch
```
Example:
```
func fn1(workerID int, datas []interface{}) (err error) {
//do something
return
}
// every 100 datas or 15 second no activity, batch will be processed (fn1 will be run) with 2 worker
mBatch := gobatch.NewMemoryBatch(fn1, 100, time.Second*15, 2)
mBatch.Insert(interface{}{})
mBatch.Insert(interface{}{})
mBatch.Insert(interface{}{})
```
Need additional param to DoFn? You can do something like this
```
type additionalParam struct {
RedisPool *redis.Pool
}
func (p additionalParam) fn1(workerID int, datas []interface{}) (err error) {
// do something log.Println(p)
return
}
mBatch := gobatch.NewMemoryBatch(additionalParam{pool}.fn1, 100, time.Second*15, 2)
```
## Future Update:
- Batch using Redis
- Batch using File
- Add / Decrease Worker on the fly