https://github.com/itsteknas/pooled-batch-processor
A utility wrapper around es6-promise-pool that lets you process data arrays in batches
https://github.com/itsteknas/pooled-batch-processor
batch-processing node node-module pooling
Last synced: 10 months ago
JSON representation
A utility wrapper around es6-promise-pool that lets you process data arrays in batches
- Host: GitHub
- URL: https://github.com/itsteknas/pooled-batch-processor
- Owner: itsTeknas
- Created: 2018-07-14T05:42:13.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2018-07-17T18:13:25.000Z (over 7 years ago)
- Last Synced: 2025-01-07T01:01:17.118Z (about 1 year ago)
- Topics: batch-processing, node, node-module, pooling
- Language: JavaScript
- Size: 3.91 KB
- Stars: 0
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# pooled-batch-processor
A utility wrapper around es6-promise-pool that lets you process data arrays in batches
## Installation
```bash
npm install pooled-batch-processor --save
```
## Usage
```js
let pool = require('pooled-batch-processor')
let longDataArray = [
1,
2,
...
]
pool.processInPool(longDataArray, 5, 10, (batch) => {
// This function should return a promise.
return new Promise(resolve => {
// Process the batch
// each batch will have 5 items from longDataArray
// this will be run in 10 promises at a time simultaneously
// If split is set to 1,
// you will get individual items in place of a array.
resolve();
})
}).then(() => {
// All items processed
})
```