https://github.com/jonathanong/bulk-insert
A writable stream that batches
https://github.com/jonathanong/bulk-insert
Last synced: about 2 months ago
JSON representation
A writable stream that batches
- Host: GitHub
- URL: https://github.com/jonathanong/bulk-insert
- Owner: jonathanong
- License: mit
- Created: 2016-09-15T23:05:50.000Z (over 8 years ago)
- Default Branch: master
- Last Pushed: 2016-11-05T08:41:39.000Z (over 8 years ago)
- Last Synced: 2025-03-28T18:11:59.031Z (3 months ago)
- Language: JavaScript
- Size: 6.84 KB
- Stars: 5
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Asymmetrical Signing
[![NPM version][npm-image]][npm-url]
[![Build status][travis-image]][travis-url]
[![Test coverage][codecov-image]][codecov-url]
[![Dependency Status][david-image]][david-url]
[![License][license-image]][license-url]
[![Downloads][downloads-image]][downloads-url]
A writable stream that batches.
## Example
```js
const createBulk = require('bulk-insert')
const onError = (err) => {
if (err) console.error(err.stack || err)
}
const writable = createBulk({
limit: 500, // maximum # of documents to insert at one time
interval: '0.5s', // minimum interval between flushes,
onError,
flush (data) {
// `data` will be an array
kinesis.putRecords({
Records: data.map((x) => ({
Data: JSON.stringify(x),
PartitionKey: 'some_key'
})),
StreamName: 'some_stream_name'
}, onError)
}
})
writable.write({
some: 'data'
})
writable.write({
some: 'more data'
})
```
## API
### const writable = bulkInsert(options)
Options:
- `limit: 1000` - maximum # of documents to insert at one time
- `interval: '300ms'` - minimum interval between flushes,
- `onError` - optional function that handles `flush()` errors
- `flush` - a function with the signature `(data) => {}`
### writable.write(data)
Write data to the stream.
### writable.flush()
Flush all the data immediately.
### writable.close()
Flushes immediately and unrefs all future timers, allowing the process to exit gracefully.
Even though you are still able to write to the stream after you've closed it, you should not.
[npm-image]: https://img.shields.io/npm/v/bulk-insert.svg?style=flat-square
[npm-url]: https://npmjs.org/package/bulk-insert
[travis-image]: https://img.shields.io/travis/jonathanong/bulk-insert/master.svg?style=flat-square
[travis-url]: https://travis-ci.org/jonathanong/bulk-insert
[codecov-image]: https://img.shields.io/codecov/c/github/jonathanong/bulk-insert/master.svg?style=flat-square
[codecov-url]: https://codecov.io/github/jonathanong/bulk-insert
[david-image]: http://img.shields.io/david/jonathanong/bulk-insert.svg?style=flat-square
[david-url]: https://david-dm.org/jonathanong/bulk-insert
[license-image]: http://img.shields.io/npm/l/bulk-insert.svg?style=flat-square
[license-url]: LICENSE
[downloads-image]: http://img.shields.io/npm/dm/bulk-insert.svg?style=flat-square
[downloads-url]: https://npmjs.org/package/bulk-insert