https://github.com/kyleross/kinesis-events
AWS Kinesis event parser and handler for Lambda consumers
https://github.com/kyleross/kinesis-events
aws aws-kinesis aws-kinesis-stream events javascript json json-parser kinesis node6 nodejs npm-module parser
Last synced: 5 months ago
JSON representation
AWS Kinesis event parser and handler for Lambda consumers
- Host: GitHub
- URL: https://github.com/kyleross/kinesis-events
- Owner: KyleRoss
- License: mit
- Created: 2017-11-15T18:19:59.000Z (almost 8 years ago)
- Default Branch: master
- Last Pushed: 2019-11-15T21:04:04.000Z (almost 6 years ago)
- Last Synced: 2024-04-14T02:29:26.277Z (over 1 year ago)
- Topics: aws, aws-kinesis, aws-kinesis-stream, events, javascript, json, json-parser, kinesis, node6, nodejs, npm-module, parser
- Language: JavaScript
- Size: 30.3 KB
- Stars: 5
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# kinesis-events
[](https://www.npmjs.com/package/kinesis-events) [](https://www.npmjs.com/package/kinesis-events) [](https://david-dm.org/KyleRoss/kinesis-events) [](https://travis-ci.org/KyleRoss/kinesis-events) [](https://github.com/KyleRoss/kinesis-events/blob/master/LICENSE) [](https://beerpay.io/KyleRoss/kinesis-events)
AWS Kinesis event parser and handler for Lambda consumers. Ability to parse kinesis events with error handling and JSON support. Supports Node 8.10+ on AWS Lambda.
---
## Install
```bash
npm i --save kinesis-events
```
## Usage
```js
const kinesisEvents = require('kinesis-events');
// Lambda function handler
exports.handler = async event => {
// Parse the records
const result = kinesisEvents.parse(event);
// Check for errors (optional)
if(result.hasErrors) {
console.error('There are errors while parsing, ending process...');
process.exit(1);
}
result.records.forEach(record => {
//... iterate through the parsed records
});
};
```
---
### kinesisEvents : [KinesisEvents](#KinesisEvents) ⏏
Instance of the [KinesisEvents](#kinesisevents) class which is exported when calling `require('kinesis-events')`.
For more advanced usage, you may create a new instance of KinesisEvents (see example below).
**Kind**: Exported KinesisEvents Instance
**Example**
```js
const kinesisEvents = require('kinesis-events');
// Advanced usage
const { KinesisEvents } = require('kinesis-events');
const kinesisEvents = new KinesisEvents({
// options...
});
```
## ParseError ⇐ Error
Custom error that is generated when there is a parsing error.
**Kind**: global class
**Extends**: Error
### parseError.payload : String
The original data that caused the error.
**Kind**: instance property of [ParseError](#ParseError)
## KinesisEvents
**Kind**: global class
* [KinesisEvents](#KinesisEvents)
* [new KinesisEvents([options])](#new_KinesisEvents_new)
* [kinesisEvents.options](#KinesisEvents+options) : Object
* [kinesisEvents.ParseError](#KinesisEvents+ParseError) : [ParseError](#ParseError)
* [kinesisEvents.parse(records, [json])](#KinesisEvents+parse) ⇒ [RecordSet](#RecordSet)
### new KinesisEvents([options])
Constructor for KinesisEvents.
| Param | Type | Default | Description |
| --- | --- | --- | --- |
| [options] | Object | {} | Options object to control certain features of KinesisEvents. |
| [options.transform(record, index)] | function | | Optional transform function to call for each record. See [Transform Function](#transform-function). |
### kinesisEvents.options : Object
Options object for KinesisEvents. Allows overridding options after instantiation.
**Kind**: instance property of [KinesisEvents](#KinesisEvents)
**Example**
```js
kinesisEvents.options.transform = function(record, index) {
// transform record...
return record;
};
```
### kinesisEvents.ParseError : [ParseError](#ParseError)
Access to the ParseError class.
**Kind**: instance property of [KinesisEvents](#KinesisEvents)
**Read only**: true
### kinesisEvents.parse(records, [json]) ⇒ [RecordSet](#RecordSet)
Parses records from the incoming Kinesis event.
**Kind**: instance method of [KinesisEvents](#KinesisEvents)
**Returns**: [RecordSet](#RecordSet) - New instance of RecordSet with the parsed records.
| Param | Type | Default | Description |
| --- | --- | --- | --- |
| records | Array | | Event data (records) to parse. |
| [json] | Boolean | true | Enable/disable JSON parsing for each event. |
**Example**
```js
const result = kinesisEvents.parse(event.Records);
result.records.forEach(record => {
// do something with each record...
});
```
## RecordSet
A set of parsed records with additional functionality.
**Kind**: global class
* [RecordSet](#RecordSet)
* [recordSet.records](#RecordSet+records) : Array
* [recordSet.failed](#RecordSet+failed) : [Array.<ParseError>](#ParseError)
* [recordSet.length](#RecordSet+length) : Number
* [recordSet.hasErrors](#RecordSet+hasErrors) : Boolean
### recordSet.records : Array
The records within this record set.
**Kind**: instance property of [RecordSet](#RecordSet)
### recordSet.failed : [Array.<ParseError>](#ParseError)
List of failed records (ParseError).
**Kind**: instance property of [RecordSet](#RecordSet)
### recordSet.length : Number
The total number of parsed records in the record set.
**Kind**: instance property of [RecordSet](#RecordSet)
**Read only**: true
### recordSet.hasErrors : Boolean
Boolean flag if this record set has failed records.
**Kind**: instance property of [RecordSet](#RecordSet)
**Read only**: true
## Transform Function
New in v3.0.0, there is now an option to pass in a transform function that will allow you to transform the record before it is added to the RecordSet. This allows custom functionality or business logic to be implemented at a higher level.
The transform function takes 2 arguments, `record` and `index`. The function must return the transformed record in order for it to be added to the RecordSet. If the record is not returned from the function, it will be ignored.
```js
const { KinesisEvents } = require('kinesis-events');
const kinesisEvents = new KinesisEvents({
transform: (record, index) => {
if(record.firstName && record.lastName) {
// example, remove record if data is missing
return null;
}
record.someCustomProperty = 'some custom value';
return record;
}
});
```
---
## Tests
Tests are written and provided as part of the module. It requires mocha to be installed which is included as a `devDependency`. You may run the tests by calling:
```bash
$ npm run test
```
## License
MIT License. See [License](https://github.com/KyleRoss/kinesis-events/blob/master/LICENSE) in the repository.