{"id":15705855,"url":"https://github.com/kyleross/kinesis-events","last_synced_at":"2025-05-12T18:44:21.780Z","repository":{"id":75753648,"uuid":"110870091","full_name":"KyleRoss/kinesis-events","owner":"KyleRoss","description":"AWS Kinesis event parser and handler for Lambda consumers","archived":false,"fork":false,"pushed_at":"2019-11-15T21:04:04.000Z","size":31,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2024-04-14T02:29:26.277Z","etag":null,"topics":["aws","aws-kinesis","aws-kinesis-stream","events","javascript","json","json-parser","kinesis","node6","nodejs","npm-module","parser"],"latest_commit_sha":null,"homepage":null,"language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/KyleRoss.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-11-15T18:19:59.000Z","updated_at":"2020-12-10T20:37:05.000Z","dependencies_parsed_at":"2023-06-28T09:56:51.090Z","dependency_job_id":null,"html_url":"https://github.com/KyleRoss/kinesis-events","commit_stats":{"total_commits":27,"total_committers":2,"mean_commits":13.5,"dds":0.07407407407407407,"last_synced_commit":"4d757c7e149ba67817b580326339649329c7544f"},"previous_names":[],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KyleRoss%2Fkinesis-events","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KyleRoss%2Fkinesis-events/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KyleRoss%2Fkinesis-events/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/KyleRoss%2Fkinesis-events/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/KyleRoss","download_url":"https://codeload.github.com/KyleRoss/kinesis-events/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221278117,"owners_count":16790243,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","aws-kinesis","aws-kinesis-stream","events","javascript","json","json-parser","kinesis","node6","nodejs","npm-module","parser"],"created_at":"2024-10-03T20:20:30.197Z","updated_at":"2024-10-24T05:01:14.175Z","avatar_url":"https://github.com/KyleRoss.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kinesis-events\n[![npm](https://img.shields.io/npm/v/kinesis-events.svg?style=for-the-badge)](https://www.npmjs.com/package/kinesis-events) [![npm](https://img.shields.io/npm/dt/kinesis-events.svg?style=for-the-badge)](https://www.npmjs.com/package/kinesis-events) [![David](https://img.shields.io/david/KyleRoss/kinesis-events.svg?style=for-the-badge)](https://david-dm.org/KyleRoss/kinesis-events) [![Travis](https://img.shields.io/travis/KyleRoss/kinesis-events/master.svg?style=for-the-badge)](https://travis-ci.org/KyleRoss/kinesis-events) [![license](https://img.shields.io/github/license/KyleRoss/kinesis-events.svg?style=for-the-badge)](https://github.com/KyleRoss/kinesis-events/blob/master/LICENSE) [![Beerpay](https://img.shields.io/beerpay/KyleRoss/kinesis-events.svg?style=for-the-badge)](https://beerpay.io/KyleRoss/kinesis-events)\n\nAWS Kinesis event parser and handler for Lambda consumers. Ability to parse kinesis events with error handling and JSON support. Supports Node 8.10+ on AWS Lambda.\n\n---\n\n## Install\n```bash\nnpm i --save kinesis-events\n```\n\n## Usage\n```js\nconst kinesisEvents = require('kinesis-events');\n\n// Lambda function handler\nexports.handler = async event =\u003e {\n    // Parse the records\n    const result = kinesisEvents.parse(event);\n    \n    // Check for errors (optional)\n    if(result.hasErrors) {\n        console.error('There are errors while parsing, ending process...');\n        process.exit(1);\n    }\n    \n    result.records.forEach(record =\u003e {\n        //... iterate through the parsed records\n    });\n};\n```\n\n---\n\n# API Documentation\n\u003ca name=\"module_kinesis-events\"\u003e\u003c/a\u003e\n\n## kinesis-events\n\u003ca name=\"exp_module_kinesis-events--kinesisEvents\"\u003e\u003c/a\u003e\n\n### kinesisEvents : [\u003ccode\u003eKinesisEvents\u003c/code\u003e](#KinesisEvents) ⏏\nInstance of the [KinesisEvents](#kinesisevents) class which is exported when calling `require('kinesis-events')`.\nFor more advanced usage, you may create a new instance of KinesisEvents (see example below).\n\n**Kind**: Exported KinesisEvents Instance  \n**Example**  \n```js\nconst kinesisEvents = require('kinesis-events');\n\n// Advanced usage\nconst { KinesisEvents } = require('kinesis-events');\nconst kinesisEvents = new KinesisEvents({\n    // options...\n});\n```\n\u003ca name=\"ParseError\"\u003e\u003c/a\u003e\n\n## ParseError ⇐ \u003ccode\u003eError\u003c/code\u003e\nCustom error that is generated when there is a parsing error.\n\n**Kind**: global class  \n**Extends**: \u003ccode\u003eError\u003c/code\u003e  \n\u003ca name=\"ParseError+payload\"\u003e\u003c/a\u003e\n\n### parseError.payload : \u003ccode\u003eString\u003c/code\u003e\nThe original data that caused the error.\n\n**Kind**: instance property of [\u003ccode\u003eParseError\u003c/code\u003e](#ParseError)  \n\u003ca name=\"KinesisEvents\"\u003e\u003c/a\u003e\n\n## KinesisEvents\n**Kind**: global class  \n\n* [KinesisEvents](#KinesisEvents)\n    * [new KinesisEvents([options])](#new_KinesisEvents_new)\n    * [kinesisEvents.options](#KinesisEvents+options) : \u003ccode\u003eObject\u003c/code\u003e\n    * [kinesisEvents.ParseError](#KinesisEvents+ParseError) : [\u003ccode\u003eParseError\u003c/code\u003e](#ParseError)\n    * [kinesisEvents.parse(records, [json])](#KinesisEvents+parse) ⇒ [\u003ccode\u003eRecordSet\u003c/code\u003e](#RecordSet)\n\n\u003ca name=\"new_KinesisEvents_new\"\u003e\u003c/a\u003e\n\n### new KinesisEvents([options])\nConstructor for KinesisEvents.\n\n\n| Param | Type | Default | Description |\n| --- | --- | --- | --- |\n| [options] | \u003ccode\u003eObject\u003c/code\u003e | \u003ccode\u003e{}\u003c/code\u003e | Options object to control certain features of KinesisEvents. |\n| [options.transform(record, index)] | \u003ccode\u003efunction\u003c/code\u003e |  | Optional transform function to call for each record. See [Transform Function](#transform-function). |\n\n\u003ca name=\"KinesisEvents+options\"\u003e\u003c/a\u003e\n\n### kinesisEvents.options : \u003ccode\u003eObject\u003c/code\u003e\nOptions object for KinesisEvents. Allows overridding options after instantiation.\n\n**Kind**: instance property of [\u003ccode\u003eKinesisEvents\u003c/code\u003e](#KinesisEvents)  \n**Example**  \n```js\nkinesisEvents.options.transform = function(record, index) {\n    // transform record...\n    return record;\n};\n```\n\u003ca name=\"KinesisEvents+ParseError\"\u003e\u003c/a\u003e\n\n### kinesisEvents.ParseError : [\u003ccode\u003eParseError\u003c/code\u003e](#ParseError)\nAccess to the ParseError class.\n\n**Kind**: instance property of [\u003ccode\u003eKinesisEvents\u003c/code\u003e](#KinesisEvents)  \n**Read only**: true  \n\u003ca name=\"KinesisEvents+parse\"\u003e\u003c/a\u003e\n\n### kinesisEvents.parse(records, [json]) ⇒ [\u003ccode\u003eRecordSet\u003c/code\u003e](#RecordSet)\nParses records from the incoming Kinesis event.\n\n**Kind**: instance method of [\u003ccode\u003eKinesisEvents\u003c/code\u003e](#KinesisEvents)  \n**Returns**: [\u003ccode\u003eRecordSet\u003c/code\u003e](#RecordSet) - New instance of RecordSet with the parsed records.  \n\n| Param | Type | Default | Description |\n| --- | --- | --- | --- |\n| records | \u003ccode\u003eArray\u003c/code\u003e |  | Event data (records) to parse. |\n| [json] | \u003ccode\u003eBoolean\u003c/code\u003e | \u003ccode\u003etrue\u003c/code\u003e | Enable/disable JSON parsing for each event. |\n\n**Example**  \n```js\nconst result = kinesisEvents.parse(event.Records);\n\nresult.records.forEach(record =\u003e {\n    // do something with each record...\n});\n```\n\u003ca name=\"RecordSet\"\u003e\u003c/a\u003e\n\n## RecordSet\nA set of parsed records with additional functionality.\n\n**Kind**: global class  \n\n* [RecordSet](#RecordSet)\n    * [recordSet.records](#RecordSet+records) : \u003ccode\u003eArray\u003c/code\u003e\n    * [recordSet.failed](#RecordSet+failed) : [\u003ccode\u003eArray.\u0026lt;ParseError\u0026gt;\u003c/code\u003e](#ParseError)\n    * [recordSet.length](#RecordSet+length) : \u003ccode\u003eNumber\u003c/code\u003e\n    * [recordSet.hasErrors](#RecordSet+hasErrors) : \u003ccode\u003eBoolean\u003c/code\u003e\n\n\u003ca name=\"RecordSet+records\"\u003e\u003c/a\u003e\n\n### recordSet.records : \u003ccode\u003eArray\u003c/code\u003e\nThe records within this record set.\n\n**Kind**: instance property of [\u003ccode\u003eRecordSet\u003c/code\u003e](#RecordSet)  \n\u003ca name=\"RecordSet+failed\"\u003e\u003c/a\u003e\n\n### recordSet.failed : [\u003ccode\u003eArray.\u0026lt;ParseError\u0026gt;\u003c/code\u003e](#ParseError)\nList of failed records (ParseError).\n\n**Kind**: instance property of [\u003ccode\u003eRecordSet\u003c/code\u003e](#RecordSet)  \n\u003ca name=\"RecordSet+length\"\u003e\u003c/a\u003e\n\n### recordSet.length : \u003ccode\u003eNumber\u003c/code\u003e\nThe total number of parsed records in the record set.\n\n**Kind**: instance property of [\u003ccode\u003eRecordSet\u003c/code\u003e](#RecordSet)  \n**Read only**: true  \n\u003ca name=\"RecordSet+hasErrors\"\u003e\u003c/a\u003e\n\n### recordSet.hasErrors : \u003ccode\u003eBoolean\u003c/code\u003e\nBoolean flag if this record set has failed records.\n\n**Kind**: instance property of [\u003ccode\u003eRecordSet\u003c/code\u003e](#RecordSet)  \n**Read only**: true  \n## Transform Function\nNew in v3.0.0, there is now an option to pass in a transform function that will allow you to transform the record before it is added to the RecordSet. This allows custom functionality or business logic to be implemented at a higher level.\n\nThe transform function takes 2 arguments, `record` and `index`. The function must return the transformed record in order for it to be added to the RecordSet. If the record is not returned from the function, it will be ignored.\n\n```js\nconst { KinesisEvents } = require('kinesis-events');\nconst kinesisEvents = new KinesisEvents({\n    transform: (record, index) =\u003e {\n        if(record.firstName \u0026\u0026 record.lastName) {\n            // example, remove record if data is missing\n            return null;\n        }\n        \n        record.someCustomProperty = 'some custom value';\n        return record;\n    }\n});\n```\n\n---\n\n## Tests\nTests are written and provided as part of the module. It requires mocha to be installed which is included as a `devDependency`. You may run the tests by calling:\n\n```bash\n$ npm run test\n```\n\n## License\nMIT License. See [License](https://github.com/KyleRoss/kinesis-events/blob/master/LICENSE) in the repository.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyleross%2Fkinesis-events","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkyleross%2Fkinesis-events","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkyleross%2Fkinesis-events/lists"}