{"id":20427878,"url":"https://github.com/reselbob/kinesis-streamer","last_synced_at":"2026-05-07T04:36:29.789Z","repository":{"id":126509350,"uuid":"427468159","full_name":"reselbob/kinesis-streamer","owner":"reselbob","description":"This project is a demonstration Node.JS application that sends messages with structured, random data continuously to a predefined AWS Kinesis Stream.","archived":false,"fork":false,"pushed_at":"2021-11-19T16:24:11.000Z","size":22,"stargazers_count":2,"open_issues_count":0,"forks_count":2,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-03-05T05:26:25.027Z","etag":null,"topics":["aws","kinesis","nodejs","streaming"],"latest_commit_sha":null,"homepage":"","language":"JavaScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/reselbob.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-12T19:03:10.000Z","updated_at":"2023-10-24T19:32:29.000Z","dependencies_parsed_at":"2023-06-17T01:30:46.514Z","dependency_job_id":null,"html_url":"https://github.com/reselbob/kinesis-streamer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/reselbob/kinesis-streamer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reselbob%2Fkinesis-streamer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reselbob%2Fkinesis-streamer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reselbob%2Fkinesis-streamer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reselbob%2Fkinesis-streamer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/reselbob","download_url":"https://codeload.github.com/reselbob/kinesis-streamer/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/reselbob%2Fkinesis-streamer/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":274384643,"owners_count":25275300,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-09-09T02:00:10.223Z","response_time":80,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","kinesis","nodejs","streaming"],"created_at":"2024-11-15T07:22:56.374Z","updated_at":"2025-10-07T20:25:09.150Z","avatar_url":"https://github.com/reselbob.png","language":"JavaScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kinesis-streamer\nThis project is a demonstration Node.JS application that sends messages with structured, random data continuously to a predefined AWS Kinesis Stream.\n\nThe way the application works is that it creates a number of CronJobs, with each [CronJob](https://www.npmjs.com/package/cron) running every second. The CronJob's logic sends one or many messages to a Kinesis Stream. This Kinesis Stream must be running under AWS before the application starts.\n\nThe credentials required to access to the Kinesis Stream are defined according to the environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.\n\nThe name of a particular Kinesis Stream to bind to is also defined by an environment variable `AWS_KINESIS_STREAM_NAME`. These values are in a file named `.env`.\n\nThe file `.env` needs to be created and configured. (A section that follows describes the details of creating and configuring the file `.env`.) Also, overriding the default number of CronJobs that will run simultaneously and the number of messages that will be sent by each CronJobs are defined in the `.env` file. \n\n# Installation\n\n`npm install`\n\n# Testing\n\n`npm test`\n\nThe tests that run in this project check that the credential information is available in the runtime environment. Also, a test will check that a ShardId is returned from the Kinesis Stream to which the application is bound.\n\nOperationally this means that all the values assigned to the [required environment variables](https://github.com/reselbob/kinesis-streamer#configuring-the-environment-variables) work as expected.\n\n# Running the project\n\nThis project depends on the existence of an accessible Kinesis Stream running on AWS.\n\n\n## Configuring the environment variables\n\nThis project uses the [`dotenv`](https://www.npmjs.com/package/dotenv) NPM package to inject environment variables into the application at runtime.\n\nThis project looks in the root directory for a file named `.env`. This project does **not** ship with the `.env` file. You need to create it in the root of this project.\n\nThe environment variables are defined in the `.env` file like so:\n\n```text\nAWS_ACCESS_KEY_ID=\"a^ACcEsS_KEY_tokEN!\"\nAWS_SECRET_ACCESS_KEY=\"a^SecRET_accESS_k3y_TOken\"\nAWS_KINESIS_STREAM_NAME=my-stream-kinesis\nCRON_JOBS_TO_GENERATE=50\nMESSAGES_PER_CRON_JOB=20\n```\n\nThe following sections describe the particulars of each environment variable.\n\n----\n\n**`AWS_ACCESS_KEY_ID`**\n\nREQUIRED\n\nThe access key that gets generated by AWS when a user is created within the AWS Dashboard or via the AWS CLI tool.\n\n`AWS_ACCESS_KEY_ID=\"\u003cYOUR_AWS_ACCESS_KEY_ID\u003e\"`\n\n**Example:**\n\n`GKIAWSDDTX85L1PDD3EG`\n\n----\n**`AWS_SECRET_ACCESS_KEY`**\n\nREQUIRED\n\nThe secret access key that gets generated by AWS when a user is created.\n\n`AWS_SECRET_ACCESS_KEY=\"\u003cYOUR_AWS_SECRET_ACCESS_KEY\u003e\"`\n\n**Example:**\n\n`+hjIhOwyOUkD0inqvu0EJiRs+XmlXmLawmJGE4V`\n\n----\n\n**`AWS_KINESIS_STREAM_NAME`**\n\nREQUIRED\n\nThis project reads the name for the Kinesis stream from environment variable. Thus, you need to set the environment variable as follows:\n\n`AWS_KINESIS_STREAM_NAME=\u003cSTREAM_NAME\u003e`\n\n**Example:**\n\n`AWS_KINESIS_STREAM_NAME=my-stream-kinesis`\n\n----\n\n**`CRON_JOBS_TO_GENERATE`**\n\nOPTIONAL\n\nDefines to the number of CronJobs to generate and run against the AWS Kinesis stream. If you do not provide this environment variable, the default number of CronJobs generated is 10.\n\n**Example:**\n\n`CRON_JOBS_TO_GENERATE=50`\n\n----\n\n**`MESSAGES_PER_CRON_JOB`**\n\nOPTIONAL\n\nDefines to the number of messages to generate to send to the AWS Kinesis Stream in a single submission. If you do not provide this environment variable, the default number of CronJobs generated is 10.\n\n**Example:**\n\n`MESSAGES_PER_CRON_JOB=20`\n\n----\n\n## Starting the application\n\nTo start the application, execute the following command:\n\n`npm start`","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freselbob%2Fkinesis-streamer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Freselbob%2Fkinesis-streamer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Freselbob%2Fkinesis-streamer/lists"}