{"id":29943260,"url":"https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream","last_synced_at":"2026-02-09T02:02:03.479Z","repository":{"id":248570524,"uuid":"821396497","full_name":"sourcefuse/terraform-aws-arc-kinesis-stream","owner":"sourcefuse","description":"Terraform Module to provision Kinesis Streams","archived":false,"fork":false,"pushed_at":"2024-08-29T16:43:31.000Z","size":4334,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-08-01T05:55:58.794Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sourcefuse.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-06-28T12:54:04.000Z","updated_at":"2024-08-29T16:43:17.000Z","dependencies_parsed_at":"2024-07-29T12:56:27.555Z","dependency_job_id":null,"html_url":"https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream","commit_stats":null,"previous_names":["sourcefuse/terraform-aws-arc-kinesis-stream"],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/sourcefuse/terraform-aws-arc-kinesis-stream","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fterraform-aws-arc-kinesis-stream","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fterraform-aws-arc-kinesis-stream/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fterraform-aws-arc-kinesis-stream/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fterraform-aws-arc-kinesis-stream/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sourcefuse","download_url":"https://codeload.github.com/sourcefuse/terraform-aws-arc-kinesis-stream/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sourcefuse%2Fterraform-aws-arc-kinesis-stream/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29254303,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-09T01:52:29.835Z","status":"online","status_checked_at":"2026-02-09T02:00:09.501Z","response_time":56,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-03T02:14:47.662Z","updated_at":"2026-02-09T02:02:03.464Z","avatar_url":"https://github.com/sourcefuse.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"![Module Structure](./static/banner.png)\n\u003c!-- ALL-CONTRIBUTORS-BADGE:START - Do not remove or modify this section --\u003e\n[![All Contributors](https://img.shields.io/badge/all_contributors-1-orange.svg?style=flat-square)](#contributors-)\n\u003c!-- ALL-CONTRIBUTORS-BADGE:END --\u003e\n# [terraform-aws-arc-kinesis-stream](https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream)\n\n\u003ca href=\"https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream/releases/latest\"\u003e\u003cimg src=\"https://img.shields.io/github/release/sourcefuse/terraform-aws-arc-kinesis-stream.svg?style=for-the-badge\" alt=\"Latest Release\"/\u003e\u003c/a\u003e \u003ca href=\"https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream/commits\"\u003e\u003cimg src=\"https://img.shields.io/github/last-commit/sourcefuse/terraform-aws-arc-kinesis-stream.svg?style=for-the-badge\" alt=\"Last Updated\"/\u003e\u003c/a\u003e ![Terraform](https://img.shields.io/badge/terraform-%235835CC.svg?style=for-the-badge\u0026logo=terraform\u0026logoColor=white) ![GitHub Actions](https://img.shields.io/badge/github%20actions-%232671E5.svg?style=for-the-badge\u0026logo=githubactions\u0026logoColor=white)\n\n\n[![Quality gate](https://sonarcloud.io/api/project_badges/quality_gate?project=sourcefuse_terraform-aws-arc-kinesis-stream\u0026token=b4da1709f70e10c956ed98d2510423b22dde8564)](https://sonarcloud.io/summary/new_code?id=sourcefuse_terraform-aws-arc-kinesis-stream)\n\n[![Known Vulnerabilities](https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream/actions/workflows/snyk.yaml/badge.svg)](https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream/actions/workflows/snyk.yaml)\n## Overview\n\nFor more information about this repository and its usage, please see [Terraform AWS ARC GitHub Kinesis Module Usage Guide](https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream/blob/main/docs/module-usage-guide/README.md).\n\n## Introduction\n\nSourceFuse's AWS Reference Architecture (ARC) Terraform module simplifies the creation and management of essential AWS infrastructure components. It is designed to provision and configure the following resources:\n\n1. Amazon Kinesis Data Stream (aws_kinesis_stream):\n    - Name: A unique name for the Kinesis stream, generated using a local variable.\n    - Shard Count: The number of shards provisioned for the stream. This can be dynamically set based  on  the stream mode (PROVISIONED or ON_DEMAND).\n    - Retention Period: The length of time that data records are accessible after they are added to the stream.\n    - Shard Level Metrics: Metrics to be enabled for monitoring the stream at the shard level.\n    - Enforce Consumer Deletion: A flag to enforce consumer deletion when the stream is deleted.\n    - Encryption Type: The encryption type to be used for the stream.\n    - KMS Key ID: The KMS key ID for encrypting data records in the stream.\n    - Stream Mode Details: Dynamic configuration block to set the stream mode (PROVISIONED or ON_DEMAND).\n    - Tags: Key-value pairs to tag the stream for identification and management.  \n\n2. Amazon Kinesis Stream Consumer (aws_kinesis_stream_consumer):  \n    - Count: The number of consumer instances to create, determined by the var.consumer_count variables.\n    - Name: A unique name for each consumer, generated using a local variable and the count index\n    - Stream ARN: The Amazon Resource Name (ARN) of the Kinesis stream to which the consumer is attached.\n\n### Prerequisites\nBefore using this module, ensure you have the following:\n\n- AWS credentials configured.\n- Terraform installed.\n- A working knowledge of Terraform.\n\n## Getting Started\n\n1. **Define the Module**\n\nInitially, it's essential to define a Terraform module, which is organized as a distinct directory encompassing Terraform configuration files. Within this module directory, input variables and output values must be defined in the variables.tf and outputs.tf files, respectively. The following illustrates an example directory structure:\n\n\n\n```plaintext\nec2/\n|-- main.tf\n|-- variables.tf\n|-- outputs.tf\n```\n\n\n2. **Define Input Variables**\n\nInside the `variables.tf` or in `*.tfvars` file, you should define values for the variables that the module requires.\n\n3. **Use the Module in Your Main Configuration**\nIn your main Terraform configuration file (e.g., main.tf), you can use the module. Specify the source of the module, and version, For Example\n\n```hcl\nmodule \"kinesis\" {\n  source         = \"sourcefuse/arc-kinesis-stream/aws\"\n  version        = \"0.0.1\"\n  name           = \"${var.namespace}-${var.environment}-kinesis\"\n  shard_count    = var.shard_count\n  consumer_count = var.consumer_count\n\n  tags = module.tags.tags\n}\n\n```\n\n4. **Output Values**\n\nInside the `outputs.tf` file of the module, you can define output values that can be referenced in the main configuration. For example:\n\n```hcl\noutput \"name\" {\n  description = \"Name of the Kinesis stream.\"\n  value       = module.kinesis.name\n}\n\noutput \"shard_count\" {\n  description = \"Number of shards provisioned.\"\n  value       = module.kinesis.shard_count\n}\n\noutput \"stream_arn\" {\n  description = \"ARN of the Kinesis stream.\"\n  value       = module.kinesis.stream_arn\n}\n\n```\n\n## First Time Usage\n***uncomment the backend block in [main.tf](./example/main.tf)***\n```shell\nterraform init -backend-config=config.dev.hcl\n```\n***If testing locally, `terraform init` should be fine***\n\nCreate a `dev` workspace\n```shell\nterraform workspace new dev\n```\n\nPlan Terraform\n```shell\nterraform plan -var-file terraform.tfvars\n```\n\nApply Terraform\n```shell\nterraform apply -var-file terraform.tfvars\n```\n\n## Production Setup\n```shell\nterraform init -backend-config=config.prod.hcl\n```\n\nCreate a `prod` workspace\n```shell\nterraform workspace new prod\n```\n\nPlan Terraform\n```shell\nterraform plan -var-file prod.tfvars\n```\n\nApply Terraform\n```shell\nterraform apply -var-file prod.tfvars  \n```\n\n## Cleanup  \nDestroy Terraform\n```shell\nterraform destroy -var-file dev.tfvars\n```\n\n\u003c!-- BEGINNING OF PRE-COMMIT-TERRAFORM DOCS HOOK --\u003e\n## Requirements\n\n| Name | Version |\n|------|---------|\n| \u003ca name=\"requirement_terraform\"\u003e\u003c/a\u003e [terraform](#requirement\\_terraform) | ~\u003e 1.5, \u003c 2.0.0 |\n| \u003ca name=\"requirement_aws\"\u003e\u003c/a\u003e [aws](#requirement\\_aws) | \u003e= 4.0, \u003c 6.0 |\n\n## Providers\n\n| Name | Version |\n|------|---------|\n| \u003ca name=\"provider_aws\"\u003e\u003c/a\u003e [aws](#provider\\_aws) | 5.58.0 |\n\n## Modules\n\nNo modules.\n\n## Resources\n\n| Name | Type |\n|------|------|\n| [aws_kinesis_stream.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_stream) | resource |\n| [aws_kinesis_stream_consumer.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_stream_consumer) | resource |\n\n## Inputs\n\n| Name | Description | Type | Default | Required |\n|------|-------------|------|---------|:--------:|\n| \u003ca name=\"input_consumer_count\"\u003e\u003c/a\u003e [consumer\\_count](#input\\_consumer\\_count) | Number of consumers to register with Kinesis stream | `number` | `0` | no |\n| \u003ca name=\"input_encryption_type\"\u003e\u003c/a\u003e [encryption\\_type](#input\\_encryption\\_type) | The encryption type to use. Acceptable values are `NONE` and `KMS`. | `string` | `\"KMS\"` | no |\n| \u003ca name=\"input_enforce_consumer_deletion\"\u003e\u003c/a\u003e [enforce\\_consumer\\_deletion](#input\\_enforce\\_consumer\\_deletion) | A boolean that indicates all registered consumers should be deregistered from the stream so that the stream can be destroyed without error. | `bool` | `true` | no |\n| \u003ca name=\"input_kms_key_id\"\u003e\u003c/a\u003e [kms\\_key\\_id](#input\\_kms\\_key\\_id) | The GUID for the customer-managed KMS key to use for encryption. | `string` | `\"alias/aws/kinesis\"` | no |\n| \u003ca name=\"input_name\"\u003e\u003c/a\u003e [name](#input\\_name) | Name of kinesis-stream | `string` | n/a | yes |\n| \u003ca name=\"input_retention_period\"\u003e\u003c/a\u003e [retention\\_period](#input\\_retention\\_period) | Length of time data records are accessible after they are added to the stream. The maximum value of a stream's retention period is 168 hours. Minimum value is 24. Default is 24. | `number` | `24` | no |\n| \u003ca name=\"input_shard_count\"\u003e\u003c/a\u003e [shard\\_count](#input\\_shard\\_count) | The number of shards that the stream will use | `number` | `1` | no |\n| \u003ca name=\"input_shard_level_metrics\"\u003e\u003c/a\u003e [shard\\_level\\_metrics](#input\\_shard\\_level\\_metrics) | A list of shard-level CloudWatch metrics which can be enabled for the stream. | `list(string)` | \u003cpre\u003e[\u003cbr\u003e  \"IncomingBytes\",\u003cbr\u003e  \"OutgoingBytes\"\u003cbr\u003e]\u003c/pre\u003e | no |\n| \u003ca name=\"input_stream_mode\"\u003e\u003c/a\u003e [stream\\_mode](#input\\_stream\\_mode) | Specifies the capacity mode of the stream. Must be either `PROVISIONED` or `ON_DEMAND`. If `ON_DEMAND` is used, then `shard_count` is ignored. | `string` | `null` | no |\n| \u003ca name=\"input_tags\"\u003e\u003c/a\u003e [tags](#input\\_tags) | Tags for AWS elasticache redis | `map(string)` | n/a | yes |\n\n## Outputs\n\n| Name | Description |\n|------|-------------|\n| \u003ca name=\"output_consumers\"\u003e\u003c/a\u003e [consumers](#output\\_consumers) | List of consumers registered with Kinesis stream. |\n| \u003ca name=\"output_name\"\u003e\u003c/a\u003e [name](#output\\_name) | Name of the Kinesis stream. |\n| \u003ca name=\"output_shard_count\"\u003e\u003c/a\u003e [shard\\_count](#output\\_shard\\_count) | Number of shards provisioned. |\n| \u003ca name=\"output_stream_arn\"\u003e\u003c/a\u003e [stream\\_arn](#output\\_stream\\_arn) | ARN of the Kinesis stream. |\n\u003c!-- END OF PRE-COMMIT-TERRAFORM DOCS HOOK --\u003e\n\n### Git commits\n\nwhile Contributing or doing git commit please specify the breaking change in your commit message whether its major,minor or patch\n\nFor Example\n\n```sh\ngit commit -m \"your commit message #major\"\n```\nBy specifying this , it will bump the version and if you dont specify this in your commit message then by default it will consider patch and will bump that accordingly\n\n\n\n## Development\n\n### Prerequisites\n\n- [terraform](https://learn.hashicorp.com/terraform/getting-started/install#installing-terraform)\n- [terraform-docs](https://github.com/segmentio/terraform-docs)\n- [pre-commit](https://pre-commit.com/#install)\n- [golang](https://golang.org/doc/install#install)\n- [golint](https://github.com/golang/lint#installation)\n\n### Configurations\n\n- Configure pre-commit hooks\n  ```sh\n  pre-commit install\n  ```\n\n### Tests\n- Tests are available in `test` directory\n- Configure the dependencies\n  ```sh\n  cd test/\n  go mod init github.com/sourcefuse/terraform-aws-refarch-\u003cmodule_name\u003e\n  go get github.com/gruntwork-io/terratest/modules/terraform\n  ```\n- Now execute the test  \n  ```sh\n  go test -timeout  30m\n  ```\n\n## Authors\n\nThis project is authored by:\n- SourceFuse\n\n## Contributors ✨\n\nThanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section --\u003e\n\u003c!-- prettier-ignore-start --\u003e\n\u003c!-- markdownlint-disable --\u003e\n\u003ctable\u003e\n  \u003ctbody\u003e\n    \u003ctr\u003e\n      \u003ctd align=\"center\" valign=\"top\" width=\"14.28%\"\u003e\u003ca href=\"https://github.com/mayank0202\"\u003e\u003cimg src=\"https://avatars.githubusercontent.com/u/83959396?v=4?s=100\" width=\"100px;\" alt=\"Mayank Sharma\"/\u003e\u003cbr /\u003e\u003csub\u003e\u003cb\u003eMayank Sharma\u003c/b\u003e\u003c/sub\u003e\u003c/a\u003e\u003cbr /\u003e\u003ca href=\"#infra-mayank0202\" title=\"Infrastructure (Hosting, Build-Tools, etc)\"\u003e🚇\u003c/a\u003e \u003ca href=\"#maintenance-mayank0202\" title=\"Maintenance\"\u003e🚧\u003c/a\u003e \u003ca href=\"#mentoring-mayank0202\" title=\"Mentoring\"\u003e🧑‍🏫\u003c/a\u003e \u003ca href=\"https://github.com/sourcefuse/terraform-aws-arc-kinesis-stream/commits?author=mayank0202\" title=\"Code\"\u003e💻\u003c/a\u003e\u003c/td\u003e\n    \u003c/tr\u003e\n  \u003c/tbody\u003e\n\u003c/table\u003e\n\n\u003c!-- markdownlint-restore --\u003e\n\u003c!-- prettier-ignore-end --\u003e\n\n\u003c!-- ALL-CONTRIBUTORS-LIST:END --\u003e\n\nThis project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefuse%2Fterraform-aws-arc-kinesis-stream","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsourcefuse%2Fterraform-aws-arc-kinesis-stream","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsourcefuse%2Fterraform-aws-arc-kinesis-stream/lists"}