Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/fdmsantos/terraform-aws-kinesis-firehose
Dynamic Terraform module, which creates a Kinesis Firehose Stream and others resources like Cloudwatch, IAM Roles and Security Groups that integrate with Kinesis Firehose. Supports all destinations and all Kinesis Firehose Features.
https://github.com/fdmsantos/terraform-aws-kinesis-firehose
aws coralogix datadog dynatrace elasticsearch honeycomb kinesis-firehose logicmonitor mongodb msk newrelic opensearch opensearch-serverless redshift s3 snowflake splunk sumologic terraform terraform-module
Last synced: 2 months ago
JSON representation
Dynamic Terraform module, which creates a Kinesis Firehose Stream and others resources like Cloudwatch, IAM Roles and Security Groups that integrate with Kinesis Firehose. Supports all destinations and all Kinesis Firehose Features.
- Host: GitHub
- URL: https://github.com/fdmsantos/terraform-aws-kinesis-firehose
- Owner: fdmsantos
- License: apache-2.0
- Created: 2022-09-06T15:48:52.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-11-08T11:36:45.000Z (3 months ago)
- Last Synced: 2024-11-08T12:30:39.294Z (3 months ago)
- Topics: aws, coralogix, datadog, dynatrace, elasticsearch, honeycomb, kinesis-firehose, logicmonitor, mongodb, msk, newrelic, opensearch, opensearch-serverless, redshift, s3, snowflake, splunk, sumologic, terraform, terraform-module
- Language: HCL
- Homepage: https://registry.terraform.io/modules/fdmsantos/kinesis-firehose/aws/latest
- Size: 450 KB
- Stars: 11
- Watchers: 1
- Forks: 8
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# AWS Kinesis Firehose Terraform module
[](https://github.com/semantic-release/semantic-release)
Dynamic Terraform module, which creates a Kinesis Firehose Stream and others resources like Cloudwatch, IAM Roles and Security Groups that integrate with Kinesis Firehose.
Supports all destinations and all Kinesis Firehose Features.
## Table of Contents
* [Table of Contents](#table-of-contents)
* [Module versioning rule](#module-versioning-rule)
* [Features](#features)
* [How to Use](#how-to-use)
* [Sources](#sources)
* [Kinesis Data Stream](#kinesis-data-stream)
* [Kinesis Data Stream Encrypted](#kinesis-data-stream-encrypted)
* [Direct Put](#direct-put)
* [WAF](#waf)
* [MSK](#msk)
* [Destinations](#destinations)
* [S3](#s3)
* [Redshift](#redshift)
* [Elasticsearch](#elasticsearch)
* [Opensearch](#opensearch)
* [Opensearch Serverless](#opensearch-serverless)
* [Splunk](#splunk)
* [Snowflake](#snowflake)
* [HTTP Endpoint](#http-endpoint)
* [Datadog](#datadog)
* [New Relic](#new-relic)
* [Coralogix](#coralogix)
* [Dynatrace](#dynatrace)
* [Honeycomb](#honeycomb)
* [Logic Monitor](#logic-monitor)
* [MongoDB](#mongodb)
* [SumoLogic](#sumologic)
* [Iceberg](#iceberg)
* [Server Side Encryption](#server-side-encryption)
* [Data Transformation with Lambda](#data-transformation-with-lambda)
* [Data Format Conversion](#data-format-conversion)
* [Dynamic Partition](#dynamic-partition)
* [S3 Backup Data](#s3-backup-data)
* [Destination Delivery Logging](#destination-delivery-logging)
* [VPC Support](#vpc-support)
* [ElasticSearch / Opensearch / Opensearch Serverless](#elasticsearch--opensearch--opensearch-serverless)
* [Redshift / Splunk](#redshift--splunk)
* [Application Role](#application-role)
* [Secrets Manager](#secrets-manager)
* [Destinations Mapping](#destinations-mapping)
* [Examples](#examples)
* [Requirements](#requirements)
* [Providers](#providers)
* [Modules](#modules)
* [Resources](#resources)
* [Inputs](#inputs)
* [Outputs](#outputs)
* [Upgrade](#upgrade)
* [Deprecations](#deprecations)
* [Version 3.1.0](#version-310)
* [Version 3.3.0](#version-330)
* [License](#license)
## Module versioning rule
| Module version | AWS Provider version |
|----------------|----------------------|
| >= 1.x.x | ~> 4.4 |
| >= 2.x.x | ~> 5.0 |
| >= 3.x.x | >= 5.33 |
## Features
- Sources
- Kinesis Data Stream
- Direct Put
- WAF
- MSK
- Destinations
- S3
- Data Format Conversion
- Dynamic Partition
- Redshift
- VPC Support. Security Groups creation supported.
- Support to Secrets Manager.
- ElasticSearch / Opensearch / Opensearch Serverless
- VPC Support. Security Groups creation supported.
- Splunk
- VPC Support. Security Groups creation supported.
- Support to Secrets Manager.
- Snowflake
- VPCE Support.
- Support to Secrets Manager
- Custom Http Endpoint
- Support to Secrets Manager
- DataDog
- Support to Secrets Manager
- Coralogix
- Support to Secrets Manager
- New Relic
- Support to Secrets Manager
- Dynatrace
- Support to Secrets Manager
- Honeycomb
- Support to Secrets Manager
- Logic Monitor
- Support to Secrets Manager
- MongoDB Cloud
- Support to Secrets Manager
- Sumo Logic
- Data Transformation With Lambda
- Original Data Backup in S3
- Logging and Encryption
- Application Role to Direct Put Sources
- Turn on/off cloudwatch logs decompressing and data message extraction
- Permissions
- IAM Roles
- Opensearch / Opensearch Serverless Service Role
- Associate Role to Redshift Cluster Iam Roles
- Cross Account S3 Bucket Policy
- Cross Account Elasticsearch / OpenSearch / Opensearch Serverless Service policy
## How to Use
### Sources
#### Kinesis Data Stream
**To Enabled it:** `input_source = "kinesis"`.
```hcl
module "firehose" {
source = "fdmsantos/kinesis-firehose/aws"
version = "x.x.x"
name = "firehose-delivery-stream"
input_source = "kinesis"
kinesis_source_stream_arn = ""
destination = "s3" # or destination = "extended_s3"
s3_bucket_arn = ""
}
```
##### Kinesis Data Stream Encrypted
If Kinesis Data Stream is encrypted, it's necessary pass this info to module .
**To Enabled It:** `input_source = "kinesis"`.
**KMS Key:** use `kinesis_source_kms_arn` variable to indicate the KMS Key to module add permissions to policy to decrypt the Kinesis Data Stream.
#### Direct Put
**To Enabled it:** `input_source = "direct-put"`.
```hcl
module "firehose" {
source = "fdmsantos/kinesis-firehose/aws"
version = "x.x.x"
name = "firehose-delivery-stream"
input_source = "direct-put"
destination = "s3" # or destination = "extended_s3"
s3_bucket_arn = ""
}
```
#### WAF
**To Enabled it:** `input_source = "waf"`.
```hcl
module "firehose" {
source = "fdmsantos/kinesis-firehose/aws"
version = "x.x.x"
name = "firehose-delivery-stream"
input_source = "waf"
destination = "s3" # or destination = "extended_s3"
s3_bucket_arn = ""
}
```
#### MSK
**To Enabled it:** `input_source = "msk"`.
```hcl
module "firehose" {
source = "fdmsantos/kinesis-firehose/aws"
version = "x.x.x"
name = "firehose-delivery-stream"
input_source = "msk"
msk_source_cluster_arn = ""
msk_source_topic_name = "test"
destination = "s3"
s3_bucket_arn = ""
}
```
### Destinations
#### S3
**To Enabled It:** `destination = "s3" or destination = "extended_s3"`
**Variables Prefix:** `s3_`
**To Enable Encryption:** `enable_s3_encryption = true`
**Note:** For other destinations, the `s3_` variables are used to configure the required intermediary bucket before delivery data to destination. Not Supported to Elasticsearch, Splunk and http destinations
```hcl
module "firehose" {
source = "fdmsantos/kinesis-firehose/aws"
version = "x.x.x"
name = "firehose-delivery-stream"
destination = "s3" # or destination = "extended_s3"
s3_bucket_arn = ""
}
```
#### Redshift
**To Enabled It:** `destination = "redshift"`
**Variables Prefix:** `redshift_`
```hcl
module "firehose" {
source = "fdmsantos/kinesis-firehose/aws"
version = "x.x.x"
name = "firehose-delivery-stream"
destination = "redshift"
s3_bucket_arn = ""
redshift_cluster_identifier = ""
redshift_cluster_endpoint = ""
redshift_database_name = ""
redshift_username = ""
redshift_password = ""
redshift_table_name = ""
redshift_copy_options = "json 'auto ignorecase'"
}
```
#### Elasticsearch
**To Enabled It:** `destination = "elasticsearch"`
**Variables Prefix:** `elasticsearch_`
```hcl
module "firehose" {
source = "fdmsantos/kinesis-firehose/aws"
version = "x.x.x"
name = "firehose-delivery-stream"
destination = "elasticsearch"
elasticsearch_domain_arn = ""
elasticsearch_index_name = "
## Requirements
| Name | Version |
|------|---------|
| [terraform](#requirement\_terraform) | >= 0.13.1 |
| [aws](#requirement\_aws) | >= 5.73 |
## Providers
| Name | Version |
|------|---------|
| [aws](#provider\_aws) | >= 5.73 |
## Modules
No modules.
## Resources
| Name | Type |
|------|------|
| [aws_cloudwatch_log_group.log](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |
| [aws_cloudwatch_log_stream.backup](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_stream) | resource |
| [aws_cloudwatch_log_stream.destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_stream) | resource |
| [aws_iam_policy.application](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.cw](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.elasticsearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.glue](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.iceberg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.kinesis](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.msk](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.opensearchserverless](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.s3_kms](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.secretsmanager](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.secretsmanager_cmk_encryption](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_policy.vpc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_policy) | resource |
| [aws_iam_role.application](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role) | resource |
| [aws_iam_role_policy_attachment.application](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.cw](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.elasticsearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.glue](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.iceberg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.kinesis](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.msk](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.opensearchserverless](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.s3_kms](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.secretsmanager](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.secretsmanager_cmk_encryption](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_role_policy_attachment.vpc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_role_policy_attachment) | resource |
| [aws_iam_service_linked_role.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
| [aws_iam_service_linked_role.opensearchserverless](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |
| [aws_kinesis_firehose_delivery_stream.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/kinesis_firehose_delivery_stream) | resource |
| [aws_redshift_cluster_iam_roles.this](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/redshift_cluster_iam_roles) | resource |
| [aws_security_group.destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group) | resource |
| [aws_security_group_rule.destination](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_security_group_rule.firehose](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_security_group_rule.firehose_egress_rule](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/security_group_rule) | resource |
| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |
| [aws_iam_policy_document.application](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.application_assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.assume_role](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cross_account_elasticsearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cross_account_opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cross_account_opensearchserverless](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cross_account_s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.cw](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.elasticsearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.glue](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.iceberg](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.kinesis](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.lambda](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.msk](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.opensearchserverless](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.s3](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.s3_kms](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.secretsmanager](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.secretsmanager_cmk_encryption](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_iam_policy_document.vpc](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |
| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |
| [aws_subnet.subnet](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/subnet) | data source |
## Inputs
| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| [append\_delimiter\_to\_record](#input\_append\_delimiter\_to\_record) | To configure your delivery stream to add a new line delimiter between records in objects that are delivered to Amazon S3. | `bool` | `false` | no |
| [application\_role\_description](#input\_application\_role\_description) | Description of IAM Application role to use for Kinesis Firehose Stream Source | `string` | `null` | no |
| [application\_role\_force\_detach\_policies](#input\_application\_role\_force\_detach\_policies) | Specifies to force detaching any policies the IAM Application role has before destroying it | `bool` | `true` | no |
| [application\_role\_name](#input\_application\_role\_name) | Name of IAM Application role to use for Kinesis Firehose Stream Source | `string` | `null` | no |
| [application\_role\_path](#input\_application\_role\_path) | Path of IAM Application role to use for Kinesis Firehose Stream Source | `string` | `null` | no |
| [application\_role\_permissions\_boundary](#input\_application\_role\_permissions\_boundary) | The ARN of the policy that is used to set the permissions boundary for the IAM Application role used by Kinesis Firehose Stream Source | `string` | `null` | no |
| [application\_role\_policy\_actions](#input\_application\_role\_policy\_actions) | List of Actions to Application Role Policy | `list(string)` |
[
"firehose:PutRecord",
"firehose:PutRecordBatch"
]
| no |
| [application\_role\_service\_principal](#input\_application\_role\_service\_principal) | AWS Service Principal to assume application role | `string` | `null` | no |
| [application\_role\_tags](#input\_application\_role\_tags) | A map of tags to assign to IAM Application role | `map(string)` | `{}` | no |
| [associate\_role\_to\_redshift\_cluster](#input\_associate\_role\_to\_redshift\_cluster) | Set it to false if don't want the module associate the role to redshift cluster | `bool` | `true` | no |
| [buffering\_interval](#input\_buffering\_interval) | Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination | `number` | `300` | no |
| [buffering\_size](#input\_buffering\_size) | Buffer incoming data to the specified size, in MBs, before delivering it to the destination. | `number` | `5` | no |
| [configure\_existing\_application\_role](#input\_configure\_existing\_application\_role) | Set it to True if want use existing application role to add the firehose Policy | `bool` | `false` | no |
| [coralogix\_endpoint\_location](#input\_coralogix\_endpoint\_location) | Endpoint Location to coralogix destination | `string` | `"ireland"` | no |
| [coralogix\_parameter\_application\_name](#input\_coralogix\_parameter\_application\_name) | By default, your delivery stream arn will be used as applicationName. | `string` | `null` | no |
| [coralogix\_parameter\_subsystem\_name](#input\_coralogix\_parameter\_subsystem\_name) | By default, your delivery stream name will be used as subsystemName. | `string` | `null` | no |
| [coralogix\_parameter\_use\_dynamic\_values](#input\_coralogix\_parameter\_use\_dynamic\_values) | To use dynamic values for applicationName and subsystemName. | `bool` | `false` | no |
| [create](#input\_create) | Controls if kinesis firehose should be created (it affects almost all resources) | `bool` | `true` | no |
| [create\_application\_role](#input\_create\_application\_role) | Set it to true to create role to be used by the source | `bool` | `false` | no |
| [create\_application\_role\_policy](#input\_create\_application\_role\_policy) | Set it to true to create policy to the role used by the source | `bool` | `false` | no |
| [create\_destination\_cw\_log\_group](#input\_create\_destination\_cw\_log\_group) | Enables or disables the cloudwatch log group creation to destination | `bool` | `true` | no |
| [create\_role](#input\_create\_role) | Controls whether IAM role for Kinesis Firehose Stream should be created | `bool` | `true` | no |
| [cw\_log\_retention\_in\_days](#input\_cw\_log\_retention\_in\_days) | Specifies the number of days you want to retain log events in the specified log group. Possible values are: 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, and 3653. | `number` | `null` | no |
| [cw\_tags](#input\_cw\_tags) | A map of tags to assign to the resource. | `map(string)` | `{}` | no |
| [data\_format\_conversion\_block\_size](#input\_data\_format\_conversion\_block\_size) | The Hadoop Distributed File System (HDFS) block size. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The Value is in Bytes. | `number` | `268435456` | no |
| [data\_format\_conversion\_glue\_catalog\_id](#input\_data\_format\_conversion\_glue\_catalog\_id) | The ID of the AWS Glue Data Catalog. If you don't supply this, the AWS account ID is used by default. | `string` | `null` | no |
| [data\_format\_conversion\_glue\_database](#input\_data\_format\_conversion\_glue\_database) | Name of the AWS Glue database that contains the schema for the output data. | `string` | `null` | no |
| [data\_format\_conversion\_glue\_region](#input\_data\_format\_conversion\_glue\_region) | If you don't specify an AWS Region, the default is the current region. | `string` | `null` | no |
| [data\_format\_conversion\_glue\_role\_arn](#input\_data\_format\_conversion\_glue\_role\_arn) | The role that Kinesis Data Firehose can use to access AWS Glue. This role must be in the same account you use for Kinesis Data Firehose. Cross-account roles aren't allowed. | `string` | `null` | no |
| [data\_format\_conversion\_glue\_table\_name](#input\_data\_format\_conversion\_glue\_table\_name) | Specifies the AWS Glue table that contains the column information that constitutes your data schema | `string` | `null` | no |
| [data\_format\_conversion\_glue\_use\_existing\_role](#input\_data\_format\_conversion\_glue\_use\_existing\_role) | Indicates if want use the kinesis firehose role to glue access. | `bool` | `true` | no |
| [data\_format\_conversion\_glue\_version\_id](#input\_data\_format\_conversion\_glue\_version\_id) | Specifies the table version for the output data schema. | `string` | `"LATEST"` | no |
| [data\_format\_conversion\_hive\_timestamps](#input\_data\_format\_conversion\_hive\_timestamps) | A list of how you want Kinesis Data Firehose to parse the date and time stamps that may be present in your input data JSON. To specify these format strings, follow the pattern syntax of JodaTime's DateTimeFormat format strings. | `list(string)` | `[]` | no |
| [data\_format\_conversion\_input\_format](#input\_data\_format\_conversion\_input\_format) | Specifies which deserializer to use. You can choose either the Apache Hive JSON SerDe or the OpenX JSON SerDe | `string` | `"OpenX"` | no |
| [data\_format\_conversion\_openx\_case\_insensitive](#input\_data\_format\_conversion\_openx\_case\_insensitive) | When set to true, Kinesis Data Firehose converts JSON keys to lowercase before deserializing them. | `bool` | `true` | no |
| [data\_format\_conversion\_openx\_column\_to\_json\_key\_mappings](#input\_data\_format\_conversion\_openx\_column\_to\_json\_key\_mappings) | A map of column names to JSON keys that aren't identical to the column names. This is useful when the JSON contains keys that are Hive keywords. | `map(string)` | `null` | no |
| [data\_format\_conversion\_openx\_convert\_dots\_to\_underscores](#input\_data\_format\_conversion\_openx\_convert\_dots\_to\_underscores) | Specifies that the names of the keys include dots and that you want Kinesis Data Firehose to replace them with underscores. This is useful because Apache Hive does not allow dots in column names. | `bool` | `false` | no |
| [data\_format\_conversion\_orc\_bloom\_filter\_columns](#input\_data\_format\_conversion\_orc\_bloom\_filter\_columns) | A list of column names for which you want Kinesis Data Firehose to create bloom filters. | `list(string)` | `[]` | no |
| [data\_format\_conversion\_orc\_bloom\_filter\_false\_positive\_probability](#input\_data\_format\_conversion\_orc\_bloom\_filter\_false\_positive\_probability) | The Bloom filter false positive probability (FPP). The lower the FPP, the bigger the Bloom filter. | `number` | `0.05` | no |
| [data\_format\_conversion\_orc\_compression](#input\_data\_format\_conversion\_orc\_compression) | The compression code to use over data blocks. | `string` | `"SNAPPY"` | no |
| [data\_format\_conversion\_orc\_dict\_key\_threshold](#input\_data\_format\_conversion\_orc\_dict\_key\_threshold) | A float that represents the fraction of the total number of non-null rows. To turn off dictionary encoding, set this fraction to a number that is less than the number of distinct keys in a dictionary. To always use dictionary encoding, set this threshold to 1. | `number` | `0` | no |
| [data\_format\_conversion\_orc\_enable\_padding](#input\_data\_format\_conversion\_orc\_enable\_padding) | Set this to true to indicate that you want stripes to be padded to the HDFS block boundaries. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. | `bool` | `false` | no |
| [data\_format\_conversion\_orc\_format\_version](#input\_data\_format\_conversion\_orc\_format\_version) | The version of the file to write. | `string` | `"V0_12"` | no |
| [data\_format\_conversion\_orc\_padding\_tolerance](#input\_data\_format\_conversion\_orc\_padding\_tolerance) | A float between 0 and 1 that defines the tolerance for block padding as a decimal fraction of stripe size. | `number` | `0.05` | no |
| [data\_format\_conversion\_orc\_row\_index\_stripe](#input\_data\_format\_conversion\_orc\_row\_index\_stripe) | The number of rows between index entries. | `number` | `10000` | no |
| [data\_format\_conversion\_orc\_stripe\_size](#input\_data\_format\_conversion\_orc\_stripe\_size) | he number of bytes in each strip. | `number` | `67108864` | no |
| [data\_format\_conversion\_output\_format](#input\_data\_format\_conversion\_output\_format) | Specifies which serializer to use. You can choose either the ORC SerDe or the Parquet SerDe | `string` | `"PARQUET"` | no |
| [data\_format\_conversion\_parquet\_compression](#input\_data\_format\_conversion\_parquet\_compression) | The compression code to use over data blocks. | `string` | `"SNAPPY"` | no |
| [data\_format\_conversion\_parquet\_dict\_compression](#input\_data\_format\_conversion\_parquet\_dict\_compression) | Indicates whether to enable dictionary compression. | `bool` | `false` | no |
| [data\_format\_conversion\_parquet\_max\_padding](#input\_data\_format\_conversion\_parquet\_max\_padding) | The maximum amount of padding to apply. This is useful if you intend to copy the data from Amazon S3 to HDFS before querying. The value is in bytes | `number` | `0` | no |
| [data\_format\_conversion\_parquet\_page\_size](#input\_data\_format\_conversion\_parquet\_page\_size) | Column chunks are divided into pages. A page is conceptually an indivisible unit (in terms of compression and encoding). The value is in bytes | `number` | `1048576` | no |
| [data\_format\_conversion\_parquet\_writer\_version](#input\_data\_format\_conversion\_parquet\_writer\_version) | Indicates the version of row format to output. | `string` | `"V1"` | no |
| [datadog\_endpoint\_type](#input\_datadog\_endpoint\_type) | Endpoint type to datadog destination | `string` | `"logs_eu"` | no |
| [destination](#input\_destination) | This is the destination to where the data is delivered | `string` | n/a | yes |
| [destination\_cross\_account](#input\_destination\_cross\_account) | Indicates if destination is in a different account. Only supported to Elasticsearch and OpenSearch | `bool` | `false` | no |
| [destination\_log\_group\_name](#input\_destination\_log\_group\_name) | The CloudWatch group name for destination logs | `string` | `null` | no |
| [destination\_log\_stream\_name](#input\_destination\_log\_stream\_name) | The CloudWatch log stream name for destination logs | `string` | `null` | no |
| [dynamic\_partition\_append\_delimiter\_to\_record](#input\_dynamic\_partition\_append\_delimiter\_to\_record) | DEPRECATED!! Use var append\_delimiter\_to\_record instead!! Use To configure your delivery stream to add a new line delimiter between records in objects that are delivered to Amazon S3. | `bool` | `false` | no |
| [dynamic\_partition\_enable\_record\_deaggregation](#input\_dynamic\_partition\_enable\_record\_deaggregation) | Data deaggregation is the process of parsing through the records in a delivery stream and separating the records based either on valid JSON or on the specified delimiter | `bool` | `false` | no |
| [dynamic\_partition\_metadata\_extractor\_query](#input\_dynamic\_partition\_metadata\_extractor\_query) | Dynamic Partition JQ query. | `string` | `null` | no |
| [dynamic\_partition\_record\_deaggregation\_delimiter](#input\_dynamic\_partition\_record\_deaggregation\_delimiter) | Specifies the delimiter to be used for parsing through the records in the delivery stream and deaggregating them | `string` | `null` | no |
| [dynamic\_partition\_record\_deaggregation\_type](#input\_dynamic\_partition\_record\_deaggregation\_type) | Data deaggregation is the process of parsing through the records in a delivery stream and separating the records based either on valid JSON or on the specified delimiter | `string` | `"JSON"` | no |
| [dynamic\_partitioning\_retry\_duration](#input\_dynamic\_partitioning\_retry\_duration) | Total amount of seconds Firehose spends on retries | `number` | `300` | no |
| [dynatrace\_api\_url](#input\_dynatrace\_api\_url) | API URL to Dynatrace destination | `string` | `null` | no |
| [dynatrace\_endpoint\_location](#input\_dynatrace\_endpoint\_location) | Endpoint Location to Dynatrace destination | `string` | `"eu"` | no |
| [elasticsearch\_domain\_arn](#input\_elasticsearch\_domain\_arn) | The ARN of the Amazon ES domain. The pattern needs to be arn:.* | `string` | `null` | no |
| [elasticsearch\_index\_name](#input\_elasticsearch\_index\_name) | The Elasticsearch index name | `string` | `null` | no |
| [elasticsearch\_index\_rotation\_period](#input\_elasticsearch\_index\_rotation\_period) | The Elasticsearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data | `string` | `"OneDay"` | no |
| [elasticsearch\_retry\_duration](#input\_elasticsearch\_retry\_duration) | The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt | `string` | `300` | no |
| [elasticsearch\_type\_name](#input\_elasticsearch\_type\_name) | The Elasticsearch type name with maximum length of 100 characters | `string` | `null` | no |
| [enable\_cloudwatch\_logs\_data\_message\_extraction](#input\_enable\_cloudwatch\_logs\_data\_message\_extraction) | Cloudwatch Logs data message extraction | `bool` | `false` | no |
| [enable\_cloudwatch\_logs\_decompression](#input\_enable\_cloudwatch\_logs\_decompression) | Enables or disables Cloudwatch Logs decompression | `bool` | `false` | no |
| [enable\_data\_format\_conversion](#input\_enable\_data\_format\_conversion) | Set it to true if you want to disable format conversion. | `bool` | `false` | no |
| [enable\_destination\_log](#input\_enable\_destination\_log) | The CloudWatch Logging Options for the delivery stream | `bool` | `true` | no |
| [enable\_dynamic\_partitioning](#input\_enable\_dynamic\_partitioning) | Enables or disables dynamic partitioning | `bool` | `false` | no |
| [enable\_lambda\_transform](#input\_enable\_lambda\_transform) | Set it to true to enable data transformation with lambda | `bool` | `false` | no |
| [enable\_s3\_backup](#input\_enable\_s3\_backup) | The Amazon S3 backup mode | `bool` | `false` | no |
| [enable\_s3\_encryption](#input\_enable\_s3\_encryption) | Indicates if want use encryption in S3 bucket. | `bool` | `false` | no |
| [enable\_secrets\_manager](#input\_enable\_secrets\_manager) | Enables or disables the Secrets Manager configuration. | `bool` | `false` | no |
| [enable\_sse](#input\_enable\_sse) | Whether to enable encryption at rest. Only makes sense when source is Direct Put | `bool` | `false` | no |
| [enable\_vpc](#input\_enable\_vpc) | Indicates if destination is configured in VPC. Supports Elasticsearch and Opensearch destinations. | `bool` | `false` | no |
| [firehose\_role](#input\_firehose\_role) | IAM role ARN attached to the Kinesis Firehose Stream. | `string` | `null` | no |
| [honeycomb\_api\_host](#input\_honeycomb\_api\_host) | If you use a Secure Tenancy or other proxy, put its schema://host[:port] here | `string` | `"https://api.honeycomb.io"` | no |
| [honeycomb\_dataset\_name](#input\_honeycomb\_dataset\_name) | Your Honeycomb dataset name to Honeycomb destination | `string` | `null` | no |
| [http\_endpoint\_access\_key](#input\_http\_endpoint\_access\_key) | The access key required for Kinesis Firehose to authenticate with the HTTP endpoint selected as the destination | `string` | `null` | no |
| [http\_endpoint\_enable\_request\_configuration](#input\_http\_endpoint\_enable\_request\_configuration) | The request configuration | `bool` | `false` | no |
| [http\_endpoint\_name](#input\_http\_endpoint\_name) | The HTTP endpoint name | `string` | `null` | no |
| [http\_endpoint\_request\_configuration\_common\_attributes](#input\_http\_endpoint\_request\_configuration\_common\_attributes) | Describes the metadata sent to the HTTP endpoint destination. The variable is list. Each element is map with two keys , name and value, that corresponds to common attribute name and value | `list(map(string))` | `[]` | no |
| [http\_endpoint\_request\_configuration\_content\_encoding](#input\_http\_endpoint\_request\_configuration\_content\_encoding) | Kinesis Data Firehose uses the content encoding to compress the body of a request before sending the request to the destination | `string` | `"GZIP"` | no |
| [http\_endpoint\_retry\_duration](#input\_http\_endpoint\_retry\_duration) | Total amount of seconds Firehose spends on retries. This duration starts after the initial attempt fails, It does not include the time periods during which Firehose waits for acknowledgment from the specified destination after each attempt | `number` | `300` | no |
| [http\_endpoint\_url](#input\_http\_endpoint\_url) | The HTTP endpoint URL to which Kinesis Firehose sends your data | `string` | `null` | no |
| [iceberg\_catalog\_arn](#input\_iceberg\_catalog\_arn) | Glue catalog ARN identifier of the destination Apache Iceberg Tables. You must specify the ARN in the format arn:aws:glue:region:account-id:catalog. | `string` | `null` | no |
| [iceberg\_database\_name](#input\_iceberg\_database\_name) | The name of the Apache Iceberg database. | `string` | `null` | no |
| [iceberg\_destination\_config\_s3\_error\_output\_prefix](#input\_iceberg\_destination\_config\_s3\_error\_output\_prefix) | The table specific S3 error output prefix. All the errors that occurred while delivering to this table will be prefixed with this value in S3 destination. | `string` | `null` | no |
| [iceberg\_destination\_config\_unique\_keys](#input\_iceberg\_destination\_config\_unique\_keys) | A list of unique keys for a given Apache Iceberg table. Firehose will use these for running Create, Update, or Delete operations on the given Iceberg table. | `list(string)` | `[]` | no |
| [iceberg\_retry\_duration](#input\_iceberg\_retry\_duration) | The period of time, in seconds between 0 to 7200, during which Firehose retries to deliver data to the specified destination. | `number` | `300` | no |
| [iceberg\_table\_name](#input\_iceberg\_table\_name) | The name of the Apache Iceberg Table. | `string` | `null` | no |
| [input\_source](#input\_input\_source) | This is the kinesis firehose source | `string` | `"direct-put"` | no |
| [kinesis\_source\_is\_encrypted](#input\_kinesis\_source\_is\_encrypted) | Indicates if Kinesis data stream source is encrypted | `bool` | `false` | no |
| [kinesis\_source\_kms\_arn](#input\_kinesis\_source\_kms\_arn) | Kinesis Source KMS Key to add Firehose role to decrypt the records. | `string` | `null` | no |
| [kinesis\_source\_role\_arn](#input\_kinesis\_source\_role\_arn) | DEPRECATED!! Use variable instead source\_role\_arn! The ARN of the role that provides access to the source Kinesis stream | `string` | `null` | no |
| [kinesis\_source\_stream\_arn](#input\_kinesis\_source\_stream\_arn) | The kinesis stream used as the source of the firehose delivery stream | `string` | `null` | no |
| [kinesis\_source\_use\_existing\_role](#input\_kinesis\_source\_use\_existing\_role) | DEPRECATED!! Use variable source\_use\_existing\_role instead! Indicates if want use the kinesis firehose role to kinesis data stream access. | `bool` | `true` | no |
| [logicmonitor\_account](#input\_logicmonitor\_account) | Account to use in Logic Monitor destination | `string` | `null` | no |
| [mongodb\_realm\_webhook\_url](#input\_mongodb\_realm\_webhook\_url) | Realm Webhook URL to use in MongoDB destination | `string` | `null` | no |
| [msk\_source\_cluster\_arn](#input\_msk\_source\_cluster\_arn) | The ARN of the Amazon MSK cluster. | `string` | `null` | no |
| [msk\_source\_connectivity\_type](#input\_msk\_source\_connectivity\_type) | The type of connectivity used to access the Amazon MSK cluster. Valid values: PUBLIC, PRIVATE. | `string` | `"PUBLIC"` | no |
| [msk\_source\_topic\_name](#input\_msk\_source\_topic\_name) | The topic name within the Amazon MSK cluster. | `string` | `null` | no |
| [name](#input\_name) | A name to identify the stream. This is unique to the AWS account and region the Stream is created in | `string` | n/a | yes |
| [newrelic\_endpoint\_type](#input\_newrelic\_endpoint\_type) | Endpoint type to New Relic destination | `string` | `"logs_eu"` | no |
| [opensearch\_document\_id\_options](#input\_opensearch\_document\_id\_options) | The method for setting up document ID. | `string` | `"FIREHOSE_DEFAULT"` | no |
| [opensearch\_domain\_arn](#input\_opensearch\_domain\_arn) | The ARN of the Amazon Opensearch domain. The pattern needs to be arn:.*. Conflicts with cluster\_endpoint. | `string` | `null` | no |
| [opensearch\_index\_name](#input\_opensearch\_index\_name) | The Opensearch (And OpenSearch Serverless) index name. | `string` | `null` | no |
| [opensearch\_index\_rotation\_period](#input\_opensearch\_index\_rotation\_period) | The Opensearch index rotation period. Index rotation appends a timestamp to the IndexName to facilitate expiration of old data | `string` | `"OneDay"` | no |
| [opensearch\_retry\_duration](#input\_opensearch\_retry\_duration) | After an initial failure to deliver to Amazon OpenSearch, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt). After this time has elapsed, the failed documents are written to Amazon S3. The default value is 300s. There will be no retry if the value is 0. | `string` | `300` | no |
| [opensearch\_type\_name](#input\_opensearch\_type\_name) | The opensearch type name with maximum length of 100 characters. Types are deprecated in OpenSearch\_1.1. TypeName must be empty. | `string` | `null` | no |
| [opensearch\_vpc\_create\_service\_linked\_role](#input\_opensearch\_vpc\_create\_service\_linked\_role) | Set it to True if want create Opensearch Service Linked Role to Access VPC. | `bool` | `false` | no |
| [opensearchserverless\_collection\_arn](#input\_opensearchserverless\_collection\_arn) | The ARN of the Amazon Opensearch Serverless Collection. The pattern needs to be arn:.*. | `string` | `null` | no |
| [opensearchserverless\_collection\_endpoint](#input\_opensearchserverless\_collection\_endpoint) | The endpoint to use when communicating with the collection in the Serverless offering for Amazon OpenSearch Service. | `string` | `null` | no |
| [policy\_path](#input\_policy\_path) | Path of policies to that should be added to IAM role for Kinesis Firehose Stream | `string` | `null` | no |
| [redshift\_cluster\_endpoint](#input\_redshift\_cluster\_endpoint) | The redshift endpoint | `string` | `null` | no |
| [redshift\_cluster\_identifier](#input\_redshift\_cluster\_identifier) | Redshift Cluster identifier. Necessary to associate the iam role to cluster | `string` | `null` | no |
| [redshift\_copy\_options](#input\_redshift\_copy\_options) | Copy options for copying the data from the s3 intermediate bucket into redshift, for example to change the default delimiter | `string` | `null` | no |
| [redshift\_data\_table\_columns](#input\_redshift\_data\_table\_columns) | The data table columns that will be targeted by the copy command | `string` | `null` | no |
| [redshift\_database\_name](#input\_redshift\_database\_name) | The redshift database name | `string` | `null` | no |
| [redshift\_password](#input\_redshift\_password) | The password for the redshift username above | `string` | `null` | no |
| [redshift\_retry\_duration](#input\_redshift\_retry\_duration) | The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt | `string` | `3600` | no |
| [redshift\_table\_name](#input\_redshift\_table\_name) | The name of the table in the redshift cluster that the s3 bucket will copy to | `string` | `null` | no |
| [redshift\_username](#input\_redshift\_username) | The username that the firehose delivery stream will assume. It is strongly recommended that the username and password provided is used exclusively for Amazon Kinesis Firehose purposes, and that the permissions for the account are restricted for Amazon Redshift INSERT permissions | `string` | `null` | no |
| [role\_description](#input\_role\_description) | Description of IAM role to use for Kinesis Firehose Stream | `string` | `null` | no |
| [role\_force\_detach\_policies](#input\_role\_force\_detach\_policies) | Specifies to force detaching any policies the IAM role has before destroying it | `bool` | `true` | no |
| [role\_name](#input\_role\_name) | Name of IAM role to use for Kinesis Firehose Stream | `string` | `null` | no |
| [role\_path](#input\_role\_path) | Path of IAM role to use for Kinesis Firehose Stream | `string` | `null` | no |
| [role\_permissions\_boundary](#input\_role\_permissions\_boundary) | The ARN of the policy that is used to set the permissions boundary for the IAM role used by Kinesis Firehose Stream | `string` | `null` | no |
| [role\_tags](#input\_role\_tags) | A map of tags to assign to IAM role | `map(string)` | `{}` | no |
| [s3\_backup\_bucket\_arn](#input\_s3\_backup\_bucket\_arn) | The ARN of the S3 backup bucket | `string` | `null` | no |
| [s3\_backup\_buffering\_interval](#input\_s3\_backup\_buffering\_interval) | Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. | `number` | `300` | no |
| [s3\_backup\_buffering\_size](#input\_s3\_backup\_buffering\_size) | Buffer incoming data to the specified size, in MBs, before delivering it to the destination. | `number` | `5` | no |
| [s3\_backup\_compression](#input\_s3\_backup\_compression) | The compression format | `string` | `"UNCOMPRESSED"` | no |
| [s3\_backup\_create\_cw\_log\_group](#input\_s3\_backup\_create\_cw\_log\_group) | Enables or disables the cloudwatch log group creation | `bool` | `true` | no |
| [s3\_backup\_enable\_encryption](#input\_s3\_backup\_enable\_encryption) | Indicates if want enable KMS Encryption in S3 Backup Bucket. | `bool` | `false` | no |
| [s3\_backup\_enable\_log](#input\_s3\_backup\_enable\_log) | Enables or disables the logging | `bool` | `true` | no |
| [s3\_backup\_error\_output\_prefix](#input\_s3\_backup\_error\_output\_prefix) | Prefix added to failed records before writing them to S3 | `string` | `null` | no |
| [s3\_backup\_kms\_key\_arn](#input\_s3\_backup\_kms\_key\_arn) | Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used. | `string` | `null` | no |
| [s3\_backup\_log\_group\_name](#input\_s3\_backup\_log\_group\_name) | he CloudWatch group name for logging | `string` | `null` | no |
| [s3\_backup\_log\_stream\_name](#input\_s3\_backup\_log\_stream\_name) | The CloudWatch log stream name for logging | `string` | `null` | no |
| [s3\_backup\_mode](#input\_s3\_backup\_mode) | Defines how documents should be delivered to Amazon S3. Used to elasticsearch, opensearch, splunk, http configurations. For S3 and Redshift use enable\_s3\_backup | `string` | `"FailedOnly"` | no |
| [s3\_backup\_prefix](#input\_s3\_backup\_prefix) | The YYYY/MM/DD/HH time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket | `string` | `null` | no |
| [s3\_backup\_role\_arn](#input\_s3\_backup\_role\_arn) | The role that Kinesis Data Firehose can use to access S3 Backup. | `string` | `null` | no |
| [s3\_backup\_use\_existing\_role](#input\_s3\_backup\_use\_existing\_role) | Indicates if want use the kinesis firehose role to s3 backup bucket access. | `bool` | `true` | no |
| [s3\_bucket\_arn](#input\_s3\_bucket\_arn) | The ARN of the S3 destination bucket | `string` | `null` | no |
| [s3\_compression\_format](#input\_s3\_compression\_format) | The compression format | `string` | `"UNCOMPRESSED"` | no |
| [s3\_configuration\_buffering\_interval](#input\_s3\_configuration\_buffering\_interval) | Buffer incoming data for the specified period of time, in seconds, before delivering it to the destination. | `number` | `300` | no |
| [s3\_configuration\_buffering\_size](#input\_s3\_configuration\_buffering\_size) | Buffer incoming data to the specified size, in MBs, before delivering it to the destination. The default value is 5. We recommend setting SizeInMBs to a value greater than the amount of data you typically ingest into the delivery stream in 10 seconds. For example, if you typically ingest data at 1 MB/sec set SizeInMBs to be 10 MB or higher. | `number` | `5` | no |
| [s3\_cross\_account](#input\_s3\_cross\_account) | Indicates if S3 bucket destination is in a different account | `bool` | `false` | no |
| [s3\_custom\_time\_zone](#input\_s3\_custom\_time\_zone) | The time zone you prefer. Valid values are UTC or a non-3-letter IANA time zones (for example, America/Los\_Angeles). Default value is UTC. | `string` | `"UTC"` | no |
| [s3\_error\_output\_prefix](#input\_s3\_error\_output\_prefix) | Prefix added to failed records before writing them to S3. This prefix appears immediately following the bucket name. | `string` | `null` | no |
| [s3\_file\_extension](#input\_s3\_file\_extension) | The file extension to override the default file extension (for example, .json). | `string` | `null` | no |
| [s3\_kms\_key\_arn](#input\_s3\_kms\_key\_arn) | Specifies the KMS key ARN the stream will use to encrypt data. If not set, no encryption will be used | `string` | `null` | no |
| [s3\_own\_bucket](#input\_s3\_own\_bucket) | Indicates if you own the bucket. If not, will be configure permissions to grants the bucket owner full access to the objects delivered by Kinesis Data Firehose | `bool` | `true` | no |
| [s3\_prefix](#input\_s3\_prefix) | The YYYY/MM/DD/HH time format prefix is automatically used for delivered S3 files. You can specify an extra prefix to be added in front of the time format prefix. Note that if the prefix ends with a slash, it appears as a folder in the S3 bucket | `string` | `null` | no |
| [secret\_arn](#input\_secret\_arn) | The ARN of the Secrets Manager secret. This value is required if enable\_secrets\_manager is true. | `string` | `null` | no |
| [secret\_kms\_key\_arn](#input\_secret\_kms\_key\_arn) | The ARN of the KMS Key to encrypt the Secret. This value is required if key used to encrypt the Secret is CMK and want the module generates the IAM Policy to access it. | `string` | `null` | no |
| [snowflake\_account\_identifier](#input\_snowflake\_account\_identifier) | The Snowflake account identifier. | `string` | `null` | no |
| [snowflake\_content\_column\_name](#input\_snowflake\_content\_column\_name) | The name of the content column. | `string` | `null` | no |
| [snowflake\_data\_loading\_option](#input\_snowflake\_data\_loading\_option) | The data loading option. | `string` | `null` | no |
| [snowflake\_database](#input\_snowflake\_database) | The Snowflake database name. | `string` | `null` | no |
| [snowflake\_key\_passphrase](#input\_snowflake\_key\_passphrase) | The Snowflake passphrase for the private key. | `string` | `null` | no |
| [snowflake\_metadata\_column\_name](#input\_snowflake\_metadata\_column\_name) | The name of the metadata column. | `string` | `null` | no |
| [snowflake\_private\_key](#input\_snowflake\_private\_key) | The Snowflake private key for authentication. | `string` | `null` | no |
| [snowflake\_private\_link\_vpce\_id](#input\_snowflake\_private\_link\_vpce\_id) | The VPCE ID for Firehose to privately connect with Snowflake. | `string` | `null` | no |
| [snowflake\_retry\_duration](#input\_snowflake\_retry\_duration) | The length of time during which Firehose retries delivery after a failure, starting from the initial request and including the first attempt. | `string` | `60` | no |
| [snowflake\_role\_configuration\_enabled](#input\_snowflake\_role\_configuration\_enabled) | Whether the Snowflake role is enabled. | `bool` | `false` | no |
| [snowflake\_role\_configuration\_role](#input\_snowflake\_role\_configuration\_role) | The Snowflake role. | `string` | `null` | no |
| [snowflake\_schema](#input\_snowflake\_schema) | The Snowflake schema name. | `string` | `null` | no |
| [snowflake\_table](#input\_snowflake\_table) | The Snowflake table name. | `string` | `null` | no |
| [snowflake\_user](#input\_snowflake\_user) | The user for authentication.. | `string` | `null` | no |
| [source\_role\_arn](#input\_source\_role\_arn) | The ARN of the role that provides access to the source. Only Supported on Kinesis and MSK Sources | `string` | `null` | no |
| [source\_use\_existing\_role](#input\_source\_use\_existing\_role) | Indicates if want use the kinesis firehose role for sources access. Only Supported on Kinesis and MSK Sources | `bool` | `true` | no |
| [splunk\_hec\_acknowledgment\_timeout](#input\_splunk\_hec\_acknowledgment\_timeout) | The amount of time, that Kinesis Firehose waits to receive an acknowledgment from Splunk after it sends it data | `number` | `600` | no |
| [splunk\_hec\_endpoint](#input\_splunk\_hec\_endpoint) | The HTTP Event Collector (HEC) endpoint to which Kinesis Firehose sends your data | `string` | `null` | no |
| [splunk\_hec\_endpoint\_type](#input\_splunk\_hec\_endpoint\_type) | The HEC endpoint type | `string` | `"Raw"` | no |
| [splunk\_hec\_token](#input\_splunk\_hec\_token) | The GUID that you obtain from your Splunk cluster when you create a new HEC endpoint | `string` | `null` | no |
| [splunk\_retry\_duration](#input\_splunk\_retry\_duration) | After an initial failure to deliver to Splunk, the total amount of time, in seconds between 0 to 7200, during which Firehose re-attempts delivery (including the first attempt) | `number` | `300` | no |
| [sse\_kms\_key\_arn](#input\_sse\_kms\_key\_arn) | Amazon Resource Name (ARN) of the encryption key | `string` | `null` | no |
| [sse\_kms\_key\_type](#input\_sse\_kms\_key\_type) | Type of encryption key. | `string` | `"AWS_OWNED_CMK"` | no |
| [sumologic\_data\_type](#input\_sumologic\_data\_type) | Data Type to use in Sumo Logic destination | `string` | `"log"` | no |
| [sumologic\_deployment\_name](#input\_sumologic\_deployment\_name) | Deployment Name to use in Sumo Logic destination | `string` | `null` | no |
| [tags](#input\_tags) | A map of tags to assign to resources. | `map(string)` | `{}` | no |
| [transform\_lambda\_arn](#input\_transform\_lambda\_arn) | Lambda ARN to Transform source records | `string` | `null` | no |
| [transform\_lambda\_buffer\_interval](#input\_transform\_lambda\_buffer\_interval) | The period of time during which Kinesis Data Firehose buffers incoming data before invoking the AWS Lambda function. The AWS Lambda function is invoked once the value of the buffer size or the buffer interval is reached. | `number` | `null` | no |
| [transform\_lambda\_buffer\_size](#input\_transform\_lambda\_buffer\_size) | The AWS Lambda function has a 6 MB invocation payload quota. Your data can expand in size after it's processed by the AWS Lambda function. A smaller buffer size allows for more room should the data expand after processing. | `number` | `null` | no |
| [transform\_lambda\_number\_retries](#input\_transform\_lambda\_number\_retries) | Number of retries for AWS Transformation lambda | `number` | `null` | no |
| [transform\_lambda\_role\_arn](#input\_transform\_lambda\_role\_arn) | The ARN of the role to execute the transform lambda. If null use the Firehose Stream role | `string` | `null` | no |
| [vpc\_create\_destination\_security\_group](#input\_vpc\_create\_destination\_security\_group) | Indicates if want create destination security group to associate to firehose destinations | `bool` | `false` | no |
| [vpc\_create\_security\_group](#input\_vpc\_create\_security\_group) | Indicates if want create security group to associate to kinesis firehose | `bool` | `false` | no |
| [vpc\_role\_arn](#input\_vpc\_role\_arn) | The ARN of the IAM role to be assumed by Firehose for calling the Amazon EC2 configuration API and for creating network interfaces. Supports Elasticsearch and Opensearch destinations. | `string` | `null` | no |
| [vpc\_security\_group\_destination\_configure\_existing](#input\_vpc\_security\_group\_destination\_configure\_existing) | Indicates if want configure an existing destination security group with the necessary rules | `bool` | `false` | no |
| [vpc\_security\_group\_destination\_ids](#input\_vpc\_security\_group\_destination\_ids) | A list of security group IDs associated to destinations to allow firehose traffic | `list(string)` | `null` | no |
| [vpc\_security\_group\_destination\_vpc\_id](#input\_vpc\_security\_group\_destination\_vpc\_id) | VPC ID to create the destination security group. Only supported to Redshift and splunk destinations | `string` | `null` | no |
| [vpc\_security\_group\_firehose\_configure\_existing](#input\_vpc\_security\_group\_firehose\_configure\_existing) | Indicates if want configure an existing firehose security group with the necessary rules | `bool` | `false` | no |
| [vpc\_security\_group\_firehose\_ids](#input\_vpc\_security\_group\_firehose\_ids) | A list of security group IDs to associate with Kinesis Firehose. | `list(string)` | `null` | no |
| [vpc\_security\_group\_same\_as\_destination](#input\_vpc\_security\_group\_same\_as\_destination) | Indicates if the firehose security group is the same as destination. | `bool` | `true` | no |
| [vpc\_security\_group\_tags](#input\_vpc\_security\_group\_tags) | A map of tags to assign to security group | `map(string)` | `{}` | no |
| [vpc\_subnet\_ids](#input\_vpc\_subnet\_ids) | A list of subnet IDs to associate with Kinesis Firehose. Supports Elasticsearch and Opensearch destinations. | `list(string)` | `null` | no |
| [vpc\_use\_existing\_role](#input\_vpc\_use\_existing\_role) | Indicates if want use the kinesis firehose role to VPC access. Supports Elasticsearch and Opensearch destinations. | `bool` | `true` | no |
## Outputs
| Name | Description |
|------|-------------|
| [application\_role\_arn](#output\_application\_role\_arn) | The ARN of the IAM role created for Kinesis Firehose Stream Source |
| [application\_role\_name](#output\_application\_role\_name) | The Name of the IAM role created for Kinesis Firehose Stream Source Source |
| [application\_role\_policy\_arn](#output\_application\_role\_policy\_arn) | The ARN of the IAM policy created for Kinesis Firehose Stream Source |
| [application\_role\_policy\_name](#output\_application\_role\_policy\_name) | The Name of the IAM policy created for Kinesis Firehose Stream Source Source |
| [destination\_security\_group\_id](#output\_destination\_security\_group\_id) | Security Group ID associated to destination |
| [destination\_security\_group\_name](#output\_destination\_security\_group\_name) | Security Group Name associated to destination |
| [destination\_security\_group\_rule\_ids](#output\_destination\_security\_group\_rule\_ids) | Security Group Rules ID created in Destination Security group |
| [elasticsearch\_cross\_account\_service\_policy](#output\_elasticsearch\_cross\_account\_service\_policy) | Elasticsearch Service policy when the opensearch domain belongs to another account |
| [firehose\_cidr\_blocks](#output\_firehose\_cidr\_blocks) | Firehose stream cidr blocks to unblock on destination security group |
| [firehose\_security\_group\_id](#output\_firehose\_security\_group\_id) | Security Group ID associated to Firehose Stream. Only Supported for elasticsearch destination |
| [firehose\_security\_group\_name](#output\_firehose\_security\_group\_name) | Security Group Name associated to Firehose Stream. Only Supported for elasticsearch destination |
| [firehose\_security\_group\_rule\_ids](#output\_firehose\_security\_group\_rule\_ids) | Security Group Rules ID created in Firehose Stream Security group. Only Supported for elasticsearch destination |
| [kinesis\_firehose\_arn](#output\_kinesis\_firehose\_arn) | The ARN of the Kinesis Firehose Stream |
| [kinesis\_firehose\_cloudwatch\_log\_backup\_stream\_arn](#output\_kinesis\_firehose\_cloudwatch\_log\_backup\_stream\_arn) | The ARN of the created Cloudwatch Log Group Stream to backup |
| [kinesis\_firehose\_cloudwatch\_log\_backup\_stream\_name](#output\_kinesis\_firehose\_cloudwatch\_log\_backup\_stream\_name) | The name of the created Cloudwatch Log Group Stream to backup |
| [kinesis\_firehose\_cloudwatch\_log\_delivery\_stream\_arn](#output\_kinesis\_firehose\_cloudwatch\_log\_delivery\_stream\_arn) | The ARN of the created Cloudwatch Log Group Stream to delivery |
| [kinesis\_firehose\_cloudwatch\_log\_delivery\_stream\_name](#output\_kinesis\_firehose\_cloudwatch\_log\_delivery\_stream\_name) | The name of the created Cloudwatch Log Group Stream to delivery |
| [kinesis\_firehose\_cloudwatch\_log\_group\_arn](#output\_kinesis\_firehose\_cloudwatch\_log\_group\_arn) | The ARN of the created Cloudwatch Log Group |
| [kinesis\_firehose\_cloudwatch\_log\_group\_name](#output\_kinesis\_firehose\_cloudwatch\_log\_group\_name) | The name of the created Cloudwatch Log Group |
| [kinesis\_firehose\_destination\_id](#output\_kinesis\_firehose\_destination\_id) | The Destination id of the Kinesis Firehose Stream |
| [kinesis\_firehose\_name](#output\_kinesis\_firehose\_name) | The name of the Kinesis Firehose Stream |
| [kinesis\_firehose\_role\_arn](#output\_kinesis\_firehose\_role\_arn) | The ARN of the IAM role created for Kinesis Firehose Stream |
| [kinesis\_firehose\_version\_id](#output\_kinesis\_firehose\_version\_id) | The Version id of the Kinesis Firehose Stream |
| [opensearch\_cross\_account\_service\_policy](#output\_opensearch\_cross\_account\_service\_policy) | Opensearch Service policy when the opensearch domain belongs to another account |
| [opensearch\_iam\_service\_linked\_role\_arn](#output\_opensearch\_iam\_service\_linked\_role\_arn) | The ARN of the Opensearch IAM Service linked role |
| [opensearchserverless\_cross\_account\_service\_policy](#output\_opensearchserverless\_cross\_account\_service\_policy) | Opensearch Serverless Service policy when the opensearch domain belongs to another account |
| [opensearchserverless\_iam\_service\_linked\_role\_arn](#output\_opensearchserverless\_iam\_service\_linked\_role\_arn) | The ARN of the Opensearch Serverless IAM Service linked role |
| [s3\_cross\_account\_bucket\_policy](#output\_s3\_cross\_account\_bucket\_policy) | Bucket Policy to S3 Bucket Destination when the bucket belongs to another account |
## Upgrade
- Version 1.x to 2.x Upgrade Guide [here](https://github.com/fdmsantos/terraform-aws-kinesis-firehose/blob/main/UPGRADE-2.0.md)
- Version 2.x to 3.x Upgrade Guide [here](https://github.com/fdmsantos/terraform-aws-kinesis-firehose/blob/main/UPGRADE-3.0.md)
## Deprecations
### Version 3.1.0
* Variable `kinesis_source_role_arn` is deprecated. Use `source_role_arn` instead.
* Variable `kinesis_source_use_existing_role` is deprecated. Use `source_use_existing_role` instead.
### Version 3.3.0
* Variable `dynamic_partition_append_delimiter_to_record` is deprecated. Use `append_delimiter_to_record` instead.
## License
Apache 2 Licensed. See [LICENSE](https://github.com/fdmsantos/terraform-aws-kinesis-firehose/tree/main/LICENSE) for full details.