Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/idealo/terraform-aws-opensearch

Terraform module to provision an OpenSearch cluster with SAML authentication.
https://github.com/idealo/terraform-aws-opensearch
aws opensearch terraform terraform-module
Last synced: 3 days ago
JSON representation
Terraform module to provision an OpenSearch cluster with SAML authentication.
Host: GitHub
URL: https://github.com/idealo/terraform-aws-opensearch
Owner: idealo
License: apache-2.0
Created: 2021-12-03T07:35:30.000Z (about 3 years ago)
Default Branch: main
Last Pushed: 2024-11-13T13:25:19.000Z (about 1 month ago)
Last Synced: 2024-12-15T03:06:42.398Z (10 days ago)
Topics: aws, opensearch, terraform, terraform-module
Language: HCL
Homepage:
Size: 88.9 KB
Stars: 67
Watchers: 15
Forks: 57
Open Issues: 2
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project

README

        # AWS OpenSearch Terraform Module

Terraform module to provision an OpenSearch cluster with SAML authentication.

## Prerequisites

- A [hosted zone](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/CreatingHostedZone.html) to route traffic to your OpenSearch domain

- An [entityID and metadata XML](https://aws.amazon.com/de/blogs/security/configure-saml-single-sign-on-for-kibana-with-ad-fs-on-amazon-elasticsearch-service/) from your SAML identity provider (in case `saml_enabled = true`)

## Features

- Create an AWS OpenSearch cluster with SAML authentication

- All node types with local NVMe for high IO performance are supported

- Create or manage various OpenSearch resources:

  - [Index templates](https://opensearch.org/docs/latest/opensearch/index-templates/)

  - [Indices](https://opensearch.org/docs/latest/opensearch/rest-api/index-apis/create-index/)

  - [ISM policies](https://opensearch.org/docs/latest/im-plugin/ism/policies/)

  - [Roles](https://opensearch.org/docs/latest/security-plugin/access-control/users-roles/#create-roles)

  - [Role mappings](https://opensearch.org/docs/latest/security-plugin/access-control/users-roles/#map-users-to-roles)

## Usage

This example is using Azure AD as SAML identity provider.

```terraform

locals {

  cluster_name      = "opensearch"

  cluster_domain    = "example.com"

  saml_entity_id    = "https://sts.windows.net/XXX-XXX-XXX-XXX-XXX/"

  saml_metadata_url = "https://login.microsoftonline.com/XXX-XXX-XXX-XXX-XXX/federationmetadata/2007-06/federationmetadata.xml?appid=YYY-YYY-YYY-YYY-YYY"

}

data "aws_region" "current" {}

data "http" "saml_metadata" {

  url = local.saml_metadata_url

}

provider "opensearch" {

  url                   = module.opensearch.cluster_endpoint

  aws_region            = data.aws_region.current.name

  healthcheck           = false

}

module "opensearch" {

  source  = "idealo/opensearch/aws"

  version = "~> 2.0"

  cluster_name    = local.cluster_name

  cluster_domain  = local.cluster_domain

  cluster_version = "2.11"

  saml_entity_id        = local.saml_entity_id

  saml_metadata_content = data.http.saml_metadata.body

  indices = {

    example-index-1 = {

      number_of_shards   = 2

      number_of_replicas = 1

    }

    example-index-2 = {

      number_of_shards   = 2

      number_of_replicas = 1

      mappings = {

        "properties" : {

          "id" : {

            "type" : "text"

          },

          "name" : {

            "type" : "text"

          },

          "containerType" : {

            "type" : "text"

          },

          "containerIds" : {

            "type" : "text"

          },

          "synonyms" : {

            "type" : "text"

          },

          "parentEvents" : {

            "type" : "text"

          },

          "valueType" : {

            "type" : "text"

          }

        }

      }

    }

  }

}

```

## Examples

Here is a working example of using this Terraform module:

- [Complete](https://github.com/idealo/terraform-aws-opensearch/tree/main/examples/complete) - Create an AWS OpenSearch cluster with all necessary resources.

- [Minimal](https://github.com/idealo/terraform-aws-opensearch/tree/main/examples/minimal) - Create an empty AWS OpenSearch cluster without saml.

## Requirements

| Name | Version |

|------|---------|

|  [terraform](#requirement\_terraform) | >= 1.3.0 |

|  [aws](#requirement\_aws) | >= 4.12.0 |

|  [opensearch](#requirement\_opensearch) | >= 2.0.0 |

## Modules

| Name | Source | Version |

|------|--------|---------|

|  [acm](#module\_acm) | terraform-aws-modules/acm/aws | ~> 5.0.0 |

## Resources

| Name | Type |

|------|------|

| [aws_cloudwatch_log_group.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_group) | resource |

| [aws_cloudwatch_log_resource_policy.allow_logging](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/cloudwatch_log_resource_policy) | resource |

| [aws_elasticsearch_domain.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain) | resource |

| [aws_elasticsearch_domain_saml_options.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/elasticsearch_domain_saml_options) | resource |

| [aws_iam_service_linked_role.es](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/iam_service_linked_role) | resource |

| [aws_route53_record.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/route53_record) | resource |

| [opensearch_composable_index_template.composable_index_template](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/composable_index_template) | resource |

| [opensearch_index.index](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/index) | resource |

| [opensearch_index_template.index_template](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/index_template) | resource |

| [opensearch_ism_policy.ism_policy](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/ism_policy) | resource |

| [opensearch_role.role](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/role) | resource |

| [opensearch_roles_mapping.master_user_arn](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/roles_mapping) | resource |

| [opensearch_roles_mapping.master_user_name](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/roles_mapping) | resource |

| [opensearch_roles_mapping.role_mapping](https://registry.terraform.io/providers/opensearch-project/opensearch/latest/docs/resources/roles_mapping) | resource |

| [aws_caller_identity.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/caller_identity) | data source |

| [aws_iam_policy_document.access_policy](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |

| [aws_iam_policy_document.allow_logging](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/iam_policy_document) | data source |

| [aws_region.current](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/region) | data source |

| [aws_route53_zone.opensearch](https://registry.terraform.io/providers/hashicorp/aws/latest/docs/data-sources/route53_zone) | data source |

## Inputs

| Name | Description | Type | Default | Required |

|------|-------------|------|---------|:--------:|

|  [access\_policies](#input\_access\_policies) | IAM policy document specifying the access policies for the domain. | `string` | `null` | no |

|  [advanced\_options](#input\_advanced\_options) | Key-value string pairs to specify advanced configuration options. | `map(string)` | `null` | no |

|  [advanced\_security\_options\_enabled](#input\_advanced\_security\_options\_enabled) | Whether advanced security is enabled. | `bool` | `true` | no |

|  [advanced\_security\_options\_internal\_user\_database\_enabled](#input\_advanced\_security\_options\_internal\_user\_database\_enabled) | Whether to enable or not internal Kibana user database for ELK OpenDistro security plugin | `bool` | `false` | no |

|  [advanced\_security\_options\_master\_user\_name](#input\_advanced\_security\_options\_master\_user\_name) | Master user username (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `null` | no |

|  [advanced\_security\_options\_master\_user\_password](#input\_advanced\_security\_options\_master\_user\_password) | Master user password (applicable if advanced\_security\_options\_internal\_user\_database\_enabled set to true) | `string` | `null` | no |

|  [auto\_tune\_enabled](#input\_auto\_tune\_enabled) | Whether to enable/disable auto-tune | `bool` | `true` | no |

|  [auto\_tune\_options](#input\_auto\_tune\_options) | Configuration block for auto-tune options. The maintenance schedule block is required if rollback\_on\_disable is set to DEFAULT\_ROLLBACK. The start\_at field must be a time and date in RFC3339 format | 
object({
    maintenance_schedule = optional(list(object({
      start_at = string
      duration = object({
        value = number
      })
      cron_expression_for_recurrence = optional(string)
    })))
    rollback_on_disable = string
  })
 | {
  "maintenance_schedule": [],
  "rollback_on_disable": "NO_ROLLBACK"
} | no |

|  [availability\_zones](#input\_availability\_zones) | The number of availability zones for the OpenSearch cluster. Valid values: 1, 2 or 3. | `number` | `3` | no |

|  [cluster\_domain](#input\_cluster\_domain) | The hosted zone name of the OpenSearch cluster. | `string` | n/a | yes |

|  [cluster\_domain\_private](#input\_cluster\_domain\_private) | Indicates whether to create records in a private (true) or public (false) zone | `bool` | `false` | no |

|  [cluster\_name](#input\_cluster\_name) | The name of the OpenSearch cluster. | `string` | `"opensearch"` | no |

|  [cluster\_version](#input\_cluster\_version) | The version of OpenSearch to deploy. | `string` | `"2.11"` | no |

|  [cognito\_options](#input\_cognito\_options) | Configuration block for authenticating Kibana with Cognito. | `map(string)` | `{}` | no |

|  [cognito\_options\_enabled](#input\_cognito\_options\_enabled) | Whether Amazon Cognito authentication with Kibana is enabled or not. | `bool` | `false` | no |

|  [composable\_index\_template\_files](#input\_composable\_index\_template\_files) | A set of all composable index template files to create. | `set(string)` | `[]` | no |

|  [composable\_index\_templates](#input\_composable\_index\_templates) | A map of all composable index templates to create. | `map(any)` | `{}` | no |

|  [create\_service\_role](#input\_create\_service\_role) | Indicates whether to create the service-linked role. See https://docs.aws.amazon.com/opensearch-service/latest/developerguide/slr.html | `bool` | `true` | no |

|  [custom\_endpoint](#input\_custom\_endpoint) | Fully qualified domain for your custom endpoint. If not specified, then it defaults to . | `string` | `null` | no |

|  [custom\_endpoint\_certificate\_arn](#input\_custom\_endpoint\_certificate\_arn) | The ARN of the custom ACM certificate. | `string` | `""` | no |

|  [ebs\_enabled](#input\_ebs\_enabled) | Indicates whether attach EBS volumes to the data nodes. | `bool` | `false` | no |

|  [ebs\_iops](#input\_ebs\_iops) | The baseline input/output (I/O) performance of EBS volumes attached to data nodes. | `number` | `3000` | no |

|  [ebs\_throughput](#input\_ebs\_throughput) | The throughput (in MiB/s) of the EBS volumes attached to data nodes. Valid values are between 125 and 1000. | `number` | `125` | no |

|  [ebs\_volume\_size](#input\_ebs\_volume\_size) | The size of EBS volumes attached to data nodes (in GiB). | `number` | `10` | no |

|  [ebs\_volume\_type](#input\_ebs\_volume\_type) | The type of EBS volumes attached to data nodes. | `string` | `"gp3"` | no |

|  [encrypt\_at\_rest\_enabled](#input\_encrypt\_at\_rest\_enabled) | Configuration block for encrypt at rest options | `bool` | `true` | no |

|  [encrypt\_kms\_key\_id](#input\_encrypt\_kms\_key\_id) | The KMS key ID to encrypt the OpenSearch cluster with. If not specified, then it defaults to using the AWS OpenSearch Service KMS key. | `string` | `""` | no |

|  [hot\_instance\_count](#input\_hot\_instance\_count) | The number of dedicated hot nodes in the cluster. | `number` | `3` | no |

|  [hot\_instance\_type](#input\_hot\_instance\_type) | The type of EC2 instances to run for each hot node. A list of available instance types can you find at https://aws.amazon.com/en/opensearch-service/pricing/#On-Demand_instance_pricing | `string` | `"r6gd.4xlarge.elasticsearch"` | no |

|  [index\_files](#input\_index\_files) | A set of all index files to create. | `set(string)` | `[]` | no |

|  [index\_template\_files](#input\_index\_template\_files) | A set of all index template files to create. | `set(string)` | `[]` | no |

|  [index\_templates](#input\_index\_templates) | A map of all index templates to create. | `map(any)` | `{}` | no |

|  [indices](#input\_indices) | A map of all indices to create. | map(object({
    number_of_shards                       = optional(number)
    number_of_replicas                     = optional(number)
    refresh_interval                       = optional(string)
    mappings                               = optional(any, {})
    aliases                                = optional(any, {})
    analysis_analyzer                      = optional(string)
    analysis_char_filter                   = optional(string)
    analysis_filter                        = optional(string)
    analysis_normalizer                    = optional(string)
    analysis_tokenizer                     = optional(string)
    analyze_max_token_count                = optional(string)
    auto_expand_replicas                   = optional(string)
    blocks_metadata                        = optional(bool)
    blocks_read                            = optional(bool)
    blocks_read_only                       = optional(bool)
    blocks_read_only_allow_delete          = optional(bool)
    blocks_write                           = optional(bool)
    codec                                  = optional(string)
    default_pipeline                       = optional(string)
    gc_deletes                             = optional(string)
    highlight_max_analyzed_offset          = optional(string)
    include_type_name                      = optional(string)
    index_similarity_default               = optional(string)
    indexing_slowlog_level                 = optional(string)
    indexing_slowlog_source                = optional(string)
    indexing_slowlog_threshold_index_debug = optional(string)
    indexing_slowlog_threshold_index_info  = optional(string)
    indexing_slowlog_threshold_index_trace = optional(string)
    indexing_slowlog_threshold_index_warn  = optional(string)
    load_fixed_bitset_filters_eagerly      = optional(bool)
    max_docvalue_fields_search             = optional(string)
    max_inner_result_window                = optional(string)
    max_ngram_diff                         = optional(string)
    max_refresh_listeners                  = optional(string)
    max_regex_length                       = optional(string)
    max_rescore_window                     = optional(string)
    max_result_window                      = optional(string)
    max_script_fields                      = optional(string)
    max_shingle_diff                       = optional(string)
    max_terms_count                        = optional(string)
    number_of_routing_shards               = optional(string)
    rollover_alias                         = optional(string)
    routing_allocation_enable              = optional(string)
    routing_partition_size                 = optional(string)
    routing_rebalance_enable               = optional(string)
    search_idle_after                      = optional(string)
    search_slowlog_level                   = optional(string)
    search_slowlog_threshold_fetch_info    = optional(string)
    search_slowlog_threshold_fetch_debug   = optional(string)
    search_slowlog_threshold_fetch_trace   = optional(string)
    search_slowlog_threshold_fetch_warn    = optional(string)
    search_slowlog_threshold_query_debug   = optional(string)
    search_slowlog_threshold_query_info    = optional(string)
    search_slowlog_threshold_query_trace   = optional(string)
    search_slowlog_threshold_query_warn    = optional(string)
    shard_check_on_startup                 = optional(string)
    sort_field                             = optional(string)
    sort_order                             = optional(string)
  })) | `{}` | no |

|  [ism\_policies](#input\_ism\_policies) | A map of all ISM policies to create. | `map(any)` | `{}` | no |

|  [ism\_policy\_files](#input\_ism\_policy\_files) | A set of all ISM policy files to create. | `set(string)` | `[]` | no |

|  [log\_streams\_enabled](#input\_log\_streams\_enabled) | Configuration for which log streams to enable sending logs to CloudWatch. | `map(string)` | {
  "AUDIT_LOGS": "false",
  "ES_APPLICATION_LOGS": "false",
  "INDEX_SLOW_LOGS": "false",
  "SEARCH_SLOW_LOGS": "false"
} | no |

|  [master\_instance\_count](#input\_master\_instance\_count) | The number of dedicated master nodes in the cluster. | `number` | `3` | no |

|  [master\_instance\_enabled](#input\_master\_instance\_enabled) | Indicates whether dedicated master nodes are enabled for the cluster. | `bool` | `true` | no |

|  [master\_instance\_type](#input\_master\_instance\_type) | The type of EC2 instances to run for each master node. A list of available instance types can you find at https://aws.amazon.com/en/opensearch-service/pricing/#On-Demand_instance_pricing | `string` | `"r6gd.large.elasticsearch"` | no |

|  [master\_user\_arn](#input\_master\_user\_arn) | The ARN for the master user of the cluster. If not specified, then it defaults to using the IAM user that is making the request. | `string` | `""` | no |

|  [node\_to\_node\_encryption\_enabled](#input\_node\_to\_node\_encryption\_enabled) | Configuration block for node-to-node encryption options | `bool` | `true` | no |

|  [role\_files](#input\_role\_files) | A set of all role files to create. | `set(string)` | `[]` | no |

|  [role\_mapping\_files](#input\_role\_mapping\_files) | A set of all role mapping files to create. | `set(string)` | `[]` | no |

|  [role\_mappings](#input\_role\_mappings) | A map of all role mappings to create. | `map(any)` | `{}` | no |

|  [roles](#input\_roles) | A map of all roles to create. | `map(any)` | `{}` | no |

|  [saml\_enabled](#input\_saml\_enabled) | Indicates whether to configure SAML for the OpenSearch dashboard. | `bool` | `true` | no |

|  [saml\_entity\_id](#input\_saml\_entity\_id) | The unique Entity ID of the application in SAML Identity Provider. | `string` | `""` | no |

|  [saml\_master\_backend\_role](#input\_saml\_master\_backend\_role) | This backend role receives full permissions to the cluster, equivalent to a new master role, but can only use those permissions within Dashboards. | `string` | `null` | no |

|  [saml\_master\_user\_name](#input\_saml\_master\_user\_name) | This username receives full permissions to the cluster, equivalent to a new master user, but can only use those permissions within Dashboards. | `string` | `null` | no |

|  [saml\_metadata\_content](#input\_saml\_metadata\_content) | The metadata of the SAML application in xml format. | `string` | `""` | no |

|  [saml\_roles\_key](#input\_saml\_roles\_key) | Element of the SAML assertion to use for backend roles. | `string` | `"http://schemas.microsoft.com/ws/2008/06/identity/claims/role"` | no |

|  [saml\_session\_timeout](#input\_saml\_session\_timeout) | Duration of a session in minutes after a user logs in. Default is 60. Maximum value is 1,440. | `number` | `60` | no |

|  [saml\_subject\_key](#input\_saml\_subject\_key) | Element of the SAML assertion to use for username. | `string` | `"http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name"` | no |

|  [security\_group\_ids](#input\_security\_group\_ids) | The list of VPC security groups IDs to attach. | `list(string)` | `[]` | no |

|  [subnet\_ids](#input\_subnet\_ids) | The list of VPC subnet IDs to use. | `list(string)` | `[]` | no |

|  [tags](#input\_tags) | A map of tags to add to all resources. | `map(string)` | `{}` | no |

|  [vpc\_enabled](#input\_vpc\_enabled) | Indicates whether the cluster is running inside a VPC. | `bool` | `false` | no |

|  [warm\_instance\_count](#input\_warm\_instance\_count) | The number of dedicated warm nodes in the cluster. | `number` | `3` | no |

|  [warm\_instance\_enabled](#input\_warm\_instance\_enabled) | Indicates whether ultrawarm nodes are enabled for the cluster. | `bool` | `true` | no |

|  [warm\_instance\_type](#input\_warm\_instance\_type) | The type of EC2 instances to run for each warm node. A list of available instance types can you find at https://aws.amazon.com/en/elasticsearch-service/pricing/#UltraWarm_pricing | `string` | `"ultrawarm1.large.elasticsearch"` | no |

## Outputs

| Name | Description |

|------|-------------|

|  [cluster\_endpoint](#output\_cluster\_endpoint) | The endpoint URL of the OpenSearch cluster. |

|  [cluster\_name](#output\_cluster\_name) | The name of the OpenSearch cluster. |

|  [cluster\_version](#output\_cluster\_version) | The version of the OpenSearch cluster. |

|  [kibana\_endpoint](#output\_kibana\_endpoint) | The endpoint URL of the OpenSearch dashboards. |

## License

Apache 2 Licensed. See [LICENSE](https://github.com/idealo/terraform-aws-opensearch/blob/main/LICENSE) for full details.