{"id":24270685,"url":"https://github.com/materializeinc/terraform-google-materialize","last_synced_at":"2026-05-20T20:12:44.765Z","repository":{"id":272344147,"uuid":"879764500","full_name":"MaterializeInc/terraform-google-materialize","owner":"MaterializeInc","description":"Terraform module for deploying Materialize on GCP with all required infrastructure components.","archived":false,"fork":false,"pushed_at":"2025-02-27T18:33:40.000Z","size":103,"stargazers_count":0,"open_issues_count":3,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-03-01T11:09:22.519Z","etag":null,"topics":["gcp","materialize","terraform"],"latest_commit_sha":null,"homepage":"https://materialize.com/docs/self-managed/v25.1/","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MaterializeInc.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-10-28T14:06:41.000Z","updated_at":"2025-02-27T18:32:55.000Z","dependencies_parsed_at":"2025-01-13T20:37:44.579Z","dependency_job_id":"41348b5a-1c7f-45d3-9a1b-f6b5db08468b","html_url":"https://github.com/MaterializeInc/terraform-google-materialize","commit_stats":null,"previous_names":["materializeinc/terraform-google-materialize"],"tags_count":6,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-google-materialize","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-google-materialize/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-google-materialize/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MaterializeInc%2Fterraform-google-materialize/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MaterializeInc","download_url":"https://codeload.github.com/MaterializeInc/terraform-google-materialize/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":241969786,"owners_count":20050576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gcp","materialize","terraform"],"created_at":"2025-01-15T16:54:29.124Z","updated_at":"2026-05-20T20:12:44.753Z","avatar_url":"https://github.com/MaterializeInc.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!-- BEGIN_TF_DOCS --\u003e\n# Materialize on Google Cloud Platform\n\nTerraform module for deploying Materialize on Google Cloud Platform (GCP) with all required infrastructure components.\n\nThis module sets up:\n- GKE cluster for Materialize workloads\n- Cloud SQL PostgreSQL instance for metadata storage\n- Cloud Storage bucket for persistence\n- Required networking and security configurations\n- Service accounts with proper IAM permissions\n\n\u003e [!WARNING]\n\u003e This module is intended for demonstration/evaluation purposes as well as for serving as a template when building your own production deployment of Materialize.\n\u003e\n\u003e This module should not be directly relied upon for production deployments: **future releases of the module will contain breaking changes.** Instead, to use as a starting point for your own production deployment, either:\n\u003e - Fork this repo and pin to a specific version, or\n\u003e - Use the code as a reference when developing your own deployment.\n\nThe module has been tested with:\n- GKE version 1.28\n- PostgreSQL 15\n- terraform-helm-materialize v0.1.12 (Materialize Operator v25.1.7)\n\n## `materialize_instances` variable\n\nThe `materialize_instances` variable is a list of objects that define the configuration for each Materialize instance.\n\n### `environmentd_extra_args`\n\nOptional list of additional command-line arguments to pass to the `environmentd` container. This can be used to override default system parameters or enable specific features.\n\n```hcl\nenvironmentd_extra_args = [\n  \"--system-parameter-default=max_clusters=1000\",\n  \"--system-parameter-default=max_connections=1000\",\n  \"--system-parameter-default=max_tables=1000\",\n]\n```\n\nThese flags configure default limits for clusters, connections, and tables. You can provide any supported arguments [here](https://materialize.com/docs/sql/alter-system-set/#other-configuration-parameters).\n\n## Requirements\n\n| Name | Version |\n|------|---------|\n| \u003ca name=\"requirement_terraform\"\u003e\u003c/a\u003e [terraform](#requirement\\_terraform) | \u003e= 1.0 |\n| \u003ca name=\"requirement_deepmerge\"\u003e\u003c/a\u003e [deepmerge](#requirement\\_deepmerge) | ~\u003e 1.0 |\n| \u003ca name=\"requirement_google\"\u003e\u003c/a\u003e [google](#requirement\\_google) | \u003e= 6.0 |\n| \u003ca name=\"requirement_helm\"\u003e\u003c/a\u003e [helm](#requirement\\_helm) | ~\u003e 2.0 |\n| \u003ca name=\"requirement_kubernetes\"\u003e\u003c/a\u003e [kubernetes](#requirement\\_kubernetes) | ~\u003e 2.0 |\n\n## Providers\n\nNo providers.\n\n## Modules\n\n| Name | Source | Version |\n|------|--------|---------|\n| \u003ca name=\"module_certificates\"\u003e\u003c/a\u003e [certificates](#module\\_certificates) | ./modules/certificates | n/a |\n| \u003ca name=\"module_database\"\u003e\u003c/a\u003e [database](#module\\_database) | ./modules/database | n/a |\n| \u003ca name=\"module_gke\"\u003e\u003c/a\u003e [gke](#module\\_gke) | ./modules/gke | n/a |\n| \u003ca name=\"module_load_balancers\"\u003e\u003c/a\u003e [load\\_balancers](#module\\_load\\_balancers) | ./modules/load_balancers | n/a |\n| \u003ca name=\"module_materialize_nodepool\"\u003e\u003c/a\u003e [materialize\\_nodepool](#module\\_materialize\\_nodepool) | ./modules/nodepool | n/a |\n| \u003ca name=\"module_networking\"\u003e\u003c/a\u003e [networking](#module\\_networking) | ./modules/networking | n/a |\n| \u003ca name=\"module_operator\"\u003e\u003c/a\u003e [operator](#module\\_operator) | github.com/MaterializeInc/terraform-helm-materialize | v0.1.66 |\n| \u003ca name=\"module_storage\"\u003e\u003c/a\u003e [storage](#module\\_storage) | ./modules/storage | n/a |\n\n## Resources\n\nNo resources.\n\n## Inputs\n\n| Name | Description | Type | Default | Required |\n|------|-------------|------|---------|:--------:|\n| \u003ca name=\"input_cert_manager_chart_version\"\u003e\u003c/a\u003e [cert\\_manager\\_chart\\_version](#input\\_cert\\_manager\\_chart\\_version) | Version of the cert-manager helm chart to install. | `string` | `\"v1.17.1\"` | no |\n| \u003ca name=\"input_cert_manager_install_timeout\"\u003e\u003c/a\u003e [cert\\_manager\\_install\\_timeout](#input\\_cert\\_manager\\_install\\_timeout) | Timeout for installing the cert-manager helm chart, in seconds. | `number` | `300` | no |\n| \u003ca name=\"input_cert_manager_namespace\"\u003e\u003c/a\u003e [cert\\_manager\\_namespace](#input\\_cert\\_manager\\_namespace) | The name of the namespace in which cert-manager is or will be installed. | `string` | `\"cert-manager\"` | no |\n| \u003ca name=\"input_database_config\"\u003e\u003c/a\u003e [database\\_config](#input\\_database\\_config) | Cloud SQL configuration | \u003cpre\u003eobject({\u003cbr/\u003e    tier     = optional(string, \"db-custom-2-4096\")\u003cbr/\u003e    version  = optional(string, \"POSTGRES_15\")\u003cbr/\u003e    password = string\u003cbr/\u003e    username = optional(string, \"materialize\")\u003cbr/\u003e    db_name  = optional(string, \"materialize\")\u003cbr/\u003e  })\u003c/pre\u003e | n/a | yes |\n| \u003ca name=\"input_database_vpc_wait_duration\"\u003e\u003c/a\u003e [database\\_vpc\\_wait\\_duration](#input\\_database\\_vpc\\_wait\\_duration) | Duration to wait for VPC resources to be ready before creating the database (e.g., '60s', '2m') | `string` | `\"60s\"` | no |\n| \u003ca name=\"input_helm_chart\"\u003e\u003c/a\u003e [helm\\_chart](#input\\_helm\\_chart) | Chart name from repository or local path to chart. For local charts, set the path to the chart directory. | `string` | `\"materialize-operator\"` | no |\n| \u003ca name=\"input_helm_values\"\u003e\u003c/a\u003e [helm\\_values](#input\\_helm\\_values) | Values to pass to the Helm chart | `any` | `{}` | no |\n| \u003ca name=\"input_install_cert_manager\"\u003e\u003c/a\u003e [install\\_cert\\_manager](#input\\_install\\_cert\\_manager) | Whether to install cert-manager. | `bool` | `true` | no |\n| \u003ca name=\"input_install_materialize_operator\"\u003e\u003c/a\u003e [install\\_materialize\\_operator](#input\\_install\\_materialize\\_operator) | Whether to install the Materialize operator | `bool` | `true` | no |\n| \u003ca name=\"input_install_metrics_server\"\u003e\u003c/a\u003e [install\\_metrics\\_server](#input\\_install\\_metrics\\_server) | Whether to install the metrics-server for the Materialize Console. Defaults to false since GKE installs one by default in the kube-system namespace. Only set to true if the GKE cluster was deployed with [monitoring explicitly turned off](https://cloud.google.com/kubernetes-engine/docs/how-to/configure-metrics#:~:text=To%20disable%20system%20metric%20collection,for%20the%20%2D%2Dmonitoring%20flag). Refer to the [GKE docs](https://cloud.google.com/kubernetes-engine/docs/how-to/configure-metrics#:~:text=To%20disable%20system%20metric%20collection,for%20the%20%2D%2Dmonitoring%20flag) for more information, including impact to GKE customer support efforts. | `bool` | `false` | no |\n| \u003ca name=\"input_labels\"\u003e\u003c/a\u003e [labels](#input\\_labels) | Labels to apply to all resources | `map(string)` | `{}` | no |\n| \u003ca name=\"input_materialize_instances\"\u003e\u003c/a\u003e [materialize\\_instances](#input\\_materialize\\_instances) | Configuration for Materialize instances | \u003cpre\u003elist(object({\u003cbr/\u003e    name                              = string\u003cbr/\u003e    namespace                         = optional(string)\u003cbr/\u003e    database_name                     = string\u003cbr/\u003e    create_database                   = optional(bool, true)\u003cbr/\u003e    create_load_balancer              = optional(bool, true)\u003cbr/\u003e    internal_load_balancer            = optional(bool, true)\u003cbr/\u003e    environmentd_version              = optional(string)\u003cbr/\u003e    cpu_request                       = optional(string, \"1\")\u003cbr/\u003e    memory_request                    = optional(string, \"1Gi\")\u003cbr/\u003e    memory_limit                      = optional(string, \"1Gi\")\u003cbr/\u003e    in_place_rollout                  = optional(bool, false)\u003cbr/\u003e    request_rollout                   = optional(string)\u003cbr/\u003e    force_rollout                     = optional(string)\u003cbr/\u003e    balancer_memory_request           = optional(string, \"256Mi\")\u003cbr/\u003e    balancer_memory_limit             = optional(string, \"256Mi\")\u003cbr/\u003e    balancer_cpu_request              = optional(string, \"100m\")\u003cbr/\u003e    license_key                       = optional(string)\u003cbr/\u003e    authenticator_kind                = optional(string, \"None\")\u003cbr/\u003e    external_login_password_mz_system = optional(string)\u003cbr/\u003e    environmentd_extra_args           = optional(list(string), [])\u003cbr/\u003e  }))\u003c/pre\u003e | `[]` | no |\n| \u003ca name=\"input_materialize_node_group_disk_size_gb\"\u003e\u003c/a\u003e [materialize\\_node\\_group\\_disk\\_size\\_gb](#input\\_materialize\\_node\\_group\\_disk\\_size\\_gb) | Size of the disk attached to each Materialize worker node | `number` | `100` | no |\n| \u003ca name=\"input_materialize_node_group_local_ssd_count\"\u003e\u003c/a\u003e [materialize\\_node\\_group\\_local\\_ssd\\_count](#input\\_materialize\\_node\\_group\\_local\\_ssd\\_count) | Number of local NVMe SSDs to attach to each Materialize node. In GCP, each disk is 375GB. | `number` | `1` | no |\n| \u003ca name=\"input_materialize_node_group_machine_type\"\u003e\u003c/a\u003e [materialize\\_node\\_group\\_machine\\_type](#input\\_materialize\\_node\\_group\\_machine\\_type) | Machine type for Materialize worker nodes | `string` | `\"n2-highmem-8\"` | no |\n| \u003ca name=\"input_materialize_node_group_max_nodes\"\u003e\u003c/a\u003e [materialize\\_node\\_group\\_max\\_nodes](#input\\_materialize\\_node\\_group\\_max\\_nodes) | Maximum number of Materialize worker nodes | `number` | `2` | no |\n| \u003ca name=\"input_materialize_node_group_min_nodes\"\u003e\u003c/a\u003e [materialize\\_node\\_group\\_min\\_nodes](#input\\_materialize\\_node\\_group\\_min\\_nodes) | Minimum number of Materialize worker nodes | `number` | `1` | no |\n| \u003ca name=\"input_namespace\"\u003e\u003c/a\u003e [namespace](#input\\_namespace) | Kubernetes namespace for Materialize | `string` | `\"materialize\"` | no |\n| \u003ca name=\"input_network_config\"\u003e\u003c/a\u003e [network\\_config](#input\\_network\\_config) | Network configuration for the GKE cluster | \u003cpre\u003eobject({\u003cbr/\u003e    subnet_cidr   = string\u003cbr/\u003e    pods_cidr     = string\u003cbr/\u003e    services_cidr = string\u003cbr/\u003e  })\u003c/pre\u003e | n/a | yes |\n| \u003ca name=\"input_operator_namespace\"\u003e\u003c/a\u003e [operator\\_namespace](#input\\_operator\\_namespace) | Namespace for the Materialize operator | `string` | `\"materialize\"` | no |\n| \u003ca name=\"input_operator_version\"\u003e\u003c/a\u003e [operator\\_version](#input\\_operator\\_version) | Version of the Materialize operator to install | `string` | `null` | no |\n| \u003ca name=\"input_orchestratord_version\"\u003e\u003c/a\u003e [orchestratord\\_version](#input\\_orchestratord\\_version) | Version of the Materialize orchestrator to install | `string` | `null` | no |\n| \u003ca name=\"input_prefix\"\u003e\u003c/a\u003e [prefix](#input\\_prefix) | Prefix to be used for resource names | `string` | `\"materialize\"` | no |\n| \u003ca name=\"input_project_id\"\u003e\u003c/a\u003e [project\\_id](#input\\_project\\_id) | The ID of the project where resources will be created | `string` | n/a | yes |\n| \u003ca name=\"input_region\"\u003e\u003c/a\u003e [region](#input\\_region) | The region where resources will be created | `string` | `\"us-central1\"` | no |\n| \u003ca name=\"input_storage_bucket_version_ttl\"\u003e\u003c/a\u003e [storage\\_bucket\\_version\\_ttl](#input\\_storage\\_bucket\\_version\\_ttl) | Sets the TTL (in days) on non current storage bucket objects. This must be set if storage\\_bucket\\_versioning is turned on. | `number` | `7` | no |\n| \u003ca name=\"input_storage_bucket_versioning\"\u003e\u003c/a\u003e [storage\\_bucket\\_versioning](#input\\_storage\\_bucket\\_versioning) | Enable bucket versioning. This should be enabled for production deployments. | `bool` | `false` | no |\n| \u003ca name=\"input_system_node_group_disk_size_gb\"\u003e\u003c/a\u003e [system\\_node\\_group\\_disk\\_size\\_gb](#input\\_system\\_node\\_group\\_disk\\_size\\_gb) | Size of the disk attached to each system node | `number` | `100` | no |\n| \u003ca name=\"input_system_node_group_machine_type\"\u003e\u003c/a\u003e [system\\_node\\_group\\_machine\\_type](#input\\_system\\_node\\_group\\_machine\\_type) | Machine type for system nodes | `string` | `\"n2-highmem-8\"` | no |\n| \u003ca name=\"input_system_node_group_max_nodes\"\u003e\u003c/a\u003e [system\\_node\\_group\\_max\\_nodes](#input\\_system\\_node\\_group\\_max\\_nodes) | Maximum number of system nodes | `number` | `2` | no |\n| \u003ca name=\"input_system_node_group_min_nodes\"\u003e\u003c/a\u003e [system\\_node\\_group\\_min\\_nodes](#input\\_system\\_node\\_group\\_min\\_nodes) | Minimum number of system nodes | `number` | `1` | no |\n| \u003ca name=\"input_system_node_group_node_count\"\u003e\u003c/a\u003e [system\\_node\\_group\\_node\\_count](#input\\_system\\_node\\_group\\_node\\_count) | Number of nodes in the system node group | `number` | `1` | no |\n| \u003ca name=\"input_use_local_chart\"\u003e\u003c/a\u003e [use\\_local\\_chart](#input\\_use\\_local\\_chart) | Whether to use a local chart instead of one from a repository | `bool` | `false` | no |\n| \u003ca name=\"input_use_self_signed_cluster_issuer\"\u003e\u003c/a\u003e [use\\_self\\_signed\\_cluster\\_issuer](#input\\_use\\_self\\_signed\\_cluster\\_issuer) | Whether to install and use a self-signed ClusterIssuer for TLS. To work around limitations in Terraform, this will be treated as `false` if no materialize instances are defined. | `bool` | `true` | no |\n\n## Outputs\n\n| Name | Description |\n|------|-------------|\n| \u003ca name=\"output_connection_strings\"\u003e\u003c/a\u003e [connection\\_strings](#output\\_connection\\_strings) | Formatted connection strings for Materialize |\n| \u003ca name=\"output_database\"\u003e\u003c/a\u003e [database](#output\\_database) | Cloud SQL instance details |\n| \u003ca name=\"output_gke_cluster\"\u003e\u003c/a\u003e [gke\\_cluster](#output\\_gke\\_cluster) | GKE cluster details |\n| \u003ca name=\"output_load_balancer_details\"\u003e\u003c/a\u003e [load\\_balancer\\_details](#output\\_load\\_balancer\\_details) | Details of the Materialize instance load balancers. |\n| \u003ca name=\"output_network\"\u003e\u003c/a\u003e [network](#output\\_network) | Network details |\n| \u003ca name=\"output_operator\"\u003e\u003c/a\u003e [operator](#output\\_operator) | Materialize operator details |\n| \u003ca name=\"output_service_accounts\"\u003e\u003c/a\u003e [service\\_accounts](#output\\_service\\_accounts) | Service account details |\n| \u003ca name=\"output_storage\"\u003e\u003c/a\u003e [storage](#output\\_storage) | GCS bucket details |\n\n## Connecting to Materialize instances\n\nAccess to the database is through the balancerd pods on:\n* Port 6875 for SQL connections.\n* Port 6876 for HTTP(S) connections.\n\nAccess to the web console is through the console pods on port 8080.\n\n#### TLS support\n\nTLS support is provided by using `cert-manager` and a self-signed `ClusterIssuer`.\n\nMore advanced TLS support using user-provided CAs or per-Materialize `Issuer`s are out of scope for this Terraform module. Please refer to the [cert-manager documentation](https://cert-manager.io/docs/configuration/) for detailed guidance on more advanced usage.\n\n## Upgrade Notes\n\n#### v0.9.0\n\nEnvironmentd now selects swap nodes by default.\n\n#### v0.8.0\n\nYou must upgrade to at least v0.7.x before upgrading to v0.8.x of this terraform code.\n\nBreaking changes:\n* The system node group is renamed and significantly modified, forcing a recreation.\n* Both node groups are now locked to consistent configurations and ON\\\\_DEMAND scheduling.\n* OpenEBS is removed, and with it all support for lgalloc, our legacy spill to disk mechanism.\n\n#### v0.7.0\n\nThis is an intermediate version to handle some changes that must be applied in stages.\nIt is recommended to upgrade to v0.8.x after upgrading to this version.\n\nBreaking changes:\n* Swap is enabled by default.\n* Support for lgalloc, our legacy spill to disk mechanism, is deprecated, and will be removed in the next version.\n* We now always use two node groups, one for system workloads and one for Materialize workloads.\n    * Variables for configuring these node groups have been renamed, so they may be configured separately.\n\nTo avoid downtime when upgrading to future versions, you must perform a rollout at this version.\n1. Ensure your `environmentd_version` is at least `v26.0.0`.\n2. Update your `request_rollout` (and `force_rollout` if already at the correct `environmentd_version`).\n3. Run `terraform apply`.\n\nYou must upgrade to at least v0.6.x before upgrading to v0.7.0 of this terraform code.\n\nIt is strongly recommended to have enabled swap on v0.6.x before upgrading to v0.7.0 or higher.\n\n#### v0.6.1\n\nWe now have some initial support for swap.\n\nWe recommend upgrading to at least v0.5.10 before upgrading to v0.6.x of this terraform code.\n\nTo use swap:\n1. Set `swap_enabled` to `true`.\n2. Ensure your `environmentd_version` is at least `v26.0.0`.\n3. Update your `request_rollout` (and `force_rollout` if already at the correct `environmentd_version`).\n4. Run `terraform apply`.\n\nThis will create a new node group configured for swap, and migrate your clusterd pods there.\n\n#### v0.6.0\n\nThis version is missing the updated helm chart.\nSkip this version, go to v0.6.1.\n\n#### v0.3.0\n\nWe now install `cert-manager` and configure a self-signed `ClusterIssuer` by default.\n\nDue to limitations in Terraform, it cannot plan Kubernetes resources using CRDs that do not exist yet. We have worked around this for new users by only generating the certificate resources when creating Materialize instances that use them, which also cannot be created on the first run.\n\nFor existing users upgrading Materialize instances not previously configured for TLS:\n1. Leave `install_cert_manager` at its default of `true`.\n2. Set `use_self_signed_cluster_issuer` to `false`.\n3. Run `terraform apply`. This will install cert-manager and its CRDs.\n4. Set `use_self_signed_cluster_issuer` back to `true` (the default).\n5. Update the `request_rollout` field of the Materialize instance.\n6. Run `terraform apply`. This will generate the certificates and configure your Materialize instance to use them.\n\u003c!-- END_TF_DOCS --\u003e\n\n\n\n#### Storage Bucket Versioning\nBy default storage bucket versioning is turned off. This both reduces\ncosts and allows for easier cleanup of resources for testing. When running in\nproduction, versioning should be turned on with a sufficient TTL to meet any\ndata-recovery requirements. See `storage_bucket_versioning` and `storage_bucket_version_ttl`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaterializeinc%2Fterraform-google-materialize","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaterializeinc%2Fterraform-google-materialize","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaterializeinc%2Fterraform-google-materialize/lists"}