Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.
Awesome Lists | Featured Topics | Projects
https://github.com/dominodatalab/terraform-gcp-gke

Terraform module for deploying a Domino on GKE
https://github.com/dominodatalab/terraform-gcp-gke
gcp gke terraform
Last synced: 29 days ago
JSON representation
Terraform module for deploying a Domino on GKE
Host: GitHub
URL: https://github.com/dominodatalab/terraform-gcp-gke
Owner: dominodatalab
License: apache-2.0
Created: 2019-10-09T15:19:53.000Z (about 5 years ago)
Default Branch: main
Last Pushed: 2024-11-12T19:04:05.000Z (about 1 month ago)
Last Synced: 2024-11-12T20:18:43.114Z (about 1 month ago)
Topics: gcp, gke, terraform
Language: HCL
Size: 185 KB
Stars: 4
Watchers: 26
Forks: 4
Open Issues: 3
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project

README

        # Domino GKE Terraform

Terraform module which creates a Domino deployment inside of GCP's GKE.

[![CircleCI](https://dl.circleci.com/status-badge/img/gh/dominodatalab/terraform-gcp-gke/tree/main.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/dominodatalab/terraform-gcp-gke/tree/main)

:warning: Important: If you have existing infrastructure created with a version of this module < `v3.0.0` you will need to update the input variable structure.

The following configuration has been removed:

* `description`

* `static_ip_enabled`

The following configuration has been moved:

| Original variable                   | New variable                            | Notes                                                       |

|-------------------------------------|-----------------------------------------|-------------------------------------------------------------|

| `filestore_disabled`                | `storage.filestore.enabled`             |                                                             |

| `filestore_capacity_gb`             | `storage.filestore.capacity_gb`         |                                                             |

| `gcs_force_destroy`                 | `storage.gcs.force_destroy_on_deletion` |                                                             |

| `kubeconfig_output_path`            | `gke.kubeconfig.path`                   |                                                             |

| `enable_network_policy`             | `gke.network_policies`                  |                                                             |

| `kubernetes_version`                | `gke.k8s_version`                       |                                                             |

| `gke_release_channel`               | `gke.release_channel`                   |                                                             |

| `enable_vertical_pod_autoscaling`   | `gke.vertical_pod_autoscaling`          |                                                             |

| `master_firewall_ports`             | `gke.control_plane_ports`               |                                                             |

| `master_authorized_networks_config` | `gke.public_access.cidrs`               | `gke.public_access.enabled` must also be set to take effect |

| `google_dns_managed_zone`           | `managed_dns`                           |                                                             |

| `database_encryption_key_name`      | `kms.database_encryption_key_name`      |                                                             |

A new, enabled-by-default variable to control [GKE dataplane v2](https://cloud.google.com/kubernetes-engine/docs/how-to/dataplane-v2) has been introduced: `gke.advanced_datapath`. For existing infrastructure, make sure to set it to `false` otherwise it will **recreate your cluster**.

## Usage

### Create a Domino development GKE cluster

```hcl

module "gke_cluster" {

  source  = "github.com/dominodatalab/terraform-gcp-gke"

  cluster = "cluster-name"

}

```

### Create a prod GKE cluster

```hcl

module "gke_cluster" {

  source   = "github.com/dominodatalab/terraform-gcp-gke"

  cluster  = "cluster-name"

  project  = "gcp-project"

  location = "us-west1"

  # Some more variables may need to be configured to meet specific needs

}

```

## Manual Deployment

1. Install [gcloud](https://cloud.google.com/sdk/docs/quickstarts) and configure the [Terraform workspace](https://www.terraform.io/docs/state/workspaces.html)

    ```

    gcloud auth application-default login

    terraform init

    terraform workspace new [your-cluster-name]

    ```

1. With the environment setup, you can now apply the terraform module

    ```

    terraform apply -auto-approve

    ```

1. Be sure to cleanup the cluster after you are done working

    ```

    terraform destroy -auto-approve

    ```

## IAM Permissions

The following project [IAM permissions](https://console.cloud.google.com/iam-admin/iam) must be granted to the provisioning user/service:

- Cloud KMS Admin

- Compute Admin

- Compute Instance Admin (v1)

- Compute Network Admin

- Kubernetes Engine Admin

- DNS Administrator

- Cloud Filestore Editor

- Security Admin

- Service Account Admin

- Service Account User

- Storage Admin

It may be possible to lower the "admin" privilage levels to a "creator" level if provisioning cleanup is not required. However, the permissions level for "creator-only" has not been tested. It is assume that a cluster creator can also cleanup (i.e. destroy) the cluster.

## Development

Please submit any feature enhancements, bug fixes, or ideas via pull requests or issues.

# Terraform Docs

## Requirements

| Name | Version |

|------|---------|

|  [terraform](#requirement\_terraform) | >= 1.3 |

|  [google](#requirement\_google) | >= 5.0, < 6.0 |

|  [google-beta](#requirement\_google-beta) | >= 5.0, < 6.0 |

|  [random](#requirement\_random) | ~> 3.1 |

## Providers

| Name | Version |

|------|---------|

|  [google](#provider\_google) | >= 5.0, < 6.0 |

## Modules

No modules.

## Resources

| Name | Type |

|------|------|

| [google_artifact_registry_repository.domino](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/artifact_registry_repository) | resource |

| [google_artifact_registry_repository_iam_member.gcr](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/artifact_registry_repository_iam_member) | resource |

| [google_artifact_registry_repository_iam_member.platform](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/artifact_registry_repository_iam_member) | resource |

| [google_compute_disk.nfs](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_disk) | resource |

| [google_compute_firewall.iap_tcp_forwarding](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |

| [google_compute_firewall.master_webhooks](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |

| [google_compute_firewall.nfs](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_firewall) | resource |

| [google_compute_global_address.static_ip](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_global_address) | resource |

| [google_compute_instance.nfs](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_instance) | resource |

| [google_compute_network.vpc_network](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_network) | resource |

| [google_compute_router.router](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_router) | resource |

| [google_compute_router_nat.nat](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_router_nat) | resource |

| [google_compute_subnetwork.default](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/compute_subnetwork) | resource |

| [google_container_cluster.domino_cluster](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_cluster) | resource |

| [google_container_node_pool.node_pools](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/container_node_pool) | resource |

| [google_dns_record_set.a](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dns_record_set) | resource |

| [google_dns_record_set.a_services](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dns_record_set) | resource |

| [google_dns_record_set.caa](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dns_record_set) | resource |

| [google_dns_record_set.caa_services](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/dns_record_set) | resource |

| [google_filestore_instance.nfs](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/filestore_instance) | resource |

| [google_kms_crypto_key.crypto_key](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/kms_crypto_key) | resource |

| [google_kms_crypto_key_iam_binding.binding](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/kms_crypto_key_iam_binding) | resource |

| [google_kms_key_ring.key_ring](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/kms_key_ring) | resource |

| [google_project_iam_member.platform_roles](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/project_iam_member) | resource |

| [google_project_iam_member.service_account](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/project_iam_member) | resource |

| [google_service_account.accounts](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account) | resource |

| [google_service_account_iam_binding.gcr](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account_iam_binding) | resource |

| [google_service_account_iam_binding.platform_gcs](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/service_account_iam_binding) | resource |

| [google_storage_bucket.bucket](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket) | resource |

| [google_storage_bucket_iam_binding.bucket](https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/storage_bucket_iam_binding) | resource |

| [google_project.domino](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/project) | data source |

| [google_storage_project_service_account.gcs_account](https://registry.terraform.io/providers/hashicorp/google/latest/docs/data-sources/storage_project_service_account) | data source |

## Inputs

| Name | Description | Type | Default | Required |

|------|-------------|------|---------|:--------:|

|  [additional\_node\_pools](#input\_additional\_node\_pools) | additional node pool definitions | 
map(object({
    min_count       = optional(number, 0)
    max_count       = optional(number, 10)
    initial_count   = optional(number, 1)
    max_pods        = optional(number, 30)
    preemptible     = optional(bool, false)
    disk_size_gb    = optional(number, 400)
    image_type      = optional(string, "COS_CONTAINERD")
    instance_type   = optional(string, "n2-standard-8")
    gpu_accelerator = optional(string, "")
    labels          = optional(map(string), {})
    taints          = optional(list(string), [])
    node_locations  = optional(list(string), [])
  })) | `{}` | no |

|  [allowed\_ssh\_ranges](#input\_allowed\_ssh\_ranges) | CIDR ranges allowed to SSH to nodes in the cluster. | `list(string)` | [
  "35.235.240.0/20"
] | no |

|  [deploy\_id](#input\_deploy\_id) | Domino Deployment ID. | `string` | n/a | yes |

|  [gke](#input\_gke) | gke = {
    k8s\_version = Cluster k8s version
    release\_channel = GKE release channel
    public\_access = {
      enabled = Enable API public endpoint
      cidrs = List of CIDR ranges permitted for accessing the public endpoint
    }
    control\_plane\_ports =  Firewall ports to open from the master, e.g., webhooks
    advanced\_datapath = Enable the ADVANCED\_DATAPATH provider
    network\_policies = Enable network policy switch. Cannot be enabled when enable\_advanced\_datapath is true
    vertical\_pod\_autoscaling = Enable GKE vertical scaling
    kubeconfig = {
      path = Specify where the cluster kubeconfig file should be generated.
    }
  } | object({
    k8s_version     = optional(string, "1.30"),
    release_channel = optional(string, "STABLE"),
    public_access = optional(object({
      enabled = optional(bool, false),
      cidrs   = optional(list(string), [])
    }), {}),
    control_plane_ports      = optional(list(string), [])
    advanced_datapath        = optional(bool, true),
    network_policies         = optional(bool, false),
    vertical_pod_autoscaling = optional(bool, true),
    kubeconfig = optional(object({
      path = optional(string, null)
    }), {})
  }) | `{}` | no |

|  [kms](#input\_kms) | kms = {
    database\_encryption\_key\_name = Use an existing KMS key for the Application-layer Secrets Encryption settings. (Optional)
  } | object({
    database_encryption_key_name = optional(string, null)
  }) | `{}` | no |

|  [location](#input\_location) | The location (region or zone) of the cluster. A zone creates a single master. Specifying a region creates replicated masters accross all zones | `string` | `"us-west1-b"` | no |

|  [managed\_dns](#input\_managed\_dns) | managed\_dns = {
    enabled = Whether to create DNS records in the given zone
    name = Managed zone to modify
    dns\_name = DNS record name to create
    service\_prefixes = List of additional prefixes to the dns\_name to create
  } | object({
    enabled          = optional(bool, false)
    name             = optional(string, "")
    dns_name         = optional(string, "")
    service_prefixes = optional(set(string), [])

  }) | `{}` | no |

|  [migration\_permissions](#input\_migration\_permissions) | Add registry permissions to platform service account for migration purposes | `bool` | `false` | no |

|  [namespaces](#input\_namespaces) | Namespace that are used for generating the service account bindings | `object({ platform = string, compute = string })` | n/a | yes |

|  [node\_pools](#input\_node\_pools) | GKE node pool params | object(
    {
      compute = object({
        min_count       = optional(number, 0)
        max_count       = optional(number, 10)
        initial_count   = optional(number, 1)
        max_pods        = optional(number, 30)
        preemptible     = optional(bool, false)
        disk_size_gb    = optional(number, 400)
        image_type      = optional(string, "COS_CONTAINERD")
        instance_type   = optional(string, "n2-highmem-8")
        gpu_accelerator = optional(string, "")
        labels = optional(map(string), {
          "dominodatalab.com/node-pool" = "default"
        })
        taints         = optional(list(string), [])
        node_locations = optional(list(string), [])
      }),
      platform = object({
        min_count       = optional(number, 1)
        max_count       = optional(number, 5)
        initial_count   = optional(number, 1)
        max_pods        = optional(number, 60)
        preemptible     = optional(bool, false)
        disk_size_gb    = optional(number, 100)
        image_type      = optional(string, "COS_CONTAINERD")
        instance_type   = optional(string, "n2-standard-8")
        gpu_accelerator = optional(string, "")
        labels = optional(map(string), {
          "dominodatalab.com/node-pool" = "platform"
        })
        taints         = optional(list(string), [])
        node_locations = optional(list(string), [])
      }),
      gpu = object({
        min_count       = optional(number, 0)
        max_count       = optional(number, 2)
        initial_count   = optional(number, 0)
        max_pods        = optional(number, 30)
        preemptible     = optional(bool, false)
        disk_size_gb    = optional(number, 400)
        image_type      = optional(string, "COS_CONTAINERD")
        instance_type   = optional(string, "n1-highmem-8")
        gpu_accelerator = optional(string, "nvidia-tesla-p100")
        labels = optional(map(string), {
          "dominodatalab.com/node-pool" = "default-gpu"
          "nvidia.com/gpu"              = "true"
        })
        taints = optional(list(string), [
          "nvidia.com/gpu=true:NoExecute"
        ])
        node_locations = optional(list(string), [])
      })
  })
 | {
  "compute": {},
  "gpu": {},
  "platform": {}
} | no |

|  [project](#input\_project) | GCP Project ID | `string` | `"domino-eng-platform-dev"` | no |

|  [storage](#input\_storage) | storage = {
    filestore = {
      enabled = Provision a Filestore instance (for production installs)
      capacity\_gb = Filestore Instance size (GB) for the cluster NFS shared storage
    }
    nfs\_instance = {
      enabled = Provision an instance as an NFS server (to avoid filestore churn during testing)
      capacity\_gb = NFS instance disk size
    }
    gcs = {
      force\_destroy\_on\_deletion = Toogle to allow recursive deletion of all objects in the bucket. if 'false' terraform will NOT be able to delete non-empty buckets.
    } | object({
    filestore = optional(object({
      enabled     = optional(bool, true)
      capacity_gb = optional(number, 1024)
    }), {}),
    nfs_instance = optional(object({
      enabled     = optional(bool, false)
      capacity_gb = optional(number, 100)
    }), {}),
    gcs = optional(object({
      force_destroy_on_deletion = optional(bool, false)
    }), {})
  }) | `{}` | no |

|  [tags](#input\_tags) | Deployment tags. | `map(string)` | `{}` | no |

## Outputs

| Name | Description |

|------|-------------|

|  [bucket\_name](#output\_bucket\_name) | Name of the cloud storage bucket |

|  [cluster](#output\_cluster) | GKE cluster information |

|  [dns](#output\_dns) | The external (public) DNS name for the Domino UI |

|  [domino\_artifact\_repository](#output\_domino\_artifact\_repository) | Domino Google artifact repository |

|  [google\_filestore\_instance](#output\_google\_filestore\_instance) | Domino Google Cloud Filestore instance, name and ip\_address |

|  [nfs\_instance](#output\_nfs\_instance) | Domino Google Cloud Filestore instance, name and ip\_address |

|  [nfs\_instance\_ip](#output\_nfs\_instance\_ip) | NFS instance IP |

|  [project](#output\_project) | GCP project ID |

|  [region](#output\_region) | Region where the cluster is deployed derived from 'location' input variable |

|  [service\_accounts](#output\_service\_accounts) | GKE cluster Workload Identity namespace IAM service accounts |

|  [static\_ip](#output\_static\_ip) | The external (public) static IPv4 for the Domino UI |

|  [uuid](#output\_uuid) | Cluster UUID |