{"id":15222190,"url":"https://github.com/googlecloudplatform/gke-private-cluster-demo","last_synced_at":"2025-10-03T15:30:32.565Z","repository":{"id":66041245,"uuid":"211943610","full_name":"GoogleCloudPlatform/gke-private-cluster-demo","owner":"GoogleCloudPlatform","description":"This guide demonstrates creating a Kubernetes private cluster in Google Kubernetes Engine (GKE) running a sample Kubernetes workload that connects to a Cloud SQL instance using the cloud-sql-proxy \"sidecar\" authenticated using Workload Identity (Beta).","archived":true,"fork":false,"pushed_at":"2020-09-28T12:44:35.000Z","size":178,"stargazers_count":153,"open_issues_count":5,"forks_count":73,"subscribers_count":22,"default_branch":"master","last_synced_at":"2024-12-18T08:40:51.133Z","etag":null,"topics":["containers","database","gcp","gke","gke-helmsman","gke-networking","kubernetes","kubernetes-engine","postgres","postgresql","private-cluster","security","service-account","workload-identity"],"latest_commit_sha":null,"homepage":"","language":"HCL","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GoogleCloudPlatform.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-09-30T19:53:00.000Z","updated_at":"2024-11-08T19:13:05.000Z","dependencies_parsed_at":"2023-04-06T07:41:03.853Z","dependency_job_id":null,"html_url":"https://github.com/GoogleCloudPlatform/gke-private-cluster-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fgke-private-cluster-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fgke-private-cluster-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fgke-private-cluster-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fgke-private-cluster-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GoogleCloudPlatform","download_url":"https://codeload.github.com/GoogleCloudPlatform/gke-private-cluster-demo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":235146544,"owners_count":18943280,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["containers","database","gcp","gke","gke-helmsman","gke-networking","kubernetes","kubernetes-engine","postgres","postgresql","private-cluster","security","service-account","workload-identity"],"created_at":"2024-09-28T15:10:59.201Z","updated_at":"2025-10-03T15:30:32.175Z","avatar_url":"https://github.com/GoogleCloudPlatform.png","language":"HCL","funding_links":[],"categories":[],"sub_categories":[],"readme":"# How to use a Private Cluster in Kubernetes Engine\n\n## Table of Contents\n\n\u003c!-- TOC --\u003e\n* [Introduction](#introduction)\n  * [Public Clusters](#public-clusters)\n  * [Private Clusters](#private-clusters)\n  * [Workload Identity Overview](#workload-identity-overview)\n* [Demo Architecture](#demo-architecture)\n  * [Bastion Host](#bastion-host)\n  * [Workload Identity](#workload-identity)\n* [Prerequisites](#prerequisites)\n  * [Cloud Project](#cloud-project)\n  * [Required GCP APIs](#required-gcp-apis)\n  * [Run Demo in a Google Cloud Shell](#run-demo-in-a-google-cloud-shell)\n    * [Install Terraform](#install-terraform)\n    * [Install Cloud SDK](#install-cloud-sdk)\n    * [Install kubectl CLI](#install-kubectl-cli)\n  * [Authenticate gcloud](#authenticate-gcloud)\n  * [Configure gcloud settings](#configure-gcloud-settings)\n* [Create Resources](#create-resources)\n* [Validation](#validation)\n* [Tear Down](#tear-down)\n* [Troubleshooting](#troubleshooting)\n* [Relevant Material](#relevant-material)\n\u003c!-- TOC --\u003e\n\n## Introduction\n\nThis guide demonstrates creating a Kubernetes private cluster in [Google Kubernetes Engine](https://cloud.google.com/kubernetes-engine/docs/concepts/kubernetes-engine-overview) (GKE) running a sample Kubernetes workload that connects to a [Cloud SQL](https://cloud.google.com/sql/docs/postgres/) instance using the [cloud-sql-proxy](https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine) \"sidecar\". In addition, the [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) (currently in Beta) feature is used to provide credentials directly to the `cloud-sql-proxy` container to facilitate secure tunneling to the `cloud sql` instance without having to handle GCP credentials manually.\n\n### Public Clusters\n\nBy default, GKE clusters are created with a public IP address in front of the Kubernetes API (aka \"masters\" or \"the control plane\").  In addition, the GCE instances that serve as the worker nodes are given both private and ephemeral public IP addresses.  This facilitates ease of administration for using tools like `kubectl` to access the Kubernetes API and `SSH` to access the GCE instances for troubleshooting purposes.  Assuming the GKE cluster was created on a subnet in the `default` VPC network of a project, the default access control allows \"any\" or `0.0.0.0/0` to reach the Kubernetes API and the default firewall rules allow \"any\" or `0.0.0.0/0` to reach the worker nodes via `SSH`.  These clusters are commonly referred to as \"public clusters\".\n\nWhile the authentication and authorization mechanisms for accessing the Kubernetes API over `TLS` and worker nodes via `SSH` offers strong protection against unauthorized access, it is strongly recommended that additional steps are taken to limit the scope of potential access:\n\n1. Restrict to a known list of source subnets for access to the Kubernetes cluster API via `TLS` (tcp/443) using the [master_authorized_networks](https://cloud.google.com/kubernetes-engine/docs/how-to/authorized-networks) access list configuration setting.\n1. Restrict to a known list of source subnets or remove the default firewall rules allowing `SSH` (tcp/22) to the worker nodes.\n\nThis provides several key benefits from a [defense-in-depth](https://en.wikipedia.org/wiki/Defense_in_depth_(computing)) perspective:\n\n1. Reducing the scope of source IPs that can potentially perform a Denial of Service or exploit against the Kubernetes API server or the `SSH` daemon running on the worker nodes.\n1. Reducing the scope of source IPs that can leverage credentials stolen from a developer laptop compromise, credentials found in source code repositories, or credentials/tokens obtained from resources inside the cluster.\n1. Decreasing the likelihood of a newly discovered vulnerability being exploitable and/or granting more time to the team to devise a patching/upgrade strategy.\n\nHowever, fom an operational perspective, managing these access control lists may not be feasible in every organization.  Larger organizations may already have remote access solutions in place either on-premise or in the cloud that they prefer to leverage.  They may also have [dedicated interconnects](https://cloud.google.com/interconnect/docs/concepts/overview) which provide direct, high-bandwidth access from their office network to their GCP environments.  In these cases, there is no need for the GKE clusters to be publicly accessible.\n\n### Private Clusters\n\nGKE offers two configuration items that combine to form what is commonly known as \"[private clusters](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster)\":\n\n1. The GCE instances that serve as the Kubernetes worker nodes do not get assigned a public IP.  Instead, they are only assigned a private IP from the VPC node subnet.  This is the `enable_private_nodes` configuration setting.\n1. The Kubernetes API/GKE API/GKE Control Plane IP is assigned a private IP address from a dedicated subnet for this purpose and is automatically accessible from the node and pod subnets.  This is the `enable_private_endpoint` configuration setting.\n\nWhen both of these are configured on a GKE cluster, a few key behaviors change:\n\n1. The worker nodes no longer have egress to the Internet, and that will prevent the `nodes` and `pods` from having external access.  In addition, `pods` defined using containers from public container image registries like [DockerHub](https://hub.docker.com/search) will not be able to access and pull those containers.  To restore this access, this demo implements a [Cloud NAT](https://cloud.google.com/nat/docs/overview) router to provide egress [Network Address Translation](https://en.wikipedia.org/wiki/Network_address_translation) functionality.\n1. Access to other Google Cloud Platform (GCP) APIs like Google Cloud Storage (GCS) and Google Cloud SQL requires enabling [private API access](https://cloud.google.com/vpc/docs/private-access-options) on the VPC Subnet to enable the private routing configuration which directs traffic headed to GCP APIs entirely over the internal GCP network.  This demo connects to the Cloud SQL instance via [private access](https://cloud.google.com/sql/docs/postgres/private-ip).\n1. Access to the Kubernetes API/GKE Cluster Control Plane will only be possible from within the VPC subnets.  This demo deploys what is known as a [bastion host](https://cloud.google.com/solutions/connecting-securely) as a dedicated GCE instance in the VPC subnet to allow for an administrator/developer to use [SSH Tunneling](https://www.ssh.com/ssh/tunneling/example) to support `kubectl` access.\n\n### Workload Identity Overview\n\nThe [current guide](https://cloud.google.com/sql/docs/mysql/connect-kubernetes-engine) for how to configure the `cloud-sql-proxy` with the necessary GCP credentials involves creating a [service account key](https://cloud.google.com/iam/docs/creating-managing-service-account-keys) in JSON format, storing that in a Kubernetes-native `secret` inside the `namespace` where the `pod` is to run, and configuring the `pod` to mount that secret on a particular file path inside the pod.  However, there are a few downsides to this approach:\n\n1. The credentials inside this JSON file are essentially, static keys and they don't expire unless manually revoked via the GCP APIs.\n1. The act of exporting the credential file to JSON means it touches the disk of the administrator and/or CI/CD system.\n1. Replacing the credential means re-exporting a new Service Account key to JSON, replacing the contents of the Kubernetes `secret` with the updated contents, and restarting the `pod` for the `cloud-sql-proxy` to make use of the new contents.\n\n[Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity) helps remove several manual steps and ensures that the `cloud-sql-proxy` is always using a short-lived credential that auto-rotates on it's own.  Workload Identity, when configured inside a GKE cluster, allows for a Kubernetes Service Account (KSA) to be mapped to a GCP Service Account (GSA) via a process called \"Federation\".  It then installs a proxy on each GKE worker `node` that intercepts all requests to the [GCE Metadata API](https://cloud.google.com/compute/docs/storing-retrieving-metadata) where the dynamic credentials are accessible and returns the current credentials for that GCP Service Account to the process in the `pod` instead of the credentials normally associated with the underlying GCE Instance.  As long as the proper IAM configuration is made to map the KSA to the GSA, the `pod` can be given a dedicated service account with just the permissions needed.\n\n## Demo Architecture\n\nGiven a GCP project, the code in this demo will create the following resources via [Terraform](https://terraform.io):\n\n* A new VPC and new VPC subnets\n* A Cloud NAT router for egress access from the VPC subnets\n* A Cloud SQL Instance with a Private IP\n* A GCE Instance serving as a Bastion Host to support SSH Tunneling\n* A GKE Cluster running Workload Identity with no public IPs on either the API or the worker nodes\n* A sample Kubernetes deployment that uses the `cloud-sql-proxy` to access the Cloud SQL instance privately and uses Workload Identity to dynamically and securely fetch the GCP credentials.\n\n![Demo Architecture Diagram](./img/architecture.png)\n\n1. Exposing workloads inside the GKE cluster is done via a standard load balancer strategy.\n1. Accessing the Kubernetes API Server/Control Plane from the Internet is through an `SSH` tunnel on the `Bastion Host`.\n1. GKE worker `nodes` and `pods` running on those `nodes` access the Internet via Cloud NAT through the Cloud Router.\n1. GKE worker `nodes` and `pods` running on those `nodes` access other GCP APIs such as Cloud SQL via Private API Access.\n\n### Bastion Host\n\nTraditionally, a \"jump host\" or \"bastion host\" is a dedicated, hardened, and heavily monitored system that was placed in the [DMZ](https://en.wikipedia.org/wiki/DMZ_(computing)) of a network to allow for secure, remote access.  In the cloud, this commonly is deployed as a shared instance that multiple users SSH into and work from when accessing cloud resources.  Tools like the [Google Cloud SDK](https://cloud.google.com/sdk/) and [Terraform](https://terraform.io) are often installed on these systems.  There are two problems with this approach:\n\n1. If users authenticate to GCP using `gcloud`, their credentials are stored in the `/home` directory on this instance.  The same issue applies to users who obtain a valid `kubeconfig` file for accessing a GKE cluster.\n1. If users log in via `SSH` and then use `su` or `sudo` to switch to a shared account to perform privileged operations, the audit logs will no longer be able to directly identify who performed an action.  In the case of `sudo` to `root`, that means all GCP credentials in the `/home` directories are available to be used for impersonation attacks (Alex performs malicious actions with Pat's credentials).\n\nThis bastion host attempts to solve for both issues.  It runs two services:\n\n1. An [OpenSSH](https://en.wikipedia.org/wiki/OpenSSH)) daemon to support `SSH` access via `gcloud compute ssh` or via [Identity Awareness Proxy](https://cloud.google.com/iap/).\n1. A [TinyProxy](https://github.com/tinyproxy/tinyproxy) daemon listening on `localhost:8888` that provides a simple [HTTP Proxy](https://en.wikipedia.org/wiki/Proxy_server).\n\nNote: The bastion host is configured to allow `SSH` access from `0.0.0.0/0` via a dedicated firewall rule, but this can and should be restricted to the list of subnets for your needs.\n\nThis means that both `gcloud` and `kubectl` commands can still be run on the local developer/administrator workstation, but `kubectl` commands can be \"proxied\" through an `SSH Tunnel` made to the bastion on their way to the Kubernetes API without disrupting the TLS connection and certificate verification process.\n\nFrom a practical standpoint, using the bastion requires two additional steps for `kubectl` to reach the private cluster's Kubernetes API IP:\n\n1. Run `gcloud compute ssh` and forward a local port (`8888`) to the `localhost:8888` on the bastion host where the `tinyproxy` daemon is listening.\n1. Provide an environment variable (`HTTPS_PROXY=localhost:8888`) when using `kubectl` to instruct it to use the forwarded port that reaches the tinyproxy daemon running on the bastion host on its way to the Kubernetes API.\n\n![Bastion SSH Proxy](/img/bastion_proxy.png)\n\n### Workload Identity\n\nThe use case of this demo requires that the `pgadmin4` (Postgres SQL Admin UI) container has a `cloud-sql-proxy` \"sidecar\" that it uses to connect securely to the Cloud SQL instance.  The IAM Role that is needed to make this connection is `roles/cloudsql.client`.  This demo creates a dedicated GCP Service Account, binds the `roles/cloudsql.client` IAM Role to it at the project level, creates a dedicated Kubernetes Service Account (`postgres`) in the `default` `namespace`, and grants `roles/iam.workloadidentityuser` on the KSA-to-GSA IAM binding.\n\nThe result is that the processes inside `pods` that use the `default/postgres` Kubernetes Service Account that reach for the GCE metadata API to retrieve GCP credentials will be given the credentials from the dedicated GCP Service Account with just the Cloud SQL access permissions.  There are no static GCP Service Account keys to export, no Kubernetes `secrets` to manage, and the credentials automatically rotate themselves.\n\n## Prerequisites\n\nThe steps described in this document require installations of several tools and the proper configuration of authentication to allow them to access your GCP resources.\n\n### Cloud Project\n\nIf you do not have a Google Cloud account, please signup for a free trial [here](https://cloud.google.com). You'll need access to a Google Cloud Project with billing enabled. See [Creating and Managing Projects](https://cloud.google.com/resource-manager/docs/creating-managing-projects) for creating a new project. To make cleanup easier it's recommended to create a new project.\n\n### Required GCP APIs\n\nThe following APIs will be enabled:\n\n* Compute Engine API\n* Kubernetes Engine API\n* Cloud SQL Admin API\n* Secret Token API\n* Stackdriver Logging API\n* Stackdriver Monitoring API\n* IAM Service Account Credentials API\n\n### Run Demo in a Google Cloud Shell\n\nClick the button below to run the demo in a [Google Cloud Shell](https://cloud.google.com/shell/docs).\n\n[![Open in Cloud Shell](http://gstatic.com/cloudssh/images/open-btn.svg)](https://console.cloud.google.com/cloudshell/open?cloudshell_git_repo=https://github.com/GoogleCloudPlatform/gke-private-cluster-demo.git\u0026amp;cloudshell_image=gcr.io/graphite-cloud-shell-images/terraform:latest\u0026amp;cloudshell_tutorial=README.md)\n\nHow to check your account's quota is documented here: [quotas](https://cloud.google.com/compute/quotas).\n\nAll the tools for the demo are installed. When using Cloud Shell execute the following command in order to setup gcloud cli. When executing this command please setup your region and zone.\n\n```console\ngcloud init\n```\n\n### Tools\n\nWhen not using Cloud Shell, the following tools are required:\n\n* Access to an existing Google Cloud project.\n* Bash and common command line tools (Make, etc.)\n* [Terraform v0.12.3+](https://www.terraform.io/downloads.html)\n* [gcloud v255.0.0+](https://cloud.google.com/sdk/downloads)\n* [kubectl](https://kubernetes.io/docs/reference/kubectl/overview/) that matches the latest generally-available GKE cluster version.\n\n#### Install Terraform\n\nTerraform is used to automate the manipulation of cloud infrastructure. Its [installation instructions](https://www.terraform.io/intro/getting-started/install.html) are also available online.\n\n#### Install Cloud SDK\n\nThe Google Cloud SDK is used to interact with your GCP resources. [Installation instructions](https://cloud.google.com/sdk/downloads) for multiple platforms are available online.\n\n#### Install kubectl CLI\n\nThe kubectl CLI is used to interteract with both Kubernetes Engine and Kubernetes in general. [Installation instructions](https://cloud.google.com/kubernetes-engine/docs/quickstart) for multiple platforms are available online.\n\n## Deployment\n\nThe steps below will walk you through using Google Kubernetes Engine to create Private Clusters.\n\n### Authenticate gcloud\n\nPrior to running this demo, ensure you have authenticated your gcloud client by running the following command:\n\n```console\ngcloud auth login\n```\n\n### Configure gcloud settings\n\nRun `gcloud config list` and make sure that `compute/zone`, `compute/region` and `core/project` are populated with values that work for you. You can choose a [region and zone near you](https://cloud.google.com/compute/docs/regions-zones/). You can set their values with the following commands:\n\n```console\n# Where the region is us-central1\ngcloud config set compute/region us-central1\n\nUpdated property [compute/region].\n```\n\n```console\n# Where the zone inside the region is us-central1-c\ngcloud config set compute/zone us-central1-c\n\nUpdated property [compute/zone].\n```\n\n```console\n# Where the project name is my-project-name\ngcloud config set project my-project-name\n\nUpdated property [core/project].\n```\n\n## Create Resources\n\nTo create the entire environment via Terraform, run the following command:\n\n```console\nmake create\n\nApply complete! Resources: 33 added, 0 changed, 0 destroyed.\n\nOutputs:\n\n...snip...\nbastion_kubectl = HTTPS_PROXY=localhost:8888 kubectl get pods --all-namespaces\nbastion_ssh = gcloud compute ssh private-cluster-bastion --project bgeesaman-gke-demos --zone us-central1-a -- -L8888:127.0.0.1:8888\ncluster_ca_certificate = \u003csensitive\u003e\ncluster_endpoint = 172.16.0.18\ncluster_name = private-cluster\ngcp_serviceaccount = private-cluster-pg-sa@my-project-name.iam.gserviceaccount.com\nget_credentials = gcloud container clusters get-credentials --project my-project-name --region us-central1 --internal-ip private-cluster\npostgres_connection = my-project-name:us-central1:private-cluster-pg-410120c4\npostgres_instance = private-cluster-pg-410120c4\npostgres_pass = \u003csensitive\u003e\npostgres_user = postgres\nFetching cluster endpoint and auth data.\nkubeconfig entry generated for private-cluster.\n```\n\nNext, review the `pgadmin` `deployment` located in the `/manifests` directory:\n\n```console\ncat manifests/pgadmin-deployment.yaml\n```\n\nThe manifest contains comments that explain the key features of the deployment configuration.  Now, deploy the application via:\n\n```console\nmake deploy\n\nDetecting SSH Bastion Tunnel/Proxy\nDid not detect a running SSH tunnel.  Opening a new one.\nPseudo-terminal will not be allocated because stdin is not a terminal.\nSSH Tunnel/Proxy is now running.\nCreating the PgAdmin Configmap\nconfigmap/connectionname created\nCreating the PgAdmin Console secret\nsecret/pgadmin-console created\nserviceaccount/postgres created\nserviceaccount/postgres annotated\nDeploying PgAdmin\ndeployment.apps/pgadmin4-deployment created\nWaiting for rollout to complete and pod available.\nWaiting for deployment \"pgadmin4-deployment\" rollout to finish: 0 of 1 updated replicas are available...\ndeployment \"pgadmin4-deployment\" successfully rolled out\n```\n\nThe `make deploy` step ran the contents of `./scripts/deploy.sh` which did a few things:\n\n1. Created an SSH tunnel to the Bastion Host (if it wasn't running already) that should still be running in the background.\n1. Used `kubectl` to create a configmap containing the connection string for connecting to the correct Cloud SQL Instance, a dedicated service account named `postgres` in the `default` namespace, and added a custom annotation to that service account.\n1. Ran `kubectl` to deploy the `pgadmin4` deployment manifest.\n1. Ran `kubectl` to wait for that deployment to be up and healthy.\n\nNow, with the SSH tunnel still running in the background, you can interact with the GKE cluster using `kubectl`.  For example:\n\n```console\nHTTPS_PROXY=localhost:8888 kubectl get pods --all-namespaces\n```\n\nBecause that environment variable must be present for each invocation of `kubectl`, you can `alias` that command to reduce the amount of typing needed each time:\n\n```console\nalias k=\"HTTPS_PROXY=localhost:8888 kubectl\"\n```\n\nAnd now, using `kubectl` looks like the following:\n\n```console\nk get pods --all-namespaces\nk get namespaces\nk get svc --all-namespaces\n```\n\nNote: `export`-ing the `HTTPS_PROXY` setting in the current terminal may alter the behavior of other common tools that honor that setting (e.g. `curl` and other web related tools).  The shell `alias` helps localize the usage to the current invocation of the command only.\n\n## Validation\n\nIf no errors are displayed during deployment, you should see your Kubernetes Engine cluster in the [GCP Console](https://console.cloud.google.com/kubernetes) with the sample application deployed. This may take a few minutes.\n\nValidation is fully automated. The validation script checks for the existence of the Postgress DB, Google Kubernetes Engine cluster, and the deployment of pgAdmin. In order to validate that resources are installed and working correctly, run:\n\n```console\nmake validate\n\nDetecting SSH Bastion Tunnel/Proxy\nDetected a running SSH tunnel.  Skipping.\nChecking that pgAdmin is deployed on the cluster... pass\nChecking that pgAdmin is able to connect to the database instance... pass\n```\n\nThe `make validate` performs two simple checks that can be done manually.  Checking the status of the `pgadmin4` deployment for health:\n\n```console\nk rollout status --timeout=10s -f manifests/pgadmin-deployment.yaml\n\ndeployment \"pgadmin4-deployment\" successfully rolled out\n```\n\nAnd using `kubectl exec` to run the `pg_isready` command from the `pgadmin4` container which performs a test connection to the Postgres database and verifies end-to-end success:\n\n```console\nk exec -it -n default \"$(k get pod -l 'app=pgadmin4' -ojsonpath='{.items[].metadata.name}')\" -c pgadmin4 -- pg_isready -h localhost -t 10\n\nlocalhost:5432 - accepting connections\n```\n\nYou may also wish to view the logs of the key `pods` in this deployment.  To see the logs of the `pgadmin4` container:\n\n```console\nk logs -l 'app=pgadmin4' -c pgadmin4 -f\n```\n\nTo see the logs of the `cloud-sql-proxy` container:\n\n```console\nk logs -l 'app=pgadmin4' -c cloudsql-proxy -f \n```\n\nTo see the logs of the `gke-metadata-proxy` containers which handle requests for \"Workload Identity\":\n\n```console\nk logs -n kube-system -l 'k8s-app=gke-metadata-server' -f\n```\n\n## Tear Down\n\nWhen you are finished with this example you will want to clean up the resources that were created so that you avoid accruing charges. Teardown is fully automated. The destroy script deletes all resources created using Terraform. Terraform variable configuration and state files are also cleaned if Terraform destroy is successful. To delete all created resources in GCP, run:\n\n```console\nmake teardown\n```\n\n## Troubleshooting\n\n* **The create script fails with a `Permission denied` when running Terraform** - The credentials that Terraform is using do not provide the necessary permissions to create resources in the selected projects. Ensure that the account listed in `gcloud config list` has necessary permissions to create resources. If it does, regenerate the application default credentials using `gcloud auth application-default login`.\n* **Terraform timeouts** - Sometimes resources may take longer than usual to create and Terraform will timeout. The solution is to just run `make create` again. Terraform should pick up where it left off.\n\n## Relevant Material\n\n* [Private GKE Clusters](https://cloud.google.com/kubernetes-engine/docs/how-to/creating-a-cluster)\n* [Workload Identity](https://cloud.google.com/kubernetes-engine/docs/how-to/workload-identity)\n* [Terraform Google Provider](https://www.terraform.io/docs/providers/google/)\n* [Securely Connecting to VM Instances](https://cloud.google.com/solutions/connecting-securely)\n* [Cloud NAT](https://cloud.google.com/nat/docs/overview)\n* [Kubernetes Engine - Hardening your cluster's security](https://cloud.google.com/kubernetes-engine/docs/how-to/hardening-your-cluster)\n\nNote, **This is not an officially supported Google product**\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecloudplatform%2Fgke-private-cluster-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgooglecloudplatform%2Fgke-private-cluster-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecloudplatform%2Fgke-private-cluster-demo/lists"}