Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/kelseyhightower/konfig


https://github.com/kelseyhightower/konfig

Last synced: 3 months ago
JSON representation

Awesome Lists containing this project

README 

        
# konfig



konfig enables serverless workloads running on GCP to reference Kubernetes configmaps and secrets stored in GKE clusters at runtime. konfig currently supports Cloud Run and Cloud Functions workloads.



## Usage



konfig is enabled via a single import statement:



```

import (

    ...



    _ "github.com/kelseyhightower/konfig"

)

```



## How Does it Work



The side effect of importing the `konfig` library will cause konfig to:



* call the Cloud Run or Cloud Functions API to get a list of env vars to process. We avoid scanning the running environment as any library can set env vars before konfig runs.

* retrieve the GKE endpoint based on the secret or configmap reference

* retrieve configmap and secret keys from the GKE cluster using the service account provided to the Cloud Run or Cloud Function instance.

* substitute the reference string with the value of the configmap or secret key.



References to Kubernetes configmaps and secrets can be made when defining Cloud Run and Cloud Functions environment variables using the [reference syntax](docs/reference-syntax.md).



## Tutorials



A GKE cluster is used to store configmaps and secrets referenced by Cloud Run and Cloud Function workloads. Ideally an existing cluster can be used. For the purpose of this tutorial create the smallest GKE cluster possible in the `us-central1-a` zone:



```

gcloud container clusters create k0 \

  --cluster-version latest \

  --no-enable-basic-auth \

  --no-enable-ip-alias \

  --metadata disable-legacy-endpoints=true \

  --no-issue-client-certificate \

  --num-nodes 1 \

  --machine-type g1-small \

  --scopes gke-default \

  --zone us-central1-a

```



Download the credentials for the `k0` cluster:



```

gcloud container clusters get-credentials k0 \

  --zone us-central1-a

```



We only need the Kubernetes API server as we only plan to use Kubernetes as an secrets and config store, so delete the default node pool.



```

gcloud container node-pools delete default-pool \

  --cluster k0 \

  --zone us-central1-a

```



With the `k0` GKE cluster in place it's time to create the secrets that will be referenced later in the tutorial.  



```

cat > config.json < The CLUSTER_ID env var should hold the fully qualified path to the k0 cluster. Assuming `hightowerlabs` as the project ID the value would be `/projects/hightowerlabs/zones/us-central1-a/clusters/k0`.



Create the `env` Cloud Run service and set the `ENVIRONMENT`, `FOO` and `CONFIG_FILE` env vars to reference the `env` configmaps and secrets in the `k0` GKE cluster:



```

gcloud alpha run deploy env \

  --allow-unauthenticated \

  --concurrency 50 \

  --image gcr.io/hightowerlabs/env:0.0.1 \

  --memory 2G \

  --region us-central1 \

  --set-env-vars "FOO=\$SecretKeyRef:${CLUSTER_ID}/namespaces/default/secrets/env/keys/foo,CONFIG_FILE=\$SecretKeyRef:${CLUSTER_ID}/namespaces/default/secrets/env/keys/config.json?tempFile=true,ENVIRONMENT=\$ConfigMapKeyRef:${CLUSTER_ID}/namespaces/default/configmaps/env/keys/environment"

```



> The `CONFIG_FILE` env var reference uses the `tempFile` option to write the contents of the `config.json` secret key to a temp file. The `CONFIG_FILE` env var will hold the path to the temp file which can be read during normal program execution.



Retrieve the `env` service HTTP endpoint:



```

ENV_SERVICE_URL=$(gcloud alpha run services describe env \

  --namespace hightowerlabs \

  --region us-central1 \

  --format='value(status.url)')

```



Make an HTTP request to the `env` service:



```

curl $ENV_SERVICE_URL

```



Output:

```

CONFIG_FILE: /tmp/363780357

ENVIRONMENT: production

FOO: bar



# /tmp/363780357

{

  "database": {

    "username": "user",

    "password": "123456789"

  }

}

```



### Cloud Functions Tutorial



konfig pulls referenced secrets and configmaps from GKE clusters using the GCP service account assigned to a Cloud Function. Create the `konfig` service account with the following IAM roles:



* roles/iam.serviceAccountTokenCreator

* roles/cloudfunctions.viewer

* roles/container.viewer



```

PROJECT_ID=$(gcloud config get-value core/project)

```



```

SERVICE_ACCOUNT_NAME="konfig"

```



```

gcloud iam service-accounts create ${SERVICE_ACCOUNT_NAME} \

  --quiet \

  --display-name "konfig service account"

```



```

gcloud projects add-iam-policy-binding ${PROJECT_ID} \

  --quiet \

  --member="serviceAccount:${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \

  --role='roles/iam.serviceAccountTokenCreator'

```



```

gcloud projects add-iam-policy-binding ${PROJECT_ID} \

  --quiet \

  --member="serviceAccount:${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \

  --role='roles/cloudfunctions.viewer'

```



```

gcloud projects add-iam-policy-binding ${PROJECT_ID} \

  --quiet \

  --member="serviceAccount:${SERVICE_ACCOUNT_NAME}@${PROJECT_ID}.iam.gserviceaccount.com" \

  --role='roles/container.developer'

```



Enable the `konfig` GCP service account to access the `env` secret and configmap created in previous section:



```

SERVICE_ACCOUNT_EMAIL="konfig@${PROJECT_ID}.iam.gserviceaccount.com"

```



Create the `konfig` role in the `k0` GKE cluster:



```

kubectl create role konfig \

  --verb get \

  --resource secrets \

  --resource configmaps \

  --resource-name env

```



Bind the `konfig` GCP service account and `konfig` role:



```

kubectl create rolebinding konfig \

  --role konfig \

  --user ${SERVICE_ACCOUNT_EMAIL}

```



At this point the `konfig` GCP service account has access to the configmap and secret named `env` in the default namespace in the `k0` GKE cluster.



> The `konfig` Kubernetes role limits the `konfig` GCP service to the defined `env` secret and configmap in a single namespace. Access to additional secrets and configmaps will require additional permissions.



Deploy the `env` function.



```

cd examples/cloudfunctions/env/

```



```

gcloud alpha functions deploy env \

  --entry-point F \

  --max-instances 10 \

  --memory 128MB \

  --region us-central1 \

  --runtime go111 \

  --service-account $SERVICE_ACCOUNT_EMAIL \

  --set-env-vars "FOO=\$SecretKeyRef:${CLUSTER_ID}/namespaces/default/secrets/env/keys/foo,CONFIG_FILE=\$SecretKeyRef:${CLUSTER_ID}/namespaces/default/secrets/env/keys/config.json?tempFile=true,ENVIRONMENT=\$ConfigMapKeyRef:${CLUSTER_ID}/namespaces/default/configmaps/env/keys/environment" \

  --timeout 30s \

  --trigger-http

```



Enable unauthenticated access to the `env` function HTTP endpoint:



```

gcloud alpha functions add-iam-policy-binding env \

  --member allUsers \

  --role roles/cloudfunctions.invoker

```



Retrieve the HTTPS trigger URL:



```

HTTPS_TRIGGER_URL=$(gcloud beta functions describe env \

  --format 'value(httpsTrigger.url)')

```



Make an HTTP request to the `env` function:



```

curl $HTTPS_TRIGGER_URL

```



```

CONFIG_FILE: /tmp/813067742

ENVIRONMENT: production

FOO: bar



# /tmp/813067742

{

  "database": {

    "username": "user",

    "password": "123456789"

  }

}

```