{"id":16861808,"url":"https://github.com/maelvls/kubectl-incluster","last_synced_at":"2025-04-11T10:48:53.593Z","repository":{"id":79937784,"uuid":"317178774","full_name":"maelvls/kubectl-incluster","owner":"maelvls","description":"📜 A kubectl plugin that prints a valid token-based kubeconfig you can use to run kubectl and controllers out-of-cluster for debugging purposes","archived":false,"fork":false,"pushed_at":"2024-04-29T14:19:22.000Z","size":205,"stargazers_count":9,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-25T07:12:26.560Z","etag":null,"topics":["kubeconfig","kubectl-plugins","kubernetes","mitmproxy","telepresence"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/maelvls.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-11-30T09:51:52.000Z","updated_at":"2024-10-12T12:35:48.000Z","dependencies_parsed_at":"2024-06-20T00:17:11.091Z","dependency_job_id":"0264144b-21fe-4e8f-a02a-08f114a0d579","html_url":"https://github.com/maelvls/kubectl-incluster","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maelvls%2Fkubectl-incluster","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maelvls%2Fkubectl-incluster/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maelvls%2Fkubectl-incluster/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/maelvls%2Fkubectl-incluster/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/maelvls","download_url":"https://codeload.github.com/maelvls/kubectl-incluster/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248379445,"owners_count":21094164,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubeconfig","kubectl-plugins","kubernetes","mitmproxy","telepresence"],"created_at":"2024-10-13T14:33:28.350Z","updated_at":"2025-04-11T10:48:53.572Z","avatar_url":"https://github.com/maelvls.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# kubectl-incluster\n\nI wrote this kubectl plugin in order to create a kubeconfig file out of an\nin-cluster configuration, i.e., using the mounted service account token and\nCA cert:\n\n```sh\n/var/run/secrets/kubernetes.io/serviceaccount/token\n/var/run/secrets/kubernetes.io/serviceaccount/ca.crt\n```\n\nRunning `kubectl incluster` from inside a running pod will print a working\nkubeconfig that you can use somewhere else.\n\n**Content:**\n\n- [Use-case: Telepresence 1 + mitmproxy for debugging cert-manager](#use-case-telepresence-1--mitmproxy-for-debugging-cert-manager)\n  - [Optional: read the Let's Encrypt `jose+json` payloads](#optional-read-the-lets-encrypt-josejson-payloads)\n- [Use-case: Telepresence 2 + mitmproxy for debugging cert-manager](#use-case-telepresence-2--mitmproxy-for-debugging-cert-manager)\n- [Use-case: Telepresence 1 + mitmproxy for debugging the preflight agent](#use-case-telepresence-1--mitmproxy-for-debugging-the-preflight-agent)\n- [Use-case: mitmproxy inside the cluster (as opposed to using Telepresence 1)](#use-case-mitmproxy-inside-the-cluster-as-opposed-to-using-telepresence-1)\n- [Use-case: mitmproxy without kubectl-incluster](#use-case-mitmproxy-without-kubectl-incluster)\n- [Use-case: mitmproxy to debug an admission webhook](#use-case-mitmproxy-to-debug-an-admission-webhook)\n- [`kubectl-incluster` manual](#kubectl-incluster-manual)\n  - [The `--print-client-cert` flag](#the---print-client-cert-flag)\n- [mitmproxy and Telepresence gotchas](#mitmproxy-and-telepresence-gotchas)\n  - [The `$TELEPRESENCE_ROOT` stays empty on Linux](#the-telepresence_root-stays-empty-on-linux)\n- [Workaround for Google Kubernetes Engine (GKE)](#workaround-for-google-kubernetes-engine-gke)\n\n## Use-case: Telepresence 1 + mitmproxy for debugging cert-manager\n\nIn order to inspect the egress traffic coming from a Kubernetes controller\n(here, [cert-manager](https://github.com/jetstack/cert-manager)), I want to\nbe able to use mitmproxy through a Telepresence `--run-shell` session. For\nexample, let's imagine you have a cert-manager deployment already running\nand that you want to see what requests it makes.\n\n\u003e ⚠️ Telepresence 1 (written in Python) has been \"superseeded\" by Telepresence 2 (written in Go).\n\u003e This use-case focuses on Telepresence 1. To see the same use-case using Telepresence 2,\n\u003e scroll down. Note that Telepresence 1 works better in certain scenarios (e.g., Telepresence 1\n\u003e supports replacing Deployments that have `runAsNonRoot: false` set on them).\n\nFirst, install Telepresence 1. To install it on Ubuntu:\n\n```sh\ncurl -s https://packagecloud.io/install/repositories/datawireio/telepresence/script.deb.sh | sudo bash\nsudo apt install --no-install-recommends telepresence\n```\n\n(see [macOS instructions](https://www.telepresence.io/docs/v1/reference/install/))\n\nThen, install mitmproxy on both Linux and macOS:\n\n```sh\nbrew install mitmproxy\n```\n\nThe next step is to start mitmproxy. We will be using [watch-stream.py](/watch-stream.py),\na script that makes sure the streaming GET requests are properly streamed by mitmproxy:\n\n```sh\ncurl -L https://raw.githubusercontent.com/maelvls/kubectl-incluster/main/watch-stream.py \u003e/tmp/watch-stream.py\nmitmproxy -p 9090 --ssl-insecure -s /tmp/watch-stream.py --set client_certs=\u003c(kubectl incluster --print-client-cert)\n```\n\n\u003e Note that we could avoid using `--ssl-insecure` by replacing it with\n\u003e something like:\n\u003e\n\u003e ```sh\n\u003e mitmproxy -p 9090 --set ssl_verify_upstream_trusted_ca=\u003c(kubectl incluster --print-ca-cert)\n\u003e ```\n\u003e\n\u003e But since I don't run mitmproxy from inside the Telepresence shell, I\n\u003e don't have access to the `$TELEPRESENCE_ROOT` variable. So I don't bother\n\u003e and use `--ssl-insecure` instead.\n\nLet us now install `kubectl-incluser`:\n\n```sh\ngo install github.com/maelvls/kubectl-incluster@latest\n```\n\nNow, let's run cert-manager locally inside a Telepresence shell:\n\n```sh\n% git clone https://github.com/jetstack/cert-manager \u0026\u0026 cd cert-manager\n% telepresence --namespace cert-manager --swap-deployment cert-manager --run-shell\nT: Using a Pod instead of a Deployment for the Telepresence proxy. If you experience problems, please file an issue!\nT: Forwarding remote port 9402 to local port 9402.\nT: Connected. Flushing DNS cache.\nT: Setup complete. Launching your command.\n\n@boring_wozniak|bash-3.2$\n```\n\nNow, from this shell, let us run cert-manager:\n\n```sh\nHTTPS_PROXY=localhost:9090 go run ./cmd/controller --leader-elect=false --kubeconfig \u003c(kubectl incluster --root $TELEPRESENCE_ROOT --replace-ca-cert ~/.mitmproxy/mitmproxy-ca.pem) -v=4\n```\n\nAnd TADA! We see all the requests made by our controller:\n\n\u003cimg alt=\"An mitmproxy screenshot when debugging cert-manager-controller. Screenshot stored in the issue https://github.com/maelvls/kubectl-incluster/issues/1\" src=\"https://user-images.githubusercontent.com/2195781/100645025-64f89880-333c-11eb-9a3f-b6aa8cde497d.png\"\u003e\n\n### Optional: read the Let's Encrypt `jose+json` payloads\n\nIn mitmproxy, it is hard to read the JSON payloads sent by the ACME server since\nthey are base64 encoded:\n\n```json\n{\n  \"protected\": \"eyJhbGciOiJSUzI1Ni...E1MzY1Mjg1MCJ9\",\n  \"payload\": \"eyJjc3IiOiJNSUlDblRDQ0FZ...EU3lHQ3BjLTlfanVBIn0\",\n  \"signature\": \"qqYGqZDSSUwuLLxm6-...nygkb5S8igKPrw\"\n}\n```\n\nYou can use the `josejson.py` script to decode the payload and protected fields\n\"inline\":\n\n```sh\ncurl -L https://raw.githubusercontent.com/maelvls/kubectl-incluster/main/josejson.py \u003e/tmp/josejson.py\nmitmproxy -p 9090 -s /tmp/josejson.py\n```\n\nAnd now you can see the payload and protected fields \"inline\":\n\n```json\n{\n  \"payload\": {\n    \"csr\": \"MIICnTCCAYUCAQAwADCCAS...duroowkXh3tqgVFDSyGCpc-9_juA\"\n  },\n  \"protected\": {\n    \"alg\": \"RS256\",\n    \"kid\": \"https://acme-v02.api.letsencrypt.org/acme/acct/204416270\",\n    \"nonce\": \"01017mM9r6R_TpKL-5zxAmMF5JmTCBI-v6AsLlGedj3pD1E\",\n    \"url\": \"https://acme-v02.api.letsencrypt.org/acme/finalize/204416270/25153652850\"\n  },\n  \"signature\": \"qqYGqZDSSUwuLLxm6-...nygkb5S8igKPrw\"\n}\n```\n\n## Use-case: Telepresence 2 + mitmproxy for debugging cert-manager\n\n⚠️ As of 17 Sept 2021, Telepresence 2 does not support deployments configured\nwith `runAsNonRoot: false` as per the issue\n[#875](https://github.com/telepresenceio/telepresence/issues/875). cert-manager,\nby default, uses `runAsNonRoot: false`, and you will see that Telepresence 2\nhangs forever:\n\n```sh\n$ telepresence intercept cert-manager -n cert-manager -- bash\nLaunching Telepresence Root Daemon\nNeed root privileges to run: /home/mvalais/bin/telepresence daemon-foreground /home/mvalais/.cache/telepresence/logs /home/mvalais/.config/telepresence ''\n[sudo] password for mvalais:\nLaunching Telepresence User Daemon\nConnected to context k3d-boring (https://0.0.0.0:39767)\n# ❌ Hangs forever.\n```\n\nTo work around this, you need to change the securityContext of your cert-manager:\n\n```sh\nkubectl patch deploy cert-manager -n cert-manager --patch 'spec: {template: {spec: {securityContext: {runAsNonRoot: false}}}}'\n```\n\nThis time, Telepresence 2 should work:\n\n```sh\n$ telepresence intercept cert-manager -n cert-manager -- bash\ntelepresence intercept cert-manager -n cert-manager -- bash\nUsing Deployment cert-manager\nintercepted\n    Intercept name    : cert-manager-cert-manager\n    State             : ACTIVE\n    Workload kind     : Deployment\n    Destination       : 127.0.0.1:8080\n    Volume Mount Point: /tmp/telfs-695026645\n    Intercepting      : all TCP connections\nmvalais@aorus:~/code/cert-manager$\n```\n\n## Use-case: Telepresence 1 + mitmproxy for debugging the preflight agent\n\nThe preflight agent is a binary that runs in your Kubernetes cluster and\nreports information about certificates to the\n\u003chttps://platform.jetstack.io\u003e dashboard. The free tier allows you to see\nif any of your certificates managed by cert-manager has an issue.\n\nTo debug the agent, the first step is to have the agent built:\n\n```sh\ngit clone https://github.com/jetstack/preflight\ncd preflight\nmake install\n```\n\nThen, you want to run telepresence:\n\n```sh\ntelepresence --namespace jetstack-secure --swap-deployment agent --run-shell\n```\n\nRun the mitmproxy instance:\n\n```sh\n# In another shell, not in the telepresence shell.\nmitmproxy -p 9090 --ssl-insecure --set client_certs=\u003c(kubectl incluster --print-client-cert)\n```\n\nFinally you can run the agent:\n\n\u003e **🔰 Tip:** to know which command-line arguments are used by a given deployment,\n\u003e you can use `kubectl-args` that extracts the `args` for the deployment.\n\u003e Imagining that you have `~/bin` in your PATH, you can install it with:\n\u003e\n\u003e ```sh\n\u003e cat \u003c\u003c'EOF' \u003e /tmp/kubectl-args\n\u003e #! /bin/bash\n\u003e set -e -o pipefail\n\u003e kubectl get deploy -ojsonpath='{.spec.template.spec.containers[0].args}' \"$@\" | jq -r '.[]' | awk '{if($2 != \"\"){print \"\\\"\" $0 \"\\\"\"}else{print $0}}' |  tr '\\n' ' '; echo\n\u003e EOF\n\u003e install /tmp/kubectl-args ~/bin\n\u003e ```\n\u003e\n\u003e Then, use it with:\n\u003e\n\u003e ```sh\n\u003e % kubectl args -n jetstack-secure agent\n\u003e agent -c /etc/jetstack-secure/agent/config/config.yaml -k /etc/jetstack-secure/agent/credentials/credentials.json -p 0h1m0s\n\u003e ```\n\n```sh\n# Inside the telepresence shell.\nHTTPS_PROXY=127.0.0.1:9090 KUBECONFIG=$(kubectl incluster --root $TELEPRESENCE_ROOT --replace-ca-cert ~/.mitmproxy/mitmproxy-ca.pem \u003e/tmp/foo \u0026\u0026 echo /tmp/foo) preflight agent -c $TELEPRESENCE_ROOT/etc/jetstack-secure/agent/config/config.yaml -k $TELEPRESENCE_ROOT/etc/jetstack-secure/agent/credentials/credentials.json -p 0h1m0s\n```\n\nYou will see:\n\n\u003cimg alt=\"An mitmproxy screenshot when debugging the preflight agent that reports to https://platform.jetstack.io. Screenshot stored in the issue https://github.com/maelvls/kubectl-incluster/issues/1\" src=\"https://user-images.githubusercontent.com/2195781/110499573-aa292500-80f8-11eb-8570-c90b56475f27.png\"\u003e\n\n## Use-case: mitmproxy inside the cluster (as opposed to using Telepresence 1)\n\nFirst, we need to have an instance of mitmproxy running:\n\n```sh\nkubectl apply -n jetstack-secure -f \u003c\u003cEOF\napiVersion: apps/v1\nkind: Deployment\nmetadata:\n  name: mitmproxy\n  labels:\n    app: mitmproxy\nspec:\n  replicas: 1\n  selector:\n    matchLabels:\n      app: mitmproxy\n  template:\n    metadata:\n      labels:\n        app: mitmproxy\n    spec:\n      containers:\n        - name: mitmproxy\n          image: mitmproxy/mitmproxy:latest\n          args: [mitmweb, -p, \"9090\"]\n          imagePullPolicy: Always\n          ports:\n            - containerPort: 8081\n              name: ui\n            - containerPort: 9090\n              name: proxy\n          resources:\n            limits:\n              memory: \"460Mi\"\n              cpu: \"200m\"\n---\nkind: Service\napiVersion: v1\nmetadata:\n  name: mitmproxy\nspec:\n  ports:\n    - name: ui\n      port: 8081\n    - name: proxy\n      port: 9090\n  selector:\n    app: mitmproxy\nEOF\n```\n\nThen, let us see the mitmweb UI:\n\n```sh\nkubectl port-forward -n jetstack-secure $(kubectl get pod -n jetstack-secure -l app.kubernetes.io/name=agent -oname) 8081:8081\n```\n\nand head to \u003chttp://localhost:8081\u003e.\n\nThen, we need to add that to the running deployment that we want to debug:\n\n```sh\nkubectl edit deploy your-deployment\n```\n\nand add the following to the container's `env`:\n\n```yaml\nspec:\n  containers:\n    - env:\n        - name: HTTPS_PROXY\n          value: http://mitmproxy:9090\n```\n\n⚠️ IMPORTANT ⚠️ : you also have to make sure the container's binary can\ndisable TLS verification. Otherwise, no way to do that...\n\n## Use-case: mitmproxy without kubectl-incluster\n\nLet us imagine we want to trace what `kubectl get pods` is doing under the\nhood.\n\nFirst, let us work around the fact that Go binaries do not honor the\n`HTTPS_PROXY` variable for the `127.0.0.1` and `localhost` domains. Instead\nof `127.0.0.1`, we will use the domain `me`:\n\n```sh\ngrep \"127.0.0.1[ ]*me$\" /etc/hosts || sudo tee --append /etc/hosts \u003c\u003c\u003c\"127.0.0.1 me\"\n```\n\nThen, let us make sure our system trusts Mitmproxy's root CA:\n\n```sh\n# Linux\nsudo mkdir -p /usr/share/ca-certificates/mitmproxy\nsudo cp ~/.mitmproxy/mitmproxy-ca-cert.pem /usr/share/ca-certificates/mitmproxy/mitmproxy-ca-cert.crt\ngrep mitmproxy/mitmproxy-ca-cert.crt /etc/ca-certificates.conf \\\n  || sudo tee --append /etc/ca-certificates.conf \u003c\u003c\u003c\"mitmproxy/mitmproxy-ca-cert.crt\"\nsudo update-ca-certificates\n\n# macOS\nsudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain ~/.mitmproxy/mitmproxy-ca-cert.pem\n```\n\nLet us start `mitmproxy`. We have to use `--ssl-insecure` due to the fact that\nwe don't want to bother having `mitmproxy` to trust the apiserver. We need to\ngive the correct client certificate to the proxy (if you are using a client\ncertificate):\n\n```sh\ncurl -L https://raw.githubusercontent.com/maelvls/kubectl-incluster/main/watch-stream.py \u003e/tmp/watch-stream.py\nkubectl config view --minify --flatten -o=go-template='{{(index ((index .users 0).user) \"client-key-data\")}}' | base64 -d \u003e/tmp/client.pem\nkubectl config view --minify --flatten -o=go-template='{{(index ((index .users 0).user) \"client-certificate-data\")}}' | base64 -d \u003e\u003e/tmp/client.pem\nmitmproxy -p 9090 --ssl-insecure --set client_certs=/tmp/client.pem -s /tmp/watch-stream.py\n```\n\nFinally, let us run the command we want to HTTP-inspect:\n\n```sh\nHTTPS_PROXY=:9090 KUBECONFIG=\u003c(kubectl config view --minify --flatten \\\n    | sed \"s|certificate-authority-data:.*|certificate-authority-data: $(base64 -w0 \u003c ~/.mitmproxy/mitmproxy-ca-cert.pem)|g\" \\\n    | sed \"s|127.0.0.1|me|\") \\\n  kubectl get pods\n```\n\n\u003e 🔰 The command `kubectl config view --minify` prints the kube config for\n\u003e the current context, which comes in very handy here.\n\n## Use-case: mitmproxy to debug an admission webhook\n\nUnlike all the previous use-cases, we will be using mitmproxy in \"reverse proxy\"\nmode:\n\n```\n+------------------+                  +------------------+                  +--------------------+\n|                  |                  |                  |                  |                    |\n|                  |-----------------\u003e|                  |-----------------\u003e|                    |\n|    apiserver     |                  |    mitmproxy     |                  |cert-manager-webhook|\n|                  |                  |      :8080       |                  |       :8081        |\n|                  |                  |                  |                  |                    |\n+------------------+                  +------------------+                  +--------------------+\n                                         reverse proxy\n```\n\nWe want to be able to see what the apiserver is sending to cert-manager webhook.\n\nTo do that, we will be running the webhook out-of-cluster to make it easier when\nrunning mitmproxy. We could be using `kubetap` that does a similar job, but it\ndoes not support setting your own CA certificate to be served, meaning that we\ncan't have a way to make the apiserver trust the webhook.\n\n```sh\n$ telepresence intercept cert-manager-webhook -n cert-manager -- bash\nUsing Deployment cert-manager-webhook\nintercepted\n    Intercept name    : cert-manager-webhook-cert-manager\n    State             : ACTIVE\n    Workload kind     : Deployment\n    Destination       : 127.0.0.1:8080\n    Volume Mount Point: /tmp/telfs-584691868\n    Intercepting      : all TCP connections\n```\n\nAnything that hits `cert-manager-webhook.cert-manager.svc:443` will be forwarded\nto the host on `127.0.0.1:8080`.\n\nWe now need to force the apiserver to trust mitmproxy:\n\n```sh\nkubectl apply -f- \u003c\u003cEOF\napiVersion: v1\nkind: Secret\nmetadata:\n  name: cert-manager-webhook-ca\n  namespace: cert-manager\ndata:\n  ca.crt: \"$(cat ~/.mitmproxy/mitmproxy-ca-cert.pem | base64 -w0)\"\n  tls.crt: \"$(cat ~/.mitmproxy/mitmproxy-ca-cert.pem | base64 -w0)\"\n  tls.key: \"$(cat ~/.mitmproxy/mitmproxy-ca.pem | base64 -w0)\"\nEOF\n```\n\ncert-manager-cainjector will then take care of stuffing the above `ca.crt` into\nthe `caBundle` in all cert-manager CRDs so that the apiserver knows how to\nverify the TLS certificate handled by mitmproxy.\n\nNow, let's run mitmproxy on port 8080:\n\n```\nmitmproxy -p 8080 --mode reverse:https://localhost:8081 --ssl-insecure\n```\n\nFinally, we can run the webhook on port 8081:\n\n```\ngo run ./cmd/webhook/ webhook --v=2 --secure-port=8081 --dynamic-serving-ca-secret-namespace=cert-manager \\\n  --dynamic-serving-ca-secret-name=cert-manager-webhook-ca \\\n  --dynamic-serving-dns-names=cert-manager-webhook,cert-manager-webhook.cert-manager,cert-manager-webhook.cert-manager.svc \\\n  --kubeconfig=$(kubectl incluster \u003e/tmp/in.pem \u0026\u0026 echo /tmp/in.pem)\n```\n\n## `kubectl-incluster` manual\n\nThe `--help` output of `kubectl-incluster` is below:\n\n```\nUsage of kubectl-incluster:\n  -context string\n      The name of the kubeconfig context to use.\n  -kubeconfig string\n      Path to the kubeconfig file to use.\n  -embed\n      Deprecated since this is now the default behavior. Embeds the token and\n      ca.crt data inside the kubeconfig instead of using file paths.\n  -print-ca-cert\n      Instead of printing a kubeconfig, print the content of the kube config's\n      certificate-authority-data.\n  -print-client-cert\n      Instead of printing the kube config, print the content of the kube\n      config's client-certificate-data followed by the client-key-data.\n  -replace-ca-cert string\n      Instead of using the cacert provided in /var/run/secrets or in the kube\n      config, use this one. Useful when using a proxy like mitmproxy.\n  -replace-cacert string\n      Deprecated, please use --replace-ca-cert instead.\n  -root string\n      The container root. You can also set CONTAINER_ROOT instead. If\n      TELEPRESENCE_ROOT is set, it will default to that.\n  -serviceaccount string\n      Instead of using the current pod's /var/run/secrets (when in cluster)\n      or the local kubeconfig (when out-of-cluster), you can use this flag to\n      use the token and ca.crt from a given service account, for example\n      'namespace-1/serviceaccount-1'. Useful when you want to force using a\n      token (only available using service accounts) over client certificates\n      provided in the kubeconfig, which is useful whenusing mitmproxy since\n      the token is passed as a header (HTTP) instead of a client certificate\n      (TLS).\n```\n\n### The `--print-client-cert` flag\n\nBy default, `kubectl-incluster` prints the \"minified\" kube config (i.e., just\nthe kube config for the current cluster if you are running out-of-cluster).\n\nFor example:\n\n```\n$ kubectl incluster\napiVersion: v1\nclusters:\n- cluster:\n    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJkekNDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdGMyVnkKZG1WeUxXTmhRREUyTWpneE5UTXpNVGt3SGhjTk1qRXdPREExTURnME9ETTVXaGNOTXpFd09EQXpNRGcwT0RNNQpXakFqTVNFd0h3WURWUVFEREJock0zTXRjMlZ5ZG1WeUxXTmhRREUyTWpneE5UTXpNVGt3V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFUbkFnZ3VDQ3p2UTJXemFZbFlRQ1BoaVNEcFcrQUNXUytLQ1ZmdFZjUlcKY2ZXdmxqZ3pnMGlWcXdaYlBoVk1xYitzRktPeXRnd1M0QnNPZXV5MlZEQ1hvMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVXNjUWVad3JKNlpkbFpzWkxGK2tpCnhRSVhTNDh3Q2dZSUtvWkl6ajBFQXdJRFNBQXdSUUlnUWpyOWo0anNFWmlZZHNjU2RBSktreStlOWxjUTZYRncKejVOQkF2SUNuMEVDSVFDdVJIMGhtbmNJcnIveGNjMDkwZEFiN0c2V0d5T2R3M2w1ZXFGeFlQWnJkUT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K\n    server: https://0.0.0.0:43519\n  name: kubectl-incluster\ncontexts:\n- context:\n    cluster: kubectl-incluster\n    user: kubectl-incluster\n  name: kubectl-incluster\ncurrent-context: kubectl-incluster\nkind: Config\npreferences: {}\nusers:\n- name: kubectl-incluster\n  user:\n    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJrVENDQVRlZ0F3SUJBZ0lJWWtuQ3g1UWRrL2t3Q2dZSUtvWkl6ajBFQXdJd0l6RWhNQjhHQTFVRUF3d1kKYXpOekxXTnNhV1Z1ZEMxallVQXhOakk0TVRVek16RTVNQjRYRFRJeE1EZ3dOVEE0TkRnek9Wb1hEVEl5TURndwpOVEE0TkRnek9Wb3dNREVYTUJVR0ExVUVDaE1PYzNsemRHVnRPbTFoYzNSbGNuTXhGVEFUQmdOVkJBTVRESE41CmMzUmxiVHBoWkcxcGJqQlpNQk1HQnlxR1NNNDlBZ0VHQ0NxR1NNNDlBd0VIQTBJQUJCWWlBUUZtZStpNHZiRjUKM3JNK29XNUp2dmliSnJSVFZBcUlPeHpIQjR0cTI0dm02QnpVbVJEbUJDbHdOKythYXdmTW9iRnJkQ25KdnExMgo3RE9sVWlHalNEQkdNQTRHQTFVZER3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUtCZ2dyQmdFRkJRY0RBakFmCkJnTlZIU01FR0RBV2dCVElhT1JUQ21rV29WM21od0ttZXFhempDN1d6REFLQmdncWhrak9QUVFEQWdOSUFEQkYKQWlFQTRwY0x5ZzFFQy85UVhGNk91cGpNRXdIVlhNUFN1R0RPRGRNRDFkY2JzNE1DSUM4ODFkd0pBSXBnSm1BTgpIbmp3UVBDTGFWVUgzYWpLNmNYZEl3czIxM0VRCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0KLS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUJlRENDQVIyZ0F3SUJBZ0lCQURBS0JnZ3Foa2pPUFFRREFqQWpNU0V3SHdZRFZRUUREQmhyTTNNdFkyeHAKWlc1MExXTmhRREUyTWpneE5UTXpNVGt3SGhjTk1qRXdPREExTURnME9ETTVXaGNOTXpFd09EQXpNRGcwT0RNNQpXakFqTVNFd0h3WURWUVFEREJock0zTXRZMnhwWlc1MExXTmhRREUyTWpneE5UTXpNVGt3V1RBVEJnY3Foa2pPClBRSUJCZ2dxaGtqT1BRTUJCd05DQUFRclltNkFFbEdBeGZ2bENiY2hwUkdQbFFPbEZ6LytJUzRZMFV2REhMVE0KQ05WbzdVRnJkL3had2IvVlU5aVFIRWRSMG1ZcVVWL3Z4aWd4YlN0elk5MzFvMEl3UURBT0JnTlZIUThCQWY4RQpCQU1DQXFRd0R3WURWUjBUQVFIL0JBVXdBd0VCL3pBZEJnTlZIUTRFRmdRVXlHamtVd3BwRnFGZDVvY0NwbnFtCnM0d3Uxc3d3Q2dZSUtvWkl6ajBFQXdJRFNRQXdSZ0loQU5ocVgrTEhIOGsrRGlMdXllWEt5N1hpNDg0UWlkeUQKM25KRjhGeEsyL2FzQWlFQXZmQjhIcmk4NWpGVmhScmc2VWQ4cFMyazZjclhUbjYvYVF6MzFuVU4wRm89Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K\n    client-key-data: LS0tLS1CRUdJTiBFQyBQUklWQVRFIEtFWS0tLS0tCk1IY0NBUUVFSUlIN2hsVTBmSkptL0drWis0cnlNQlVpYkFsVnVwSlFHdEF6emVqUlJTczlvQW9HQ0NxR1NNNDkKQXdFSG9VUURRZ0FFRmlJQkFXWjc2TGk5c1huZXN6Nmhia20rK0pzbXRGTlVDb2c3SE1jSGkycmJpK2JvSE5TWgpFT1lFS1hBMzc1cHJCOHloc1d0MEtjbStyWGJzTTZWU0lRPT0KLS0tLS1FTkQgRUMgUFJJVkFURSBLRVktLS0tLQo=\n```\n\nIf you would prefer to get the chain of certificates in PEM format (including\nthe private key, displayed first), you can use the `--print-client-cert` flag:\n\n```\n$ kubectl incluster --print-client-cert\n-----BEGIN EC PRIVATE KEY-----\nMHcCAQEEIIH7hlU0fJJm/GkZ+4ryMBUibAlVupJQGtAzzejRRSs9oAoGCCqGSM49\nAwEHoUQDQgAEFiIBAWZ76Li9sXnesz6hbkm++JsmtFNUCog7HMcHi2rbi+boHNSZ\nEOYEKXA375prB8yhsWt0Kcm+rXbsM6VSIQ==\n-----END EC PRIVATE KEY-----\n-----BEGIN CERTIFICATE-----\nMIIBkTCCATegAwIBAgIIYknCx5Qdk/kwCgYIKoZIzj0EAwIwIzEhMB8GA1UEAwwY\nazNzLWNsaWVudC1jYUAxNjI4MTUzMzE5MB4XDTIxMDgwNTA4NDgzOVoXDTIyMDgw\nNTA4NDgzOVowMDEXMBUGA1UEChMOc3lzdGVtOm1hc3RlcnMxFTATBgNVBAMTDHN5\nc3RlbTphZG1pbjBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABBYiAQFme+i4vbF5\n3rM+oW5JvvibJrRTVAqIOxzHB4tq24vm6BzUmRDmBClwN++aawfMobFrdCnJvq12\n7DOlUiGjSDBGMA4GA1UdDwEB/wQEAwIFoDATBgNVHSUEDDAKBggrBgEFBQcDAjAf\nBgNVHSMEGDAWgBTIaORTCmkWoV3mhwKmeqazjC7WzDAKBggqhkjOPQQDAgNIADBF\nAiEA4pcLyg1EC/9QXF6OupjMEwHVXMPSuGDODdMD1dcbs4MCIC881dwJAIpgJmAN\nHnjwQPCLaVUH3ajK6cXdIws213EQ\n-----END CERTIFICATE-----\n-----BEGIN CERTIFICATE-----\nMIIBeDCCAR2gAwIBAgIBADAKBggqhkjOPQQDAjAjMSEwHwYDVQQDDBhrM3MtY2xp\nZW50LWNhQDE2MjgxNTMzMTkwHhcNMjEwODA1MDg0ODM5WhcNMzEwODAzMDg0ODM5\nWjAjMSEwHwYDVQQDDBhrM3MtY2xpZW50LWNhQDE2MjgxNTMzMTkwWTATBgcqhkjO\nPQIBBggqhkjOPQMBBwNCAAQrYm6AElGAxfvlCbchpRGPlQOlFz/+IS4Y0UvDHLTM\nCNVo7UFrd/xZwb/VU9iQHEdR0mYqUV/vxigxbStzY931o0IwQDAOBgNVHQ8BAf8E\nBAMCAqQwDwYDVR0TAQH/BAUwAwEB/zAdBgNVHQ4EFgQUyGjkUwppFqFd5ocCpnqm\ns4wu1swwCgYIKoZIzj0EAwIDSQAwRgIhANhqX+LHH8k+DiLuyeXKy7Xi484QidyD\n3nJF8FxK2/asAiEAvfB8Hri85jFVhRrg6Ud8pS2k6crXTn6/aQz31nUN0Fo=\n-----END CERTIFICATE-----\n```\n\n## mitmproxy and Telepresence gotchas\n\n- `mitmproxy`, when using the flag `--set client_certs`, needs to be able to\n  read the client certificates file multiple times, which means that using a\n  \"temporary named pipe\":\n\n  ```sh\n  mitmproxy --set client_certs=\u003c(kubectl incluster --print-client-cert)\n  #                           ^^^\n  ```\n\n  Instead, you will have to store the client certs in a temporary file that can be read multiple times:\n\n  ```sh\n  mitmproxy --set client_certs=$(kubectl incluster --print-client-cert \u003e/tmp/client-certs \u0026\u0026 echo /tmp/client-certs)\n  ```\n\n  Note that your Go programs won't have this issue and you can use a temporary named pipe for them.\n\n- If you notice that your Go binary does not seem to take into account the `HTTPS_PROXY=:9090` environment variable,\n  it may be due to your cluster hostname being `localhost` or `127.0.0.1`. I documented this Go limitation in the blog\n  post [What to do when Go ignores HTTP_PROXY for 127.0.0.1](https://maelvls.dev/go-ignores-proxy-localhost/).\n  For example, let us use `kubectl`. Let us imagine that the current Kubernetes context targets a `kind` or `k3d`\n  cluster. Then, you should see:\n\n  ```sh\n  $ kubectl incluster | grep server\n    server: https://127.0.0.1:33203\n  ```\n\n  The `HTTPS_PROXY` variable won't be taken into account:\n\n  ```sh\n  $ HTTPS_PROXY=:9090 kubectl get nodes --kubeconfig=\u003c(kubectl incluster --replace-ca-cert ~/.mitmproxy/mitmproxy-ca-cert.pem)\n  Unable to connect to the server: x509: certificate signed by unknown authority\n  ```\n\n  You will have to use another trick:\n\n  ```sh\n  grep \"^127.0.0.1.*me$\" /etc/hosts || sudo tee -a /etc/hosts \u003c\u003c\u003c\"127.0.0.1 me\"\n  HTTPS_PROXY=:9090 kubectl get nodes --kubeconfig=\u003c(kubectl incluster --replace-ca-cert ~/.mitmproxy/mitmproxy-ca-cert.pem | sed \"s|127.0.0.1|me|\")\n  ```\n\n  This time, kubectl should be using the proxy.\n\n- Trusting your mitmproxy CA cert on Linux:\n\n  ```sh\n  sudo cp ~/.mitmproxy/mitmproxy-ca-cert.pem /usr/share/ca-certificates/mitmproxy/mitmproxy-ca-cert.crt\n  grep mitmproxy/mitmproxy-ca-cert.crt /etc/ca-certificates.conf \\\n    || sudo tee -a /etc/ca-certificates.conf \u003c\u003c\u003cmitmproxy/mitmproxy-ca-cert.crt\n  sudo update-ca-certificates\n  ```\n\n### The `$TELEPRESENCE_ROOT` stays empty on Linux\n\nAs per https://github.com/telepresenceio/telepresence/issues/1944, the workaround is to run:\n\n```sh\nsudo tee -a  /etc/fuse.conf \u003c\u003c\u003cuser_allow_other\n```\n\n## Workaround for Google Kubernetes Engine (GKE)\n\nThe GKE kubeconfig created by `gcloud container cluster get-credentials` doesn't have a token or\na client certificate, which means it won't work with `kubectl incluster`.\n\nThe workaround is to create a \"god mode\" service account so that it can do its thing:\n\n```bash\n$ kubectl apply -f- \u003c\u003cEOF\napiVersion: v1\nkind: ServiceAccount\nmetadata:\n  name: kubectl-incluster\n  namespace: kube-system\n---\napiVersion: rbac.authorization.k8s.io/v1\nkind: ClusterRoleBinding\nmetadata:\n  name: kubectl-incluster\nroleRef:\n  apiGroup: rbac.authorization.k8s.io\n  kind: ClusterRole\n  name: cluster-admin\nsubjects:\n- kind: ServiceAccount\n  name: kubectl-incluster\n  namespace: kube-system\nEOF\n```\n\nThen, for example:\n\n```bash\nexport KUBECONFIG=$(kubectl incluster --sa kube-system/kubectl-incluster)\nkubectl get pods\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaelvls%2Fkubectl-incluster","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmaelvls%2Fkubectl-incluster","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmaelvls%2Fkubectl-incluster/lists"}