{"id":13580025,"url":"https://github.com/torchbox/kdtool","last_synced_at":"2025-04-06T00:30:38.407Z","repository":{"id":66129421,"uuid":"95397612","full_name":"torchbox/kdtool","owner":"torchbox","description":"Kubernetes deployment utility","archived":true,"fork":false,"pushed_at":"2019-04-29T15:01:14.000Z","size":193,"stargazers_count":42,"open_issues_count":2,"forks_count":4,"subscribers_count":8,"default_branch":"master","last_synced_at":"2024-11-05T18:51:37.196Z","etag":null,"topics":["ci","docker","gitlab","gitlab-ci","kubernetes"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/torchbox.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2017-06-26T01:45:23.000Z","updated_at":"2024-06-17T01:07:54.000Z","dependencies_parsed_at":"2023-02-21T20:31:33.779Z","dependency_job_id":null,"html_url":"https://github.com/torchbox/kdtool","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/torchbox%2Fkdtool","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/torchbox%2Fkdtool/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/torchbox%2Fkdtool/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/torchbox%2Fkdtool/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/torchbox","download_url":"https://codeload.github.com/torchbox/kdtool/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247419597,"owners_count":20936009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ci","docker","gitlab","gitlab-ci","kubernetes"],"created_at":"2024-08-01T15:01:46.230Z","updated_at":"2025-04-06T00:30:38.057Z","avatar_url":"https://github.com/torchbox.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# DEPRECATED\nThis tool is no longer supported or expected to work.\n\n# Kubernetes deployment tool\n\nNote: Prior to version 1.8.0-1, kdtool was known as gitlab-kube-deploy.  It has\nbeen renamed as its scope has expanded and it's no longer specific to GitLab.\n\nkdtool is a utility for deploying applications and interacting with deployed\napplications on Kubernetes, with an emphasis on deploying from GitLab and other\nCI applications.  It can be installed and used standalone from the command\nline, or as a container image in Docker-based CI workflows.\n\nkdtool can deploy simple applications without needing a manifest.  Based on\ncommand-line options, it can provision Deployments, Service and Ingresses,\nand configure environment variables (via Secrets or ConfigMaps), persistent\nvolumes and databases (with an\n[external database controller](https://github.com/torchbox/k8s-database-controller/)),\nas well as ACME/Let's Encrypt TLS certificates (with\n[kube-lego](https://github.com/jetstack/kube-lego)) and HTTP authentication.\n\nFor more complicated applications, kdtool can simplify YAML (or JSON)\nmanifests, providing environment-based template variable substitution to deploy\nseveral different copies of an application (for example, a staging site and any\nnumber of review apps) from a single manifest.  When deploying from CI, you can\nsubstitute `$IMAGE` in the manifest to easily deploy the image that was built.\n\nEven if you don't use kdtool to deploy your application, its `shell` command\nmakes it easier to interact with running applications, e.g. to debug problems,\ncopy files from to or from the application, or connect to the application's\ndatabase.  kdtool can start a shell (or any other command) using the same\nimage, environment variables, volume mounts, and other configuration as any\nexisting deployment, using a single command.\n\nkdtool's `status` command provides an overview of a Deployment, its ReplicaSets\nand their pods, to make problems with deployments easier to diagnose without\nhaving to examine each pod or ReplicaSet individually.\n\n## Screenshot\n\nUse it from GitLab CI:\n\n![A screenshot of a Gitlab CI build showing deploy being used](deploy.png)\n\nOr from the command line:\n\n```\n% kdtool deploy -H testapp.example.com --port=8080 gcr.io/google_containers/echoserver:1.4 testapp\ndeployment \"testapp\" created\nservice \"testapp\" created\ningress \"testapp\" created\n% curl -sSi http://testapp.example.com\nHTTP/1.1 200 OK\n...\n```\n\n## Installation\n\nDownload `kdtool.pyz` from the latest release and copy it to a convenient\nlocation, such as `/usr/local/bin/kdtool`.  This is a Python zipapp and requires\nPython 3.4 or later to run.\n\nTo use `kdtool deploy`, you must have `kubectl` installed.  If you have an\nexisting kubeconfig file (e.g. `$HOME/.kube/config`, or specified in\n`$KUBECONFIG`), kdtool will take configuration from there by default.\n\nOr, build from source:\n\n```\n% make dist\n% ./kdtool.pyz -h\n```\n\nOr, run from the Docker image:\n\n```\n% docker run --rm -ti torchbox/kdtool:latest -h\n```\n\nYou don't need to install anything to use kdtool from GitLab CI; just specify\nthe Docker image in the job.\n\n## Using with GitLab CI\n\nSince GitLab 9.4, no additional configuration is required for use with GitLab.\nUse the Docker image `torchbox/kdtool:latest` in your CI job and invoke\n`kdtool deploy ...` as normal.\n\nPrior to GitLab 9.4, you must use `kdtool --gitlab deploy ...` to pick up the\nauthentication configuration.\n\nEither way, kdtool requires that you have\n[set up Kubernetes integration](https://docs.gitlab.com/ce/user/project/integrations/kubernetes.html)\nfor your GitLab project.\n\n## General options\n\nThese options affect the overall behaviour of kdtool and how it connects to\nyour Kubernetes cluster.  If you already have a working kubeconfig (e.g. for\nkubectl), you won't normally need any of these options except `--namespace`.\n\n* `-n ns, --namespace=ns`: Set the Kubernetes namespace to operate in. Default:\n  `default`.\n* `-c ctx, --context=ctx`: Set the cluster context to use; the context must\n  exist in the loaded kubeconfig file.\n* `-K path, --kubectl=path`: Specify the location of `kubectl` (default:\n  autodetect).\n* `-G, --gitlab`: Take Kubernetes cluster details from GitLab environment\n variables.  (Replaces `--namespace`, `--server`, `--ca-certificate` and\n `--token`; unnecessary since GitLab 9.4.)\n* `-S url, --server=url`: Set the URL of the Kubernetes API server (default:\n  `http://localhost:8080`).\n* `-T token, --token=token`: Set Kubernetes authentication token.  This can be a\n  user token, a serviceaccount JWT token or whatever.  No default.\n* `-C path, --ca-certificate=path`: Set Kubernetes API server CA certificate.\n  No default.\n\nIf you're running in-cluster and want to authenticate with the pod service\naccount credentials, do not specify any authentication options; kubectl will\npick up the service account details from the pod.\n\n## Simple applications\n\nYou can use `kdtool deploy` to deploy a simple application without creating a\nmanifest.  To deploy the image \"myapp:latest\" on \"www.mysite.com\":\n\n```\nkdtool deploy --hostname=www.mysite.com myapp:latest myapp\n```\n\nIn a GitLab CI job, you can take the image and deployment name from environment\nvariables:\n\n```\nkdtool deploy --hostname=www.mysite.com $CI_REGISTRY_IMAGE:$CI_BUILD_REF $CI_ENVIRONMENT_SLUG\n```\n\nThis will create a Deployment, Service and Ingress in the Kubernete namespace\nconfigured in Gitlab.\n\n### Undeploying\n\nUse `kdtool undeploy \u003cname\u003e` to delete a deployment.  This will list the\nresource(s) that will be deleted and prompt for confirmation.\n\nTo avoid the confirmation prompt, use `undeploy -f`.\n\nTo delete all resources associated with the deployment, such as ingresses,\ndatabases and volumes, use `undeploy -A`.  The list of resources to delete is\ntaken from an annotation added to the deployment at creation time, so you don't\nneed to tell kdtool what to delete.\n\nOlder versions supported a different undeploy command, `kdtool deploy --undeploy`.\nThis is obsolete and should not be used.\n\n### Application options\n\nThese options to `kdtool deploy` control how the application will be deployed.\n\n* `-H HOST, --hostname=HOST`: create an Ingress resource to route requets for\n  the given hostname to the application.  Providing a URL will also work, but\n  everything except the hostname will be ignored.  May be specified multiple\n  times.\n* `-p PORT, --port=PORT`: Set the port the application container listens on\n  (default 80).  If your application doesn't listen on port 80, you must\n  specify this for `--hostname` to work.\n* `-A, --acme`: Add annotations to the created Ingress to tell\n  [kube-lego](https://github.com/jetstack/kube-lego) to issue a TLS certificate.\n* `-r N, --replicas=N`: Create N replicas of the application.\n* `-P policy, --image-pull-policy=policy`: Set the Kubernete images pull\n  policy to `IfNotPresent` or `Always`.  `Always` is only required if you push\n  new versions without updating the image tag, which usually should not be the\n  case with CI builds.\n* `-e VAR[=VALUE], --env=VAR[=VALUE]`: Set the given environment variable in the\n  applications's environment.  If no value is specified, it will be taken from\n  the current environment.\n* `-s VAR=VALUE, --secret=VAR=VALUE`: Set the given environment variable in\n  application's environment using a Kubernetes Secret.\n* `--memory-request`: Set Kubernetes memory request.\n* `--memory-limit`: Set Kubernetes memory limit.\n* `--cpu-request`: Set Kubernetes CPU request.\n* `--cpu-limit`: Set Kubernetes CPU limit.\n* `--strategy=TYPE`: set Deployment\n  [update strategy](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#strategy).\n  `rollingupdate` will replace each replica one at a time, enabling\n  zero-downtime deployments.  `recreate` will delete all pods, then create new\n  pods to replace them; this will cause downtime during the deployment.  The\n  default is `rollingupdate`.\n\n### Service options\n\n* `-v NAME:PATH, --volume=NAME:PATH`: Create a Persistent Volume Claim called\n  `NAME` and mount it at `PATH`.  For this to work, your cluster must have a\n  functional PVC provisioner.   Currently this always requests a `ReadWriteMany`\n  volume, which means it does work with `--replicas` but does not work with GCE\n  or AWS volumes.  (It does work with NFS, CephFS, GlusterFS, etc.)\n* `--database=TYPE`: Attach a persistent database of the given type, which\n  should be `mysql` or `postgresql`.  This requires the Torchbox\n  [database controller](https://github.com/torchbox/k8s-database-controller).\n  Database connection details will be placed in `$DATABASE_URL`.\n* `--postgres=VERSION` (e.g. `--postgres=9.6`; EXPERIMENTAL, UNTESTED):\n  Deploy a PostgreSQL sidecar container alongside the application and configure\n  `$DATABASE_URL` with the access details.  The PostgreSQL data is stored in a\n  PVC, so this requires that your cluster has a functional PVC provisioner.  The\n  PVC will be deleted when the application is undeployed.  This is intended for\n  review apps, not production sites.  **This will not work with --replicas**.\n* `--redis=MEMORY` (e.g. `--redis=64m`; EXPERIMENTAL, UNTESTED): Deploy a Redis\n  sidecar container alongside the application and set `$CACHE_URL` to its\n  location.  Data stored in Redis is not persisted and the container is deleted\n  when the application is undeployed.  This is intended for review apps, not\n  production sites.  If used with `--replicas`, every replica will get its own\n  Redis instance, which is probably not what you want.\n\n### HTTP authentication options\n\n* `--htauth-user=USERNAME:PASSWORD`: Require HTTP basic authentication using\n  this username and plaintext password.  This may be specified multiple times.\n* `--htauth-address=1.2.3.0/24`: Reject requests from outside this IP range.\n  May be specified multiple times.\n* `--htauth-satisfy=\u003cany|all\u003e`: Control behaviour when both `--htauth-user` and\n  `--htauth-address` are specified.  If `all` (default), a valid password _and_\n  a whitelisted IP address are required or the connection will be rejected.  If\n  `any`, either is sufficient for access.\n\nSupport for HTTP authentication varies greatly among Kubernetes Ingress\ncontrollers.  As far as I know, the GKE/GCE Ingress controller doesn't support\nit at all.  The nginx controller supports all the options except\n`--htauth-satisfy`.  The only controller that supports `--htauth-satisfy` is\n[Traffic Server](https://github.com/torchbox/k8s-ts-ingress).\n\nThis authentication is not intended to be secure: it accepts passwords in\nplaintext and hashes them using FreeBSD MD5.  It's intended to prevent search\nengines and curious users from finding your staging sites, not to replace proper\napplication-level authentication.\n\n### Example .gitlab-ci.yml\n\nUse Gitlab dynamic environments to deploy any branch at\n`https://\u003cbranchname\u003e.myapp-staging.com`, except for `master` which is\ndeployed at `https://www.myapp.com/` with two replicas:\n\n```\n---\nimage: docker:latest\nvariables:\n  IMAGE_TAG: $CI_REGISTRY_IMAGE:$CI_BUILD_REF\n\nstages:\n- build\n- deploy\n\nbuild:\n  stage: build\n  before_script:\n  - docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY\n  script:\n  - docker build -t $IMAGE_TAG .\n  - docker push $IMAGE_TAG\n\ndeploy_production:\n  stage: deploy\n  only:\n  - master\n  environment:\n    name: $CI_BUILD_REF_NAME\n    url: https://www.myapp.com\n  image: torchbox/kdtool:latest\n  script:\n  - kdtool deploy -r2 -A -H www.myapp.com $IMAGE_TAG $CI_ENVIRONMENT_SLUG\n\ndeploy_review:\n  stage: deploy\n  only:\n  - branches\n  except:\n  - master\n  environment:\n    name: $CI_BUILD_REF_NAME\n    url: https://$CI_ENVIRONMENT_SLUG.myapp-staging.com\n    on_stop: undeploy_review\n  image: torchbox/gitlab-kube-deploy:latest\n  script:\n  - kdtool deploy -A -H $CI_ENVIRONMENT_URL $IMAGE_TAG $CI_ENVIRONMENT_SLUG\n\nundeploy_review:\n  stage: deploy\n  when: manual\n\n  environment:\n    name: $CI_BUILD_REF_NAME\n    action: stop\n\n  image: torchbox/gitlab-kube-deploy:latest\n\n  script:\n  - deploy -G --undeploy -A -H $CI_ENVIRONMENT_URL $IMAGE_TAG $CI_ENVIRONMENT_SLUG\n```\n\n## Custom manifests\n\nkdtool's automatic manifest generation isn't intended to cover every possible\nuse case.  If you like, you can provide your own manifest; kdtool will do\nvariable substitution inside the manifest.\n\nSpecifically, any string `$varname` or `${varname}` in the manifest will be\nreplaced with the corresponding environment variable.  This includes variables\ndefined in `.gitlab-ci.yml`, like `$IMAGE`, and any variables defined as Gitlab\nPipeline secrets.\n\nA variable of the form `${varname:b64encode}` will be Base64-encoded, which is\nuseful for populating Kubernetes Secrets.\n\nHere is an example manifest that assumes `DATABASE_URL` and `SECRET_KEY` have\nbeen set as Gitlab secrets:\n\n```\n---\n\napiVersion: v1\nkind: Secret\nmetadata:\n  namespace: $KUBE_NAMESPACE\n  name: $CI_ENVIRONMENT_SLUG\ntype: Opaque\ndata:\n  DATABASE_URL: ${DATABASE_URL:b64encode}\n  SECRET_KEY: ${SECRET_KEY:b64encode}\n\n---\n\napiVersion: extensions/v1beta1\nkind: Deployment\nmetadata:\n  namespace: $KUBE_NAMESPACE\n  name: $CI_ENVIRONMENT_SLUG\nspec:\n  replicas: 1\n  strategy:\n    type: Recreate\n  selector:\n    matchLabels:\n      app: $CI_ENVIRONMENT_SLUG\n  template:\n    metadata:\n      labels:\n        app: $CI_ENVIRONMENT_SLUG\n    spec:\n      containers:\n      - name: app\n        image: $IMAGE\n        envFrom:\n        - secretRef:\n            name: $CI_ENVIRONMENT_SLUG\n        ports:\n        - name: http\n          containerPort: 80\n          protocol: TCP\n\n---\n\napiVersion: v1\nkind: Service\nmetadata:\n  name: $CI_ENVIRONMENT_SLUG\n  namespace: $KUBE_NAMESPACE\nspec:\n  type: ClusterIP\n  ports:\n  - name: http\n    port: 80\n    protocol: TCP\n    targetPort: http\n  selector:\n    app: $CI_ENVIRONMENT_SLUG\n\n---\n\napiVersion: extensions/v1beta1\nkind: Ingress\nmetadata:\n  annotations:\n    kubernetes.io/tls-acme: \"true\"\n  name: $CI_ENVIRONMENT_SLUG\n  namespace: $KUBE_NAMESPACE\nspec:\n  rules:\n  - host: www.myapp.com\n    http:\n      paths:\n      - backend:\n          serviceName: $CI_ENVIRONMENT_SLUG\n          servicePort: http\n  tls:\n  - hosts:\n    - www.myapp.com\n    secretName: ${CI_ENVIRONMENT_SLUG}-tls\n```\n\nYou could use this manifest in `.gitlab-ci.yml` like this:\n\n```\ndeploy_production:\n  stage: deploy\n  only:\n  - master\n  environment:\n    name: $CI_BUILD_REF_NAME\n    url: https://www.myapp.com\n  image: torchbox/gitlab-kube-deploy:latest\n  script:\n  - kdtool deploy --manifest=deployment.yaml $IMAGE_TAG $CI_ENVIRONMENT_SLUG\n```\n\n## Shell mode\n\nUse `kdtool shell` to start a shell for a deployment:\n\n```\nkdtool shell myapp\n```\n\nThe argument should be the name of the deployment.  Environment variables and\nvolume mounts will be configured from the deployment, so the application's data\nand configuration will be available in the shell.\n\nBy default, the application's image will be used for the shell.  To use a\ndifferent image, use `-i` / `--image`:\n\n```\nkdtool shell -i fedora:latest myapp\n```\n\nTo run a different shell, use `-c` / `--command`:\n\n```\nkdtool shell -c /bin/zsh myapp\n```\n\nTo run a non-interactive command, use `kdtool exec`:\n\n```\nkdtool exec myapp pg_dump '$(DATABASE_URL)'\n```\n\n## Status\n\nUse `kdtool status` to show the status for a deployment.  kdtool will attempt\nto detect errors and include them in the output:\n\n```\n% kdtool status testapp\ndeployment testapp: 1 replica(s), current generation 5\n  2 active replica sets (* = current, ! = error):\n    *!testapp-54d6fdb796: generation 5, 1 replicas configured, 0 ready\n        pod testapp-54d6fdb796-94pck: Pending\n          ImagePullBackOff: Back-off pulling image \"torchbox/invalid-image:latest\"\n      testapp-755c4c48f: generation 4, 1 replicas configured, 1 ready\n        pod testapp-755c4c48f-lxn57: Running\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftorchbox%2Fkdtool","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftorchbox%2Fkdtool","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftorchbox%2Fkdtool/lists"}