{"id":21885028,"url":"https://github.com/clarksource/k8t","last_synced_at":"2025-04-15T07:19:57.371Z","repository":{"id":35169418,"uuid":"152283063","full_name":"ClarkSource/k8t","owner":"ClarkSource","description":"Kubernetes manifest templating for multi cluster and environment deployments with code scaffolding.","archived":false,"fork":false,"pushed_at":"2023-08-10T23:30:52.000Z","size":400,"stargazers_count":16,"open_issues_count":10,"forks_count":4,"subscribers_count":38,"default_branch":"master","last_synced_at":"2025-04-15T07:19:50.780Z","etag":null,"topics":["jinja2","kubernetes","multi-cluster","templates"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"isc","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ClarkSource.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null}},"created_at":"2018-10-09T16:18:53.000Z","updated_at":"2024-04-11T15:02:22.000Z","dependencies_parsed_at":"2023-01-15T15:09:44.130Z","dependency_job_id":null,"html_url":"https://github.com/ClarkSource/k8t","commit_stats":null,"previous_names":[],"tags_count":44,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClarkSource%2Fk8t","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClarkSource%2Fk8t/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClarkSource%2Fk8t/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ClarkSource%2Fk8t/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ClarkSource","download_url":"https://codeload.github.com/ClarkSource/k8t/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":249023792,"owners_count":21199970,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["jinja2","kubernetes","multi-cluster","templates"],"created_at":"2024-11-28T10:18:27.392Z","updated_at":"2025-04-15T07:19:57.313Z","avatar_url":"https://github.com/ClarkSource.png","language":"Python","readme":"# k8t\n\n*Pronounced katie [ˈkeɪti]*\n\n[![CircleCI](https://circleci.com/gh/ClarkSource/k8t/tree/master.svg?style=shield)](https://circleci.com/gh/ClarkSource/k8t/tree/master)\n[![PyPI](https://img.shields.io/pypi/v/k8t?color=blue)](https://pypi.org/project/k8t/)\n[![PyPI - Downloads](https://img.shields.io/pypi/dm/k8t)](https://pypi.org/project/k8t/)\n[![CLARK Open Source](https://img.shields.io/badge/CLARK-Open%20Source-%232B6CDE.svg)](https://www.clark.de/de/jobs)\n\nSimple cluster and environment specific aware templating for kubernetes manifests.\n\n\u003c!-- START doctoc generated TOC please keep comment here to allow auto update --\u003e\n\u003c!-- DON'T EDIT THIS SECTION, INSTEAD RE-RUN doctoc TO UPDATE --\u003e\n**Table of Contents**  *generated with [DocToc](https://github.com/thlorenz/doctoc)*\n\n- [Installation](#installation)\n  - [Docker](#docker)\n  - [Completion](#completion)\n- [Concepts](#concepts)\n  - [*Clusters* and *Environments*](#clusters-and-environments)\n  - [Templating](#templating)\n    - [Template helper functions](#template-helper-functions)\n- [Configuration inheritance](#configuration-inheritance)\n- [Usage](#usage)\n  - [Scaffolding](#scaffolding)\n  - [Config management](#config-management)\n  - [Validate templates](#validate-templates)\n    - [Shortcomings](#shortcomings)\n      - [is defined](#is-defined)\n  - [Generate manifests](#generate-manifests)\n  - [Overriding templates](#overriding-templates)\n  - [Managing secrets](#managing-secrets)\n    - [Providers](#providers)\n      - [SSM](#ssm)\n        - [role assumption](#role-assumption)\n      - [Random](#random)\n      - [Hash](#hash)\n  - [Using as a pre-commit hook](#using-as-a-pre-commit-hook)\n- [TODO](#todo)\n\n\u003c!-- END doctoc generated TOC please keep comment here to allow auto update --\u003e\n\n## Installation\n\nrun this\n\n```bash\n$ pip install --user --upgrade k8t\n```\n\nrun the following to install [ujson](https://pypi.org/project/ujson/) as a dependency\n\n```bash\n$ pip install --user --upgrade k8t[ujson]\n```\n\n**note**: k8t is not Python 2 compatible\n\n### Docker\n\nYou can also run k8t via docker\n\n```bash\n$ docker run clarksource/k8t:latest\n```\n\n**hint**: the docker image comes with [aws-cli](https://aws.amazon.com/cli/), and [kubectl](https://github.com/kubernetes/kubectl).\n\n### Completion\n\nRun the following and store the file in your distribution/OS specific spot\n\nbash:\n\n```bash\n$ _K8T_COMPLETE=source k8t \u003e k8t-completion.sh\n```\n\nzsh:\n\n```zsh\n$ _K8T_COMPLETE=source_zsh k8t \u003e k8t-completion.sh\n```\n\n## Concepts\n\nBy combining those concepts you can quickly add completely new environments to your deployment pipeline just by\nmodifying specializing values and sharing the rest.\n\nCheck out our examples [here](examples/).\n\n### *Clusters* and *Environments*\n\nk8t comes with a builtin framework for *clusters* and *environments* (e.g. production, staging). This came from the need to be able to deploy\nthe same application over multiple clusters and in different environments with completely different setups and values.\nThis idea is helped by the fact that k8t deep-merges values and configs, allowing easy variation through different\nstages of your application deployment.\n\nBoth *clusters* and *environments* are intentionally working the same way and can be used to add another degree of freedom when\ncombined. *Environments* however are also available globally, meaning clusters can share environment specific\nconfiguration while specifying differences in those environments.\n\n### Templating\n\nTemplating is supported via [Jinja](https://jinja.palletsprojects.com). k8t also comes with some additional\n[helper functions](#template-helper-functions) and a [validation function](#validate-templates) with verbose output to\nquickly verify the written templates.\n\n#### Template helper functions\n\n* `random_password(N: int)` - generate a random string of length N\n* `envvar(key: str, [default])` - get a value from any environment variable with optional default\n* `b64encode(value: str)` - encodes a value in base64 (usually required for secrets)\n* `b64decode(value: str)` - decodes a value from base64\n* `hash(value: str, [method: str])` - hashes a given value (default using `sha256`)\n* `get_secret(key: str)` - provides a secret value from a given provider (see [here](#managing-secrets))\n* `bool(value: Any)` - casts value to boolean (\"true\", \"on\", \"yes\", \"1\", 1 are considered as `True`)\n* `sanitize_label(value: str)` - sanitizes label values according to kubernetes spec\n* `sanitize_cpu(value: str | int)` - sanitize cpu value to millicores\n* `sanitize_memory(value: str | int)` - sanitize memory value to megabyte (note: values in scientific notation need to be converted to strings)\n* `standardize_cpu(value: str | int)` - standardize cpu value to millicores (as int)\n* `standardize_memory(value: str | int)` - standardize memory value to megabyte (as int; note: values in scientific notation need to be converted to strings)\n\n## Configuration inheritance\n\nConfiguration, values and templates are used according to the scope they are in. The following snippet shows an example project with low scores (1) and high scores (4) for evaluation order.\n\nSo variables and templates will be overridden from `project` -\u003e `environments` -\u003e `clusters` -\u003e `cluster-environments` resulting in more specific configuration overriding lower values.\n\n```bash\n.                                           (1) # k8t new project .\n├── clusters\n│   ├── foo                                 (3) # k8t new cluster foo\n│   │   ├── config.yaml\n│   │   ├── values.yaml\n│   │   ├── environments\n│   │   │    ├── production                 (4) # k8t new environment production -c foo\n│   │   │    │   ├── config.yaml\n│   │   │    │   └── values.yaml\n│   │   │    └── staging                    (4) # k8t new environment staging -c foo\n│   │   │        ├── config.yaml\n│   │   │        ├── values.yaml\n│   │   │        └── templates\n│   │   │           └── deployment.yaml.j2  (4) # k8t new template deployment -c foo -e staging\n│   │   └── templates\n│   │      └── deployment.yaml.j2           (3) # k8t new template deployment -c foo\n│   └── bar                                 (3) # k8t new cluster bar\n│       ├── config.yaml\n│       └── values.yaml\n├── environments\n│   ├── production                          (2) # k8t new environment production\n│   │   ├── config.yaml\n│   │   └── values.yaml\n│   └── staging                             (2) # k8t new environment staging\n│       ├── config.yaml\n│       └── values.yaml\n├── config.yaml                             (1)\n└── values.yaml                             (1)\n\n```\n\n## Usage\n\n### Scaffolding\n\nCreate a new project folder with a cluster directory and an empty defaults file\n\n```bash\n$ k8t new project .\n```\n\nCreate a new cluster\n\n```bash\n$ k8t new cluster MyCluster\n```\n\nCreate a new global environment\n\n```bash\n$ k8t new environment staging\n```\n\nAnd a new cluster environment\n\n```bash\nk8t new environment staging -c MyCluster\n```\n\nGenerate a new deployment template for cluster MyCluster (for a list of available templates see the `k8t new template --help`)\n\n```bash\n$ k8t new template deployment -c MyCluster -e staging\n```\n\n### Config management\n\nTo ease file access a little bit k8t can open config and value files in your `$EDITOR` or fallback to a sensible\ndefault.\n\n```bash\n$ k8t edit values --environment staging\n```\n\n```bash\n$ k8t edit config --cluster MyCluster\n```\n\n### Validate templates\n\nWhile validation is done before generating, templates can be validated for environment files easily.\n\n```bash\n$ k8t validate\n```\n\nTo validate for clusters/environments the usual options can be used\n\n```bash\n$ k8t validate -c MyCluster -e production\n```\n\n#### Shortcomings\n\nThe validation is currently not a 100% correct and can miss certain edge cases.  If you notice any other issues please let us know.\n\n##### is defined\n\nThe following will result in a false negative for `foobar` being defined\n\n```\n{{ foobar }}\n\n{% if foobar is defined %}\n{{ foobar }}\n{% endif %}\n```\n\nTo avoid this make sure that the `is defined` test is applied to all instances of the variable.\n\nThe following may result in a false positive for `bar` being undefined\n\n```\n{% if foobar is defined %}\n{{ bar }}\n{% endif %}\n```\n\n### Generate manifests\n\nThe **--cluster** flag will load variables from a directory. By default the file **default.yaml** in that directory will be\nloaded, however an environment can be specified with **--environment**.\n\n```bash\n$ k8t gen -c MyCluster -e staging\n```\n\nAdditionally k8t will attempt to load a file **defaults.yaml** in the root directory. This way a set of default\nvariables can be specified and selectively overriden via cluster and environment.\n\nAdditional values can be given via flag **--value-file** in the form of a file or **--value KEY VALUE**, both can be\nsupplied multiple times.\n\nVariables will be merged via deep merging. Default merge strategy is left-to-right.\n\n### Overriding templates\n\nTemplates can be overriden on a cluster/environment level.\n\nIf a file `application.yaml` exists in the root templates folder, simply add a file with the same name to the\ncluster/environment template folder.\n\n### Managing secrets\n\nSecrets can be interpolated with the helper function `get_secret`. It requires a key as first argument and providers\nare configurable by environment/cluster.\n\n```yaml\nfoobar: \"{{ get_secret('/my-key') }}\"\n```\n\n#### Providers\n\n##### SSM\n\nSetup secrets on SSM\n\n```yaml\nsecrets:\n  provider: ssm\n  region: \"eu-central-1\"\n  prefix: \"/foobar\"\n```\n\n\u003e Keep in mind that SSM parameter names can be formed as a path and  they can only consist of sub-paths divided by slash symbol; each sub-path can be formed as a mix of letters, numbers and the following 3 symbols: `.-_`\n\u003e\n\u003e Be careful to follow this format when setting up the provider `prefix` and `get_secret(key)`.\n\nGlobal secrets config can be overridden in `get_secret` helper function call by specifying `config_override` argument.\n\n```yaml\nfoobar: \"{{ get_secret('/my-key', config_override={'prefix': '/dev'}) }}\"\n```\n\n###### role assumption\n\nYou can optionally assume an IAM role to retrieve secrets by specyfing `role_arn` in the config:\n\n```\nsecrets:\n  provider: ssm\n  region: \"eu-central-1\"\n  role_arn: \"arn:aws:iam::account:role/role-name-with-path\"\n```\n\n##### Random\n\nRandom secrets can be generated easily by using the random provider. This provider uses a global dictionary to store\nresults for the time of the run in python so keys should always produce the same result.\n\n```yaml\nsecrets:\n  provider: random\n```\n\n##### Hash\n\nIn case consistent (fake) secrets are needed, the `hash` provider can be used that hashes the secret key for the value.\n\n```yaml\nsecrets:\n  provider: hash\n```\n### Using as a pre-commit hook\n\nYou can also use this repo as a https://github.com/pre-commit/pre-commit hook\n\nAdd this to your `.pre-commit-config.yaml`:\n\n```yaml\n-   repo: https://github.com/ClarkSource/k8t\n    rev: ''  # Use the sha / tag you want to point at\n    hooks:\n    -   id: k8t-validate\n        # args: [ -e dev -c us-west-2 ]\n    -   id: k8t-generate\n        name: k8t(dev)\n        args: [ -o dev.yaml -e dev ]\n```\n\n## TODO\n\n* testing needs to be expanded\n* the ability to add additional template directories via the CLI\n* validation functions for template values (e.g. memory/cpu values)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclarksource%2Fk8t","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fclarksource%2Fk8t","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fclarksource%2Fk8t/lists"}