{"id":22797326,"url":"https://github.com/jameslikeslinux/kubecm","last_synced_at":"2025-03-30T18:29:08.671Z","repository":{"id":267699736,"uuid":"902067071","full_name":"jameslikeslinux/kubecm","owner":"jameslikeslinux","description":"Bring Kubernetes resources under control and deploy anything your way with Puppet","archived":false,"fork":false,"pushed_at":"2024-12-16T19:42:14.000Z","size":40,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-25T21:12:35.031Z","etag":null,"topics":["bolt","helm","kubernetes","kustomize","puppet"],"latest_commit_sha":null,"homepage":"https://forge.puppet.com/modules/jtl/kubecm","language":"Ruby","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/jameslikeslinux.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-12-11T20:58:05.000Z","updated_at":"2024-12-16T19:46:10.000Z","dependencies_parsed_at":"2025-02-05T20:51:59.257Z","dependency_job_id":"68bd04aa-e592-44a7-8caa-c6ea35bb6d01","html_url":"https://github.com/jameslikeslinux/kubecm","commit_stats":null,"previous_names":["jameslikeslinux/puppet-kubecm","jameslikeslinux/kubecm"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jameslikeslinux%2Fkubecm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jameslikeslinux%2Fkubecm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jameslikeslinux%2Fkubecm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/jameslikeslinux%2Fkubecm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/jameslikeslinux","download_url":"https://codeload.github.com/jameslikeslinux/kubecm/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246117738,"owners_count":20726069,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bolt","helm","kubernetes","kustomize","puppet"],"created_at":"2024-12-12T06:05:42.596Z","updated_at":"2025-03-30T18:29:08.640Z","avatar_url":"https://github.com/jameslikeslinux.png","language":"Ruby","readme":"# KubeCM: Kubernetes Configuration Management\n\n[![pipeline status](https://gitlab.james.tl/james/kubecm/badges/main/pipeline.svg)](https://gitlab.james.tl/james/kubecm/-/commits/main)\n\n- [Overview](#overview)\n- [Features](#features)\n- [Motivation](#motivation)\n- [Getting Started](#getting-started)\n  - [Prerequisites](#prerequisites)\n  - [Installation](#installation)\n- [Usage](#usage)\n  - [Deploying Applications](#deploying-applications)\n  - [Configuring Deployments](#configuring-deployments)\n- [Custom Resources, Values, and Patches](#custom-resources-values-and-patches)\n- [YAML Plans](#yaml-plans)\n  - [Example 1: Deploying Vaultwarden](#example-1-deploying-vaultwarden)\n  - [Example 2: Deploying WordPress](#example-2-deploying-wordpress)\n- [Troubleshooting](#troubleshooting)\n  - [Using the `render_to` Parameter](#using-the-render_to-parameter)\n  - [Exploring the Build Directory](#exploring-the-build-directory)\n- [Contributing](#contributing)\n\n\n## Overview\n\n**KubeCM (Kubernetes Configuration Management)** empowers you to control and\ndeploy Kubernetes resources your way, combining the strengths of Puppet, Helm,\nand Kustomize. With KubeCM, you can manage, customize, and configure Kubernetes\ndeployments using a hierarchical data model and YAML plans that provide clarity\nand reusability.\n\n## Features\n\n- **Unified Management**: Deploy Helm charts, custom resources, and Kustomize\n  patches under one plan.\n- **Hierarchical Configuration**: Use Hiera to store, merge, and override\n  configurations at multiple levels.\n- **Secure Secrets**: Encrypt sensitive data using Hiera-EYAML.\n- **Custom Resource Deployment**: Deploy any Kubernetes resource directly\n  without requiring a Helm chart.\n- **Integrated Orchestration**: Bolt-powered orchestration for on-demand or\n  scheduled deployments.\n- **Cloud Transition**: Leverage existing Puppet data and automation to ease\n  migrations to Kubernetes.\n\n## Motivation\n\nKubernetes excels at managing clusters, but it struggles with managing entire\ndeployments as cohesive units. Helm addresses this gap with templated \"charts,\"\nbut real-world applications often require configuration changes that Helm\ntemplates do not expose. KubeCM steps in to bridge this gap by leveraging\nPuppet's powerful configuration management features (like Hiera and Bolt) to\nmanage and customize Helm, Kustomize, and raw Kubernetes resources as a unified\nwhole.\n\n## Getting Started\n\n### Prerequisites\n\nEnsure the following tools are installed and configured on a Linux system:\n\n- **kubectl**: Kubernetes CLI for managing clusters.\n- **helm**: Kubernetes package manager.\n- **bolt**: Puppet orchestration tool.\n- **hiera-eyaml**: Optional, for encrypted secrets.\n\n### Installation\n\n1. **Create a Bolt project**:\n\n   ```bash\n   mkdir acme_inc\n   cd acme_inc\n   bolt project init\n   ```\n\n2. **Install the module**:\n\n   ```bash\n   bolt module add jtl-kubecm\n   ```\n\n### Usage\n\n#### Deploying Applications\n\nThe following parameters in the `kubecm::deploy` plan specify key aspects of a\ndeployment:\n\n- **release**: The unique name of the deployment within the Kubernetes\n  namespace. Each deployment should have a distinct release name.\n- **chart**: The name of the Helm chart to be used for the deployment.\n- **chart\\_source**: The location of the Helm chart from a repo, a local\n  directory, or an OCI registry reference. This tells Helm where to retrieve\n  the chart. It can also be undefined to deploy without a chart, using only\n  resources from Hiera.\n- **repo_url**: If using a repo reference in `chart_source` (e.g.\n  `repo_name/chart_name`), the HTTP location of the Helm repository.\n\nUse the `kubecm::deploy` plan to install applications via Helm charts.\n\nExample:\n\n```bash\nbolt plan run kubecm::deploy release=shop chart=wordpress chart_source=oci://registry-1.docker.io/bitnamicharts/wordpress\nbolt plan run kubecm::deploy release=blog chart=wordpress chart_source=oci://registry-1.docker.io/bitnamicharts/wordpress\nbolt plan run kubecm::deploy release=vaultwarden chart_source=guerzon/vaultwarden repo_url=https://guerzon.github.io/vaultwarden\n```\n\nThis deploys two WordPress instances (\"shop\" and \"blog\") and a Vaultwarden\ninstance.\n\n#### Configuring Deployments\n\nConfiguration is managed using Hiera's hierarchical structure. Create a\n`hiera.yaml` in the project directory:\n\n```yaml\n---\nversion: 5\n\ndefaults:\n  datadir: data\n  data_hash: yaml_data\n\nhierarchy:\n  - name: 'Releases'\n    paths:\n      - \"release/%{kubecm::deploy::release}/%{kubecm::deploy::namespace}.yaml\"\n      - \"release/%{kubecm::deploy::release}.yaml\"\n  - name: 'Charts'\n    paths:\n      - \"chart/%{kubecm::deploy::chart}/%{kubecm::deploy::namespace}.yaml\"\n      - \"chart/%{kubecm::deploy::chart}.yaml\"\n  - name: 'Common'\n    paths:\n      - 'common.yaml'\n```\n\nTo provide additional customization, you can override KubeCM's default key\nnames for various deployment settings. These keys include:\n\n- **include\\_key**: Used to define additional Puppet classes to be included.\n- **resources\\_key**: Used to define Kubernetes resources to be created as part of the deployment.\n- **values\\_key**: Used to provide Helm chart values for the deployment.\n- **patches\\_key**: Used to define Kustomize patches to apply to resources.\n\nThese key names can be set in `data/common.yaml` to override the defaults as\nshown below:\n\n```yaml\n---\nkubecm::deploy::include_key: 'include'\nkubecm::deploy::resources_key: 'resources'\nkubecm::deploy::values_key: 'values'\nkubecm::deploy::patches_key: 'patches'\n```\n\nThis configuration allows for a more customized, DSL-like experience when\nmanaging Kubernetes deployments with KubeCM. You can modify these key names to\navoid conflicts with other existing keys in your control repository.\n\n## Custom Resources, Values, and Patches\n\n**Values Examples**:\n\n**Memory Resource Parameterization**:\n\nThis example demonstrates how to set default memory requests for Helm\ndeployments and override them at a namespace-specific level.\n\n`data/chart/wordpress.yaml`\n\n```yaml\n---\nmemory: 200Mi\n\nvalues:\n  resources:\n    requests:\n      memory: \"%{lookup('memory')}\"\n```\n\n`data/chart/wordpress/test.yaml`\n\n```yaml\n---\nmemory: 100Mi\n```\n\n**Ingress Hostname Parameterization**:\n\nThis example shows how to set a default ingress hostname pattern for all\nreleases and override it for specific releases.\n\n`data/chart/wordpress.yaml`\n\n```yaml\n---\ningress_hostname: \"%{kubecm::deploy::release}.example.com\"\n\nvalues:\n  ingress:\n    enabled: true\n    hostname: \"%{lookup('ingress_hostname')}\"\n```\n\n`data/release/shop.yaml`\n\n```yaml\n---\ningress_hostname: 'store.example.com'\n```\n\n**Vaultwarden Admin Token Hash Calculation**:\n\nThis example demonstrates how to hash the Vaultwarden admin token dynamically\nand make it available as a Helm chart value.\n\n`data/chart/vaultwarden.yaml`\n\n```yaml\n---\ninclude:\n  - 'acme_inc::vaultwarden'\n\nadmin_token: supersecuretoken # encrypt this with hiera-eyaml!\n\nvalues:\n  adminToken:\n    value: \"%{acme_inc::vaultwarden::admin_token_hash}\"\n```\n\n`manifests/vaultwarden.pp`\n\n```puppet\nclass acme_inc::vaultwarden {\n  $admin_token = lookup('admin_token')\n  $admin_token_hash = generate(\n    '/bin/sh',\n    '-c',\n    \"echo -n ${admin_token.shellquote} | argon2 `openssl rand -base64 32` -e -id -k 65540 -t 3 -p 4\",\n  ).chomp\n}\n```\n\n**Registry Auths Example**:\n\nThis example demonstrates how to manage Docker registry authentication secrets\nto be used by Kubernetes deployments.\n\n`data/common.yaml`\n\n```yaml\n---\ninclude:\n  - 'acme_inc'\n\nregistry_tokens:\n  registry.example.com: fake-token # encrypt this with hiera-eyaml!\n\nresources:\n  registry_auths:\n    - apiVersion: v1\n      kind: Secret\n      metadata:\n        name: \"%{kubecm::deploy::service}-registry-auths\"\n        namespace: \"%{kubecm::deploy::namespace}\"\n      data:\n        .dockerconfigjson: \"%{acme_inc::registry_auths_base64}\"\n      type: kubernetes.io/dockerconfigjson\n```\n\n`manifests/init.pp`\n\n```puppet\nclass acme_inc {\n  $registry_tokens = lookup('registry_tokens')\n  $registry_auths_base64 = base64('encode', stdlib::to_json({\n    'auths' =\u003e $registry_tokens.reduce({}) |$result, $token| {\n      $result + { $token[0] =\u003e { 'auth' =\u003e base64('encode', $token[1]).chomp } }\n    },\n  }))\n}\n```\n\n**Patch Example**\n\nThis example demonstrates how to use a Kustomize patch to modify Kubernetes\nresources in place. Patches like this are useful when you need to modify\nresource properties that aren't exposed as chart values, such as node affinity,\ntolerations, or annotations. Patching can be especially helpful when dealing\nwith third-party charts where direct configuration is limited:\n\n`data/chart/vaultwarden.yaml`\n\n```yaml\n---\nnode_role: special\n\npatches:\n  10-placement:\n    patch:\n      kind: not-important\n      metadata:\n        name: not-important\n      spec:\n        template:\n          spec:\n            affinity:\n              nodeAffinity:\n                requiredDuringSchedulingIgnoredDuringExecution:\n                  nodeSelectorTerms:\n                    - matchExpressions:\n                      - key: \"node-role.kubernetes.io/%{lookup('node_role')}\"\n                        operator: Exists\n            tolerations:\n              - key: \"node-role.kubernetes.io/%{lookup('node_role')}\"\n                operator: Exists\n                effect: NoSchedule\n    targets:\n      kind: (Deployment|StatefulSet|Job)\n```\n\n## YAML Plans\n\nYAML plans provide a simple way to define deployment workflows using YAML\nsyntax. They enable you to automate the execution of multiple tasks, making it\neasier to manage and update Kubernetes resources in a consistent manner.\n\nYAML plans are especially useful for repeatable deployments, where you want to\nensure that all necessary steps are executed in the correct order. Below are\ntwo example YAML plans for deploying Vaultwarden and WordPress applications.\n\n### Example 1: Deploying Vaultwarden\n\nThis YAML plan defines a simple process to deploy the Vaultwarden application\nusing KubeCM's `kubecm::deploy` plan.\n\n`plans/deploy_vaultwarden.yaml`\n\n```yaml\n---\ndescription: 'Deploy Acme Inc Vaultwarden'\n\nsteps:\n  - description: 'Deploy Vaultwarden'\n    plan: kubecm::deploy\n    parameters:\n      release: 'vaultwarden'\n      chart_source: 'guerzon/vaultwarden'\n      repo_url: 'https://guerzon.github.io/vaultwarden'\n```\n\nThis plan deploys the Vaultwarden release using the specified chart and\nrepository URL.\n\n### Example 2: Deploying WordPress\n\nThis YAML plan defines a more flexible deployment workflow for WordPress. It\nallows the user to specify which instance (either 'shop' or 'blog') should be\ndeployed.\n\n`plans/deploy_wordpress.yaml`\n\n```yaml\n---\ndescription: 'Deploy Acme Inc WordPress'\n\nparameters:\n  site:\n    description: The release to deploy\n    type: Enum['shop', 'blog']\n\nsteps:\n  - description: 'Deploy WordPress'\n    plan: kubecm::deploy\n    parameters:\n      release: $site\n      chart: wordpress\n      chart_source: oci://registry-1.docker.io/bitnamicharts/wordpress\n```\n\nThis plan introduces the `parameters` block, which requires the user to specify\nthe site (either `shop` or `blog`) to be deployed. This is useful when you want\nto reuse the same plan logic for multiple deployments.\n\nTo run these plans, you would use the following Bolt commands:\n\n```bash\nbolt plan run acme_inc::deploy_vaultwarden\nbolt plan run acme_inc::deploy_wordpress site=shop\nbolt plan run acme_inc::deploy_wordpress site=blog\n```\n\nThese YAML plans streamline deployments, enforce consistency, and ensure\nrepeatable, parameterized workflows that are easy to maintain and execute.\n\n## Troubleshooting\n\nIf you encounter issues with your deployments, KubeCM provides several tools to\nhelp diagnose and debug problems.\n\n### Using the `render_to` Parameter\n\nThe `render_to` parameter allows you to render the final output of the Helm or\nKustomize configuration to a local file. This is useful for verifying the\nrendered Kubernetes manifests before deploying them.\n\nExample usage:\n\n```bash\nbolt plan run kubecm::deploy release=shop chart=wordpress chart_source=oci://registry-1.docker.io/bitnamicharts/wordpress render_to=output.yaml\n```\n\nAfter the command runs, the rendered configuration will be saved in\n`output.yaml` for review.\n\n### Exploring the Build Directory\n\nKubeCM generates a build directory during deployment. This directory contains\nintermediate files and configurations produced during the rendering and\npatching process. The build directory provides insight into the final state of\nconfigurations before they are applied to the Kubernetes cluster.\n\nBy default, the build directory is named `build` and is created in the current\nworking directory.\n\nTo explore the build directory, you can navigate into it and review the files:\n\n```bash\ncd build\nls -al\n```\n\nInside, you will find rendered manifests and debug information, which can be\ninvaluable when troubleshooting failed deployments or understanding how your\nconfigurations were combined.\n\n## Contributing\n\nContributions are welcome! Follow these steps to contribute:\n\n1. **Use your fork for development**:\n\n   Modify the `bolt-project.yaml` to use your forked repository:\n\n   ```yaml\n   modules:\n     - name: kubecm\n       git: \u003crepo-url\u003e\n       ref: \u003cbranch-name\u003e\n   ```\n\n   Then run:\n\n   ```bash\n   bolt module install\n   ```\n\n2. **Project is managed with PDK**:\n\n   Before contributing changes back to the project, ensure that your changes\n   pass PDK validation and tests:\n\n   ```bash\n   pdk validate\n   pdk test unit\n   ```\n\n3. **Submit a pull request**.\n\n---\n\nHappy Deploying!\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjameslikeslinux%2Fkubecm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjameslikeslinux%2Fkubecm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjameslikeslinux%2Fkubecm/lists"}