{"id":21658210,"url":"https://github.com/segmentio/kubeapply","last_synced_at":"2025-08-07T10:09:42.407Z","repository":{"id":37534947,"uuid":"282353096","full_name":"segmentio/kubeapply","owner":"segmentio","description":"A lightweight tool for git-based management of Kubernetes configs","archived":false,"fork":false,"pushed_at":"2024-10-30T20:23:30.000Z","size":761,"stargazers_count":146,"open_issues_count":8,"forks_count":16,"subscribers_count":12,"default_branch":"master","last_synced_at":"2024-10-30T21:25:31.917Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/segmentio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2020-07-25T02:12:17.000Z","updated_at":"2024-10-30T20:23:33.000Z","dependencies_parsed_at":"2024-06-18T22:33:23.034Z","dependency_job_id":"127c1c3d-a08a-474f-ae03-bc967132770f","html_url":"https://github.com/segmentio/kubeapply","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/segmentio%2Fkubeapply","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/segmentio%2Fkubeapply/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/segmentio%2Fkubeapply/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/segmentio%2Fkubeapply/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/segmentio","download_url":"https://codeload.github.com/segmentio/kubeapply/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226304468,"owners_count":17603601,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T09:28:53.745Z","updated_at":"2024-11-25T09:28:56.459Z","avatar_url":"https://github.com/segmentio.png","language":"Go","funding_links":[],"categories":["Go","Configuration Management"],"sub_categories":[],"readme":"# kubeapply\n\n## Contents\n\n  * [Overview](#overview)\n  * [Motivation](#motivation)\n  * [Getting started](#getting-started)\n  * [Configuration](#configuration)\n  * [Usage (CLI)](#usage-cli)\n  * [Usage (Github webhooks)](#usage-github-webhooks)\n  * [Experimental features](#experimental-features)\n  * [Testing](#testing)\n\n## Overview\n\n`kubeapply` is a lightweight tool for git-based management of Kubernetes configs.\nIt supports configuration in raw YAML, templated YAML,\n[Helm charts](https://helm.sh/docs/topics/charts/), and/or [skycfg](https://github.com/stripe/skycfg),\nand facilitates the complete change workflow including config expansion, validation,\ndiff generation, and applying.\n\nIt can be run from either the command-line or in a webhooks mode that responds\ninteractively to Github pull request events:\n\n\u003cimg width=\"823\" src=\"https://user-images.githubusercontent.com/54862872/88240197-4c180200-cc3b-11ea-933d-edf3e15f7e8c.png\"\u003e\n\n## Motivation\n\nManaging Kubernetes configuration in a large organization can be painful. Existing tools like\n[Helm](https://helm.sh/) are useful for certain parts of the workflow (e.g., config generation),\nbut are often too heavyweight for small, internal services. Once configs are generated, it's hard\nto understand what impact they will have in production and then apply them in a consistent way.\n\nWe built `kubeapply` to make the end-to-end config management process easier and more\nconsistent. The design choices made were motivated by the following goals:\n\n1. Support git-based workflows. i.e., can look at a repo to understand the state of a cluster\n2. Make it easy to share configuration between different environments (e.g., staging vs. production)\n  and cluster types\n3. Wrap existing tooling (`kubectl`, `helm`, etc.) whenever possible as opposed to\n  reimplementing their functionality\n4. Allow running on either the command-line or in Github\n5. Support Helm charts, simple templates, and skycfg\n\nSee [this blog post](https://segment.com/blog/kubernetes-configuration/) for more details.\n\n### Disclaimer\n\nThe tool is designed for our Kubernetes-related workflows at [Segment](https://segment.com).\nWhile we hope it can work for others, not all features might be directly applicable to other\nenvironments. We welcome feedback and collaboration to make `kubeapply` useful to more people!\n\n## 🆕 Terraform version of `kubeapply`\n\nWe recently open-sourced a Terraform-provider-based version of this tool; see the\n[repository](https://github.com/segmentio/terraform-provider-kubeapply) and\n[documentation in the Terraform registry](https://registry.terraform.io/providers/segmentio/kubeapply/latest/docs) for more details.\n\nNote that the Terraform version has slightly different interfaces and assumptions (e.g., no\nsupport for Helm charts), so it's not a drop-in replacement for the tooling here, but it follows\nthe same general flow and configuration philosophy.\n\n## Getting started\n\n### Prerequisites\n\n`kubeapply` depends on the following tools:\n\n- [`kubectl`](https://kubernetes.io/docs/tasks/tools/install-kubectl/): v1.16 or newer\n- [`helm`](https://helm.sh/docs/intro/install/): v3.5.0 or newer (only needed if using helm charts)\n\nMake sure that they're installed locally and available in your path.\n\n### Installing\n\nInstall the `kubeapply` binary by running:\n\n```go install github.com/segmentio/kubeapply/cmd/kubeapply@latest```\n\nYou can also build and install the binary by running `make install` in the root of this\nrepo.\n\n### Quick tour\n\nSee the [README](/examples/kubeapply-test-cluster/README.md) in the\n[`examples/kubeapply-test-cluster`](/examples/kubeapply-test-cluster) directory.\n\n## Configuration\n\n### Repo layout\n\nEach cluster type to be managed by `kubeapply` lives in a directory in a source-controlled\nrepo. This directory has a set of *cluster configs* and a *profile* that is shared by the former.\nA cluster config plus the profile is *expanded* by `kubeapply` to create a set of expanded,\nkubectl-compatible YAML configs for the cluster. Although these expanded configs can be derived in\na fully reproducible way from the cluster configs and profile, they're typically checked-in to the\nrepo for easier code review.\n\nThe following diagram shows the recommended directory layout:\n\n```\n.\n└── clusters\n    └── [cluster type]\n        ├── expanded\n        |   ├── ...\n        ├── profile\n        │   ├── [namespace 1]\n        │   │   ├── [config 1]\n        │   │   ├── [config 2]\n        │   ├── [namespace 2]\n        │   ├── ...\n        ├── [cluster config1]\n        └── [cluster config2]\n```\n\nEach of the subcomponents is described in more detail in the sections below. See also\n[`examples/kubeapply-test-cluster`](/examples/kubeapply-test-cluster) for a full example.\n\n### Cluster config\n\nEach cluster instance is configured in a single YAML file. Typically, these instances\nwill vary by the environment (e.g., staging vs. production) and/or the region/account\nin which they're deployed, but will share the same profile. At Segment, we name the files\nby the environment and region, e.g., `stage-us-west-2.yaml`, `production-us-west-2.yaml`,\nbut you can use any naming convention that feels comfortable.\n\nEach cluster config has the following format:\n\n```yaml\n# Basic information about the cluster. The combination of these should uniquely identify\n# a single cluster instance running in a single location.\ncluster: my-cluster    # Name of the cluster\nregion: us-west-2      # Region in which the cluster is running\nenv: staging           # Environment/account in which the cluster is running\n\n# Where charts can be found by default. Only required if using Helm chart sources.\n# See the section below for the supported URL formats.\ncharts: \"file://../../charts\"\n\n# Arbitrary parameters that can be used in templates, Helm charts, and skycfg modules.\n#\n# These are typically used for things that will vary by cluster instance and/or will\n# frequently change, e.g. the number of replicas for deployments, container image URIs, etc.\nparameters:\n  service1:\n    imageTag: abc123\n    replicas: 2\n\n  service2:\n    imageTag: def678\n    replicas: 5\n  ...\n```\n\n### Profile\n\nThe `profile` directory contains source files that are used to generate Kubernetes\nconfigs for a specific cluster. By convention, these files are organized into subdirectories\nby namespace, and can be further subdivided below that.\n\nThe tool currently supports four kinds of input source configs, described in more detail\nbelow.\n\n#### (1) Raw YAML\n\nFiles of the form `[name].yaml` will be treated as normal YAML files and copied to the\n`expanded` directory as-is.\n\n#### (2) Templated YAML\n\nFiles with names ending in `.gotpl.yaml` will be templated using the\n[golang `text/template` package](https://golang.org/pkg/text/template/) with the cluster config\nas the input data. You can also use the functions in the\n[sprig library](http://masterminds.github.io/sprig/).\n\nSee [this file](/examples/kubeapply-test-cluster/profile/apps/echoserver/deployment.gotpl.yaml)\nfor an example.\n\nNote that template expansion happens before Helm chart evaluation, so you can template Helm\nvalue files as well.\n\n#### (3) Helm chart values\n\nFiles named `[chart name].helm.yaml` will be treated as values files for the associated chart.\nThe chart will be expanded using `helm template ...` and the outputs copied into the `expanded`\ndirectory. See [this file](/examples/kubeapply-test-cluster/profile/apps/envoy/envoy.helm.yaml)\nfor an example (which references the [envoy chart](/examples/kubeapply-test-cluster/charts/envoy)).\n\nBy default, charts are sourced from the URL set in the cluster config `charts` parameter.\nCurrently, the tool supports URLs of the form `file://`, `http://`, `https://`, `git://`,\n`git-https://`, and `s3://`.\n\nYou can override the source for a specific chart by including a `# charts: [url]`\ncomment at the top of the values file. This is helpful for testing out a new version\nfor just one chart in the profile.\n\n#### (4) Skycfg/starlark modules\n\nFiles ending in `.star` will be evaluated using the\n[skycfg framework](https://github.com/stripe/skycfg) to generate one or more Kubernetes protobufs.\nThe latter are then converted to kubectl-compatible YAML and copied into the `expanded` directory.\n\nSkycfg uses the [Starlark](https://github.com/bazelbuild/starlark) language along with typed\nKubernetes structs (from [Protocol Buffers](https://developers.google.com/protocol-buffers/)),\nso it can provide more structure and less repetition than YAML-based sources. See\n[this file](/examples/kubeapply-test-cluster/profile/apps/redis/deployment.star) for an example.\n\nThe skycfg support in `kubeapply` is experimental and unsupported.\n\n### Expanded configs\n\nThe `expanded` directory contains the results of expanding out the `profile` for\na cluster instance. These configs are pure YAML that can be applied directly via `kubectl apply`\nor, preferably, using the `kubeapply apply` command (described below).\n\n## Usage (CLI)\n\n#### Expand\n\n`kubeapply expand [path to cluster config]`\n\nThis will expand out all of the configs for the cluster instance, and put them into\na subdirectory of the `expanded` directory. Helm charts are expanded via `helm template`;\nother source types use custom code in the `kubeapply` binary.\n\n#### Validate\n\n`kubeapply validate [path to cluster config] --policy=[path to OPA policy in rego format]`\n\nThis validates all of the expanded configs for the cluster using the\n[`kubeconform`](https://github.com/yannh/kubeconform) library. It also, optionally, supports\nvalidating configs using one or more [OPA](https://www.openpolicyagent.org/) policies in\nrego format; see the \"Experimental features\" section below for more details.\n\n#### Diff\n\n`kubeapply diff [path to cluster config] --kubeconfig=[path to kubeconfig]`\n\nThis wraps `kubectl diff` to show a diff between the expanded configs on disk and the\nassociated resources in the cluster.\n\n#### Apply\n\n`kubeapply apply [path to cluster config] --kubeconfig=[path to kubeconfig]`\n\nThis wraps `kubectl apply`, with some extra logic to apply in a \"safe\" order\n(e.g., configmaps before deployments, etc.).\n\n## Usage (Github webhooks)\n\nIn addition to interactions through the command-line, `kubeapply` also supports an\n[Atlantis](https://www.runatlantis.io/)-inspired, Github-based workflow for the `diff` and\n`apply` steps above. This allows the diffs to be more closely reviewed before being\napplied, and also ensures that the configuration in the repo stays in-sync with the cluster.\n\n### Workflow\n\nThe end-to-end user flow is fairly similar to the one used with Atlantis for\nTerraform changes:\n\n1. Team member changes a cluster config or profile file in the repo, runs\n  `kubeapply expand` locally\n2. A pull request is opened in Github with the changes\n3. Kubeapply server gets webhook from Github, posts a friendly \"help\" message and then\n  runs an initial diff\n4. PR owner iterates on change, gets it reviewed\n5. When ready, PR owner posts a `kubeapply apply` comment\n6. Kubeapply server gets webhook, checks that change has green status and is approved,\n  then applies it\n7. If all changes have been successfully applied, change is automatically merged\n\n### Backend\n\nUsing the Github webhooks flow requires that you run an HTTP service somewhere that is accessible\nto Github. Since the requests are sporadic and can be handled without any local state,\nprocessing them is a nice use case for a serverless framework like\n[AWS Lambda](https://aws.amazon.com/lambda/), and this is how we run it at Segment. Alternatively,\nyou can run a long-running server that responds to the webhooks.\n\nThe sections below contain some implementation details for each option.\n\n#### Option 1: Run via AWS Lambda\n\nThe exact setup steps will vary based on your environment and chosen tooling. At a high-level,\nhowever, the setup process is:\n\n1. Build a lambda bundle by running `make lambda-zip`\n2. Upload the bundle zip to a location in S3\n3. Generate a Github webhook token and a shared secret that will be used for webhook\n  authentication; store these in SSM\n4. Using Terraform, the AWS console, or other tooling of your choice, create:\n    1. An-externally facing ALB\n    2. An IAM role for your lambda function that has access to the zip bundle in S3, secrets in SSM,\n      etc.\n    3. A security group for your lambda function that has access to your cluster control planes\n    4. A lambda function that runs the code in the zip bundle when triggered by ALB requests\n\nThe lambda is configured via a set of environment variables that are documented in\nthe [lambda entrypoint](/cmd/kubeapply-lambda/main.go). We use SSM for storing secrets like\nGithub tokens, but it's possible to adapt the code to get these from other places.\n\n#### Option 2: Run via long-running server\n\nWe've provided a basic server entrypoint [here](/cmd/kubeapply-server/main.go). Build a binary\nvia `make kubeapply-server`, configure and deploy this on your infrastructure of choice,\nand expose the server to the Internet.\n\n### Github configuration\n\nOnce you have an externally accessible webhook URL, go to the settings for your repo\nand add a new webhook:\n\n\u003cimg width=\"958\" src=\"https://user-images.githubusercontent.com/54862872/88247376-5bef1080-cc52-11ea-83fb-2603dbaccd47.png\"\u003e\n\nIn the \"Event triggers\" section, select \"Issue comments\" and \"Pull requests\" only. Then, test it\nout by opening up a new pull request that modifies an expanded kubeapply config.\n\n## Experimental features\n\n### `kubestar`\n\nThis repo now contains an experimental tool, `kubestar`, for converting YAML to\nskycfg-compatible starlark. See [this README](/cmd/kubestar/README.md) for details.\n\n### Multi-profile support\n\nThe cluster config now supports using multiple profiles. Among other use cases, this is useful if\nyou want to share profile-style YAML templates across multiple clusters without dealing with Helm.\n\nTo use this, add a `profiles` section to the cluster config:\n\n```yaml\ncluster: my-cluster\n...\nprofiles:\n  - name: [name of first profile]\n    url: [url for first profile]\n  - name: [name of second profile]\n    url: [url for second profile]\n  ...\n```\n\nwhere the `url`s are in the same format as those for Helm chart locations,\ne.g. `file://path/to/my/file`. The outputs of each profile will be expanded into\n`[expanded dir]/[profile name]/...`.\n\n### OPA policy checks\n\nThe `kubeapply validate` subcommand now supports checking expanded configs against policies in\n[Open Policy Agent (OPA)](https://www.openpolicyagent.org/) format. This can be helpful for\nenforcing organization-specific standards, e.g. that images need to be pulled from a particular\nprivate registry, that all labels are in a consistent format, etc.\n\nTo use this, write up your policies as `.rego` files as described in the OPA documentation and run\nthe former subcommand with one or more `--policy=[path to policy]` arguments. By default, policies\nshould be in the `com.segment.kubeapply` package. Denial reasons, if any, are returned by\nsetting a `deny` variable with a set of denial reason strings. If this set is empty,\n`kubeapply` will assume that the config has passed all checks in the policy file.\n\nIf a denial reason begins with the string `warn:`, then that denial will be treated as a\nnon-blocking warning as opposed to an error that causes validation to fail.\n\nSee [this unit test](/pkg/validation/policy_test.go) for some examples.\n\n## Testing\n\n### Unit tests\n\nRun `make test` in the repo root.\n\n### On Github changes\n\nYou can simulate Github webhook responses by running `kubeapply` with the `pull-request`\nsubcommand:\n\n```\nkubeapply pull-request \\\n  --github-token=[personal token] \\\n  --repo=[repo in format owner/name] \\\n  --pull-request=[pull request num] \\\n  --comment-body=\"kubeapply help\"\n```\n\nThis will respond locally using the codepath that would be executed\nin response to a Github webhook for the associated repo and pull request.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsegmentio%2Fkubeapply","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsegmentio%2Fkubeapply","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsegmentio%2Fkubeapply/lists"}