{"id":13798266,"url":"https://github.com/kaweezle/krmfnbuiltin","last_synced_at":"2025-07-05T06:02:02.385Z","repository":{"id":65407676,"uuid":"591785019","full_name":"kaweezle/krmfnbuiltin","owner":"kaweezle","description":"Kustomize KRM function to run builtin transformers","archived":false,"fork":false,"pushed_at":"2024-05-01T22:59:50.000Z","size":395,"stargazers_count":5,"open_issues_count":5,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-24T00:14:55.447Z","etag":null,"topics":["gitops","kubernetes","kustomize","kustomize-plugin"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kaweezle.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-01-21T21:20:52.000Z","updated_at":"2025-02-26T17:23:31.000Z","dependencies_parsed_at":"2024-05-01T23:54:29.915Z","dependency_job_id":"3ead1de7-6d36-43ca-9b1b-616dd9cf818a","html_url":"https://github.com/kaweezle/krmfnbuiltin","commit_stats":null,"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaweezle%2Fkrmfnbuiltin","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaweezle%2Fkrmfnbuiltin/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaweezle%2Fkrmfnbuiltin/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kaweezle%2Fkrmfnbuiltin/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kaweezle","download_url":"https://codeload.github.com/kaweezle/krmfnbuiltin/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248120232,"owners_count":21050913,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gitops","kubernetes","kustomize","kustomize-plugin"],"created_at":"2024-08-04T00:00:41.139Z","updated_at":"2025-04-09T22:21:40.450Z","avatar_url":"https://github.com/kaweezle.png","language":"Go","funding_links":[],"categories":["Plugins"],"sub_categories":["Generators"],"readme":"# krmfnbuiltin\n\n[![stability-beta](https://img.shields.io/badge/stability-beta-33bbff.svg)](https://github.com/mkenney/software-guides/blob/master/STABILITY-BADGES.md#beta)\n\nkrmfnbuiltin is a\n[kustomize plugin](https://kubectl.docs.kubernetes.io/guides/extending_kustomize/)\nproviding a set of [KRM Functions] that you can use to perform in place\ntransformation in your kustomize projects.\n\n\u003c!-- markdownlint-disable MD033 --\u003e\n\n\u003c!-- TABLE OF CONTENTS --\u003e\n\u003cdetails open=\"true\"\u003e\n  \u003csummary\u003eTable of Contents\u003c/summary\u003e\n  \u003col\u003e\n    \u003cli\u003e\u003ca href=\"#rationale\"\u003eRationale\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#usage-example\"\u003eUsage Example\u003c/a\u003e\n    \u003cul\u003e\u003cli\u003e\u003ca href=\"#internal-annotations-cleanup\"\u003eInternal annotations cleanup\u003c/a\u003e\u003c/li\u003e\u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#use-of-generators\"\u003eUse of generators\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#keeping-or-deleting-generated-resources\"\u003eKeeping or deleting generated resources\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#extensions\"\u003eExtensions\u003c/a\u003e\n        \u003cul\u003e\n            \u003cli\u003e\u003ca href=\"#remove-transformer\"\u003eRemove Transformer\u003c/a\u003e\u003c/li\u003e\n            \u003cli\u003e\u003ca href=\"#configmap-generator-with-git-properties\"\u003eConfigMap generator with git properties\u003c/a\u003e\u003c/li\u003e\n            \u003cli\u003e\u003ca href=\"#heredoc-generator\"\u003eHeredoc generator\u003c/a\u003e\u003c/li\u003e\n            \u003cli\u003e\u003ca href=\"#kustomization-generator\"\u003eKustomization generator\u003c/a\u003e\u003c/li\u003e\n            \u003cli\u003e\u003ca href=\"#sops-decryption-generator\"\u003eSops decryption generator\u003c/a\u003e\u003c/li\u003e\n            \u003cli\u003e\u003ca href=\"#extended-replacement-in-structured-content\"\u003eExtended replacement in structured content\u003c/a\u003e\n                \u003cul\u003e\u003cli\u003e\u003ca href=\"#replacements-source-reuse\"\u003eReplacements source reuse\u003c/a\u003e\u003c/li\u003e\u003c/ul\u003e\n            \u003c/li\u003e\n        \u003c/ul\u003e\n    \u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#argo-cd-integration\"\u003eArgo CD integration\u003c/a\u003e\u003c/li\u003e\n    \u003cli\u003e\u003ca href=\"#related-projects\"\u003eRelated projects\u003c/a\u003e\u003c/li\u003e\n  \u003c/ol\u003e\n\u003c/details\u003e\n\u003c!-- markdownlint-enable MD033 --\u003e\n\n## Rationale\n\n`kustomize fn run` allows performing _in place_ transformation of KRM\n(Kubernetes Resource Model) resources. This is handy to perform structured\nmodification operations on GitOps repositories (aka _shift left_, see the\n[functions tutorial] and the [KRM Functions Specification][krm functions]).\nUnfortunately, the builtin transformers are not available to `kustomize fn run`,\nas it expects the function to be contained in an external `container` or\n`exec`utable .\n\n`krmfnbuiltin` provides both the image and executable allowing the use of any\nkustomize builtin transformer or generator, along with some additional goodies.\n\n## Usage Example\n\nLet's imagine that you have a GitOps repository containing **10** Argo CD\napplications in the `applications` folder. The following is the manifest for one\nof them:\n\n```yaml\napiVersion: argoproj.io/v1alpha1\nkind: Application\nmetadata:\n  name: argo-cd\n  namespace: argocd\n  annotations:\n    autocloud/local-application: \"true\"\nspec:\n  destination:\n    namespace: argocd\n    server: https://kubernetes.default.svc\n  ignoreDifferences:\n    - group: argoproj.io\n      jsonPointers:\n        - /status\n      kind: Application\n  project: default\n  source:\n    repoURL: https://github.com/kaweezle/autocloud.git\n    targetRevision: main\n    path: packages/argocd\n  syncPolicy:\n    automated:\n      allowEmpty: true\n      prune: true\n      selfHeal: true\n    syncOptions:\n      - CreateNamespace=true\n```\n\nYou have 9 other application manifests sharing the same snippet:\n\n```yaml\nsource:\n  repoURL: https://github.com/kaweezle/autocloud.git\n  targetRevision: main\n```\n\nLet's imagine now that you want to fork this repository for developing on\nanother cluster. You obtain a new repository,\n`https://github.com/myname/autocloud.git`, on which you create a branch named\n`feature/experiment` for development. For the deployment to the development\ncluster to use the right repository and branch, you need to change `repoURL` and\n`targetRevision` for all the applications. You can do that by hand, but this is\ncumbersome and **error prone**.\n\nOn a Kustomization, you would have done:\n\n```yaml\npatches:\n    - patch: |-\n        - op: replace\n            path: /spec/source/repoURL\n            value: https://github.com/myname/autocloud.git\n        - op: replace\n            path: /spec/source/targetRevision\n            value: feature/experiment\n    target:\n        group: argoproj.io\n        version: v1alpha1\n        kind: Application\n        # This annotation allow us to identify applications pointing locally\n        annotationSelector: \"autocloud/local-application=true\"\n```\n\nBut here you don't want to add a new kustomization nesting level. You just want\nto modify the actual application manifests on your branch. This is where KRM\nfunctions shine. To do that, you can write a function file in a `functions`\ndirectory:\n\n```yaml\n# functions/fn-change-repo-and-branch.yaml\napiVersion: builtin\nkind: PatchTransformer\nmetadata:\n  name: fn-change-repo-and-branch\n  annotations:\n    # This will remove the internal annotations the transformer adds.\n    config.kaweezle.com/cleanup: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\n    # Can also be:\n    #  container:\n    #    image: ghcr.io/kaweezle/krmfnbuiltin:v0.4.1\npatch: |-\n  - op: replace\n      path: /spec/source/repoURL\n      value: https://github.com/myname/autocloud.git\n  - op: replace\n      path: /spec/source/targetRevision\n      value: feature/experiment\ntarget:\n  group: argoproj.io\n  version: v1alpha1\n  kind: Application\n  # This annotation allow us to identify applications pointing locally\n  annotationSelector: \"autocloud/local-application=true\"\n```\n\nAnd then you can apply your modification with the following command:\n\n```console\n\u003e kustomize fn run --enable-exec --fn-path functions applications\n```\n\n**NOTE:** _the `--enable-exec` parameter is not needed if you use the\ncontainer._\n\nYou obtain the desired modifications in the manifests:\n\n```yaml\nsource:\n  repoURL: https://github.com/mysuer/autocloud.git\n  targetRevision: feature/experiment\n```\n\nYou now can commit the 10 modified manifests in your branch and deploy the\napplications.\n\n### Internal annotations cleanup\n\nSome kustomize transformers add annotations to enable for instance reference\nreconciliation at the end of the build. These annotations have the prefix\n`internal.config.kubernetes.io`. When performing a `kustomize build`, kustomize\nremoves them at the end of the process. When we are using the transformations in\nthe context of a KRM function with `kustomize fn run`, the build cleanup is not\nperformed. In consequence, the annotations would be added to the resource file\ntouched by the transformation. For instance:\n\n```diff\n--- original.argocd.yaml\n+++ transformed.argocd.yaml\n@@ -5,6 +5,9 @@\n   namespace: argocd\n   annotations:\n     autocloud/local: \"true\"\n+    internal.config.kubernetes.io/previousKinds: Application\n+    internal.config.kubernetes.io/previousNames: argo-cd\n+    internal.config.kubernetes.io/previousNamespaces: argocd\n spec:\n   destination:\n     namespace: argocd\n```\n\nTo avoid that, you can insert the following annotation:\n\n```yaml\nconfig.kaweezle.com/cleanup: \"true\"\n```\n\nIt will inform krmfnbuiltin that you are not using the transformer in the\ncontext of a bulid and that the internal annotations need to be removed.\n\n## Use of generators\n\n`krmfnbuiltin` provides all the Kustomize\n[builtin generators](https://kubectl.docs.kubernetes.io/references/kustomize/builtins/).\n\nLet's imagine that one or more of your applications use an Helm chart that in\nturn creates applications. You pass the repo URL and target branch as values to\nthe Helm Chart with the following:\n\n```yaml\nhelm:\n  parameters:\n    - name: common.targetRevision\n      value: main\n    - name: common.repoURL\n      value: https://github.com/kaweezle/autocloud.git\n```\n\nFor that particular transformation, JSON patches are not practical:\n\n```yaml\npatch: |-\n  - op: replace\n    path: /spec/source/repoURL\n    value: https://github.com/antoinemartin/autocloud.git\n  - op: replace\n    path: /spec/source/targetRevision\n    value: deploy/citest\n  - op: replace\n    path: /spec/source/helm/parameters/1/value\n    value: https://github.com/antoinemartin/autocloud.git\n  - op: replace\n    path: /spec/source/helm/parameters/0/value\n    value: deploy/citest\n```\n\nYou need to hardcode the index of the value to replace in the array, which is\nerror prone, and you start duplicating values.\n\nIt would be better to have a unique _source_ with the right values, and do\n`replacements` where needed. You can inject the values with a\n`ConfigMapGenerator`\n\n```yaml\n# 01_configmap-generator.yaml\napiVersion: builtin\n# Use this to inject current git values\n# kind: GitConfigMapGenerator\nkind: ConfigMapGenerator\nmetadata:\n  name: configuration-map\n  annotations:\n    # This annotation will be transferred to the generated ConfigMap\n    config.kaweezle.com/local-config: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\n# When using GitConfigMapGenerator, these are automatically injected\nliterals:\n  - repoURL=https://github.com/kaweezle/autocloud.git\n  - targetRevision=deploy/citest\n```\n\nAnd then use a `ReplacementTransformer` to inject the values:\n\n```yaml\n# 02_replacement-transformer.yaml\napiVersion: builtin\nkind: ReplacementTransformer\nmetadata:\n  name: replacement-transformer\n  namespace: argocd\n  annotations:\n    # Put this annotation in the last transformation to remove the generated resource\n    config.kaweezle.com/prune-local: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\nreplacements:\n  - source:\n      kind: ConfigMap\n      fieldPath: data.repoURL\n    targets:\n      - select:\n          kind: Application\n          annotationSelector: \"autocloud/local=true\"\n        fieldPaths:\n          - spec.source.repoURL\n          # This field specification is not related to the index\n          - spec.source.helm.parameters.[name=common.repoURL].value\n  - source:\n      kind: ConfigMap\n      fieldPath: data.targetRevision\n    targets:\n      - select:\n          kind: Application\n          annotationSelector: \"autocloud/local=true\"\n        fieldPaths:\n          - spec.source.targetRevision\n          - spec.source.helm.parameters.[name=common.targetRevision].value\n```\n\nSome remarks:\n\n- ✔️ The actual values (repo url and revision) are only specified once in the\n  config map generator.\n- ✔️ `spec.source.helm.parameters.[name=common.repoURL].value` is a path more\n  specific than `/spec/source/helm/parameters/1/value`. The transformation would\n  survive reordering.\n- ✔️ The functions file names are prefixed with a number prefix (`01_`, `02_`)\n  in order to ensure that the functions are executed in the right order. Note\n  that you can group the two functions in one file separated by `---` (this\n  would make it unusable from [kpt] though).\n- ✔️ The generators contains the annotation:\n\n  ```yaml\n  config.kaweezle.com/local-config: \"true\"\n  ```\n\n  that is injected in the generated resource.\n\n- ✔️ In the last transformation, we add the following annotation:\n\n  ```yaml\n  config.kaweezle.com/prune-local: \"true\"\n  ```\n\n  In order to avoid saving the generated resources. In the presence of this\n  annotation, `krmfnbuiltin` will remove all the resource having the\n  `config.kaweezle.com/local-config` annotation.\n\n## Keeping or deleting generated resources\n\nAs said above, generated resources are saved by default. To prevent that,\nadding:\n\n```yaml\nconfig.kaweezle.com/local-config: \"true\"\n```\n\non the generators and:\n\n```yaml\nconfig.kaweezle.com/prune-local: \"true\"\n```\n\nOn the last transformation will remove those resources. In the absence of these\nannotations, the generated resources will be saved in a file named\n`.krmfnbuiltin.yaml` located in the configuration directory. You may want to add\nthis file name to your `.gitignore` file in order to avoid committing it.\n\nIn some cases however, we want to _inject_ new resources in the configuration.\nThis can be done by just omitting the `config.kaweezle.com/local-config`\nannotation.\n\nThe name of the file containing the generated resources can be set with the\nfollowing annotations:\n\n- `config.kaweezle.com/path` for the filename. If it contains directories, they\n  will be created.\n- `config.kaweezle.com/index` For the starting index of the resources in the\n  file.\n\nExample:\n\n```yaml\napiVersion: builtin\nkind: ConfigMapGenerator\nmetadata:\n  name: configuration-map\n  annotations:\n    # config.kaweezle.com/local-config: \"true\"\n    config.kaweezle.com/path: local-config.yaml\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\n```\n\nWith these annotations, the generated config map will be saved in the\n`local-config.yaml` file in the configuration directory.\n\nIf the file name is empty, i.e. the annotation is:\n\n```yaml\nconfig.kaweezle.com/path: \"\"\n```\n\nThe generated resources will be saved each in its own file with the pattern:\n\n```text\n\u003cnamespace\u003e/\u003ckind\u003e_\u003cname\u003e.yaml\n```\n\nFor instance:\n\n```text\nkube-flannel/daemonset_kube-flannel-ds.yaml\n```\n\n## Extensions\n\nThis section describes the krmfnbuiltin additions to the Kustomize transformers\nand generators as well as the _enhancements_ that have been made to some of\nthem.\n\n### Remove Transformer\n\nIn the case the transformation(s) involves other transformers than\n`krmfnbuiltin`, the `config.kaweezle.com/prune-local` may not be available to\nremove resources injected in the transformation pipeline. For this use case,\n`krmfnbuiltin` provides `RemoveTransformer`:\n\n```yaml\napiVersion: builtin\nkind: RemoveTransformer\nmetadata:\n  name: replacement-transformer\n  annotations:\n    config.kubernetes.io/function: |\n      exec:\n        path: ../../krmfnbuiltin\ntargets:\n  - annotationSelector: config.kaweezle.com/local-config\n```\n\nEach target specified in the `targets` field follows the\n[patches target convention](https://kubectl.docs.kubernetes.io/references/kustomize/builtins/#field-name-patches).\n\nNote that you can use the Kustomize recommended method with a\n`PatchStrategicMergeTransformer` and a `$patch: delete` field. The above\ntransformation is however more explicit.\n\n### ConfigMap generator with git properties\n\n`GitConfigMapGenerator` work identically to `ConfigMapGenerator` except it adds\ntwo properties of the current git repository to the generated config map:\n\n- `repoURL` contains the URL or the remote specified by `remoteName`. by\n  default, it takes the URL of the remote named `origin`.\n- `targetRevision` contains the name of the current branch.\n\nThis generator is useful in transformations that use those values, like for\ninstance Argo CD application customization. Information about the configuration\nof the generator can be found in the [ConfigMapGenerator kustomize\ndocumentation].\n\nThe following function configuration:\n\n```yaml\n# 01_configmap-generator.yaml\napiVersion: builtin\nkind: GitConfigMapGenerator\nmetadata:\n  name: configuration-map\n  annotations:\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\nremoteName: origin # default\n```\n\nproduces the following config map (comments mine):\n\n```yaml\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: configuration-map\n  namespace: argocd\n  annotations:\n    # add config.kaweezle.com/prune-local: \"true\" to last transformer to remove\n    config.kubernetes.io/local-config: \"true\"\n    # Add .generated.yaml to .gitignore to avoid mistakes\n    internal.config.kubernetes.io/path: .generated.yaml\n    config.kubernetes.io/path: .generated.yaml\ndata:\n  repoURL: git@github.com:kaweezle/krmfnbuiltin.git\n  targetRevision: feature/extended-replacement-transformer\n```\n\n### Heredoc generator\n\nWe have seen in [Use of generators](#use-of-generators) how to use\n`ConfigMapGenerator` to _inject_ values in order to use them in downstream\ntransformers, replacements in particular. It has however some limitations, due\nto the _flat nature_ of ConfigMaps and the fact that values are only strings.\nThe former makes it difficult to organize replacement variables and the later\nprevents structural (_object_) replacement. For object replacements we can use\n`PatchStrategicMergeTransformer`, but then we loose the `ReplacementTransformer`\nadvantage of using the same source for several targets and end up having\nduplicate YAML snippets.\n\n`krmfnbuiltin` allows injecting any KRM resource in the transformation by just\nadding the `config.kaweezle.com/inject-local: \"true\"` annotation to the function\nconfiguration. For instance:\n\n```yaml\napiVersion: config.kaweezle.com/v1alpha1\nkind: LocalConfiguration\nmetadata:\n  name: traefik-customization\n  annotations:\n    # This will inject this resource. like a ConfigMapGenerator, but with hierarchical\n    # properties\n    config.kaweezle.com/inject-local: \"true\"\n    # This annotation will allow pruning at the end\n    config.kaweezle.com/local-config: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\ndata:\n  # kustomization\n  traefik:\n    dashboard_enabled: true\n    expose: true\n  sish:\n    # New properties\n    server: target.link\n    hostname: myhost.target.link\n    host_key: AAAAGnNrLXNzaC1lZDI1NTE5QG9wZW5zc2guY29tAAAAID+4/eqtPTLC18TE8ZP7NeF4ZP68/wnY2d7mhH/KVs79AAAABHNzaDo=\n```\n\nWhen the function configuration contains the `config.kaweezle.com/inject-local`,\nannotation, `krmfnbuiltin` bypasses the generation/transformation process for\nthis function and return the content of the function config _as if_ it had been\ngenerated. The `config.kaweezle.com/inject-local` annotation as well as the\n`config.kubernetes.io/function` annotation are removed from the injected\nresource.\n\nThe resource contents can then be used in the following transformations, in\nparticular in replacements, and deleted at the end (with\n`config.kaweezle.com/local-config` and `config.kaweezle.com/prune-local`) or\neven saved (with `config.kaweezle.com/path`). See\n[Keeping or deleting generated resources](#keeping-or-deleting-generated-resources))\nfor more details.\n\n### Kustomization generator\n\n`KustomizationGenerator` is the kustomize equivalent to\n`HelmChartInflationGenerator`. It allows generating resources from a\nkustomization.\n\nExample:\n\n```yaml\napiVersion: builtin\nkind: KustomizationGenerator\nmetadata:\n  name: kustomization-generator\n  annotations:\n    config.kaweezle.com/path: \"uninode.yaml\" # file name to save resources\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\nkustomizeDirectory: https://github.com/antoinemartin/autocloud.git//packages/uninode?ref=deploy/citest\n```\n\nIf this function is run with the following command:\n\n```console\n\u003e kustomize fn run --enable-exec --fn-path functions applications\n```\n\nIt will generate a file named `uninode.yaml` containing all the resources of the\nbuilt kustomization in the `applications` directory. With:\n\n```yaml\nconfig.kaweezle.com/path: \"\"\n```\n\nOne file will be created per resource (see\n[Keeping or deleting generated resources](#keeping-or-deleting-generated-resources)).\n\n**IMPORTANT** The current directory `krmfnbuiltin` runs from is the directory in\nwhich the `kustomize run fn` command has been launched, and **not from the\nfunction configuration folder**. Any relative path should take this into\nconsideration.\n\n### Sops decryption generator\n\nThe `SopsGenerator` generates resources from encrypted content. This content can\nbe the actual function configuration, in [heredoc](#heredoc-generator) style, or\ncan come from other files.\n\nIt is an inclusion of [krmfnsops](https://github.com/kaweezle/krmfnsops). See\nits README file for more information.\n\nIn the simplest use case, Imagine you have an unencrypted secret that looks like\nthis :\n\n```yaml\n# argocd-secret.yaml\napiVersion: v1\nkind: Secret\ntype: Opaque\nmetadata:\n  name: argocd-secret\nstringData:\n  admin.password: $2a$10$xdlX460lf/WbJNZU5bBoROj6U7oKgPbEcBrnXaemA6gsCzrAJtQ3y\n  admin.passwordMtime: \"2022-08-30T11:26:42Z\"\n  webhook.github.secret: ZxqGggxGD070l3dx\n  dex.github.clientSecret: 7lqt6nasit6kjtvptmy2dzy1dr796orn5xh05ru1\n```\n\nIf you encrypt it with [sops], you get something like this:\n\n```yaml\n# argocd-secret.yaml\napiVersion: v1\nkind: Secret\ntype: Opaque\nmetadata:\n  name: argocd-secret\nstringData:\n  admin.password: ENC[AES256_GCM,data:...,type:str]\n  admin.passwordMtime: ENC[AES256_GCM,data:...,type:str]\n  webhook.github.secret: ENC[AES256_GCM,data:...,type:str]\n  dex.github.clientSecret: ENC[AES256_GCM,data:...==,type:str]\nsops:\n  age:\n    - recipient: age166k86d56...\n      enc: |\n        -----BEGIN AGE ENCRYPTED FILE-----\n        ...\n        -----END AGE ENCRYPTED FILE-----\n  lastmodified: \"2023-02-06T11:36:44Z\"\n  mac: ENC[AES256_GCM,data:...,type:str]\n  pgp: []\n  encrypted_regex: ^(data|stringData|.*_keys?|admin|adminKey|password)$\n  version: 3.7.3\n```\n\nIf you want this resource to be unencrypted at kustomization build, you can\ncreate the following generator configuration:\n\n```yaml\n# argocd-secret-generator.yaml\napiVersion: krmfnbuiltin.kaweezle.com/v1alpha1\nkind: SopsGenerator\nmetadata:\n  name: argocd-secret-generator\n  annotations:\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\nfiles:\n  - argocd-secret.yaml\n```\n\nAnd insert it in the `generators:` section of your `kustomization.yaml` file:\n\n```yaml\ngenerators:\n  - argocd-secret-generator.yaml\n```\n\nTo avoid adding a generator configuration file to your kustomization, you can\ndirectly transform the encrypted secret file into a KRM generator:\n\n```yaml\n# argocd-secret.yaml\napiVersion: krmfnbuiltin.kaweezle.com/v1alpha1\nkind: SopsGenerator\ntype: Opaque\nmetadata:\n  name: argocd-secret\n  annotations:\n    config.kaweezle.com/kind: \"Secret\"\n    config.kaweezle.com/apiVersion: \"v1\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\nstringData:\n  admin.password: ENC[AES256_GCM,data:...,type:str]\n  admin.passwordMtime: ENC[AES256_GCM,data:...,type:str]\n  webhook.github.secret: ENC[AES256_GCM,data:...,type:str]\n  dex.github.clientSecret: ENC[AES256_GCM,data:...==,type:str]\nsops:\n  age:\n    - recipient: age166k86d56...\n      enc: |\n        -----BEGIN AGE ENCRYPTED FILE-----\n        ...\n        -----END AGE ENCRYPTED FILE-----\n  lastmodified: \"2023-02-06T11:36:44Z\"\n  mac: ENC[AES256_GCM,data:...,type:str]\n  pgp: []\n  encrypted_regex: ^(data|stringData|.*_keys?|admin|adminKey|password)$\n  version: 3.7.3\n```\n\nAnd your `kustomization.yaml` file would look like:\n\n```yaml\ngenerators:\n  - argocd-secret.yaml\n```\n\nNote the use of the following annotations:\n\n```yaml\nconfig.kaweezle.com/kind: \"Secret\"\nconfig.kaweezle.com/apiVersion: \"v1\"\n```\n\nIn order to have the generated resource with the proper kind and api version.\n\n**WARNING** While this second inclusion method reduces the number of files, it\ndisables the [sops] Message authentication code (MAC) verification that prevents\nfile tampering. Use it at your own risk.\n\n### Extended replacement in structured content\n\nThe `ReplacementTransformer` provided in `krmfnbuiltin` is _extended_ compared\nto the standard one because it allows structured replacements in properties\ncontaining a string representation of some structured content. It currently\nsupports the following structured formats:\n\n- YAML\n- JSON\n- TOML\n- INI\n\nIt also provides helpers for changing content in base64 encoded properties as\nwell as a simple regexp based replacer for edge cases. The standard\nconfiguration of the transformer can be found in the [replacements kustomize\ndocumentation].\n\nThe typical use case for this is when you have an Argo CD application using a\nHelm chart as source with some custom values:\n\n```yaml\napiVersion: argoproj.io/v1alpha1\nkind: Application\nmetadata:\n  name: traefik\n  namespace: argocd\n  finalizers:\n    - resources-finalizer.argocd.argoproj.io\nspec:\n  destination:\n    namespace: traefik\n    server: https://kubernetes.default.svc\n  project: default\n  source:\n    chart: traefik\n    repoURL: https://helm.traefik.io/traefik\n    targetRevision: \"10.19.5\"\n    helm:\n      parameters: []\n      values: |-\n        ingressClass:\n          enabled: true\n          isDefaultClass: true\n        ingressRoute:\n          dashboard:\n            enabled: false\n        providers:\n          kubernetesCRD:\n            allowCrossNamespace: true\n            allowExternalNameServices: true\n          kubernetesIngress:\n            allowExternalNameServices: true\n            publishedService:\n              enabled: true\n        logs:\n          general:\n            level: ERROR\n          access:\n            enabled: true\n        tracing:\n          instana: false\n        gobalArguments: {}\n        # BEWARE: use only for debugging\n        additionalArguments:\n         - --api.insecure=false\n        ports:\n          # BEWARE: use only for debugging\n          # traefik:\n          #   expose: false\n          web:\n            redirectTo: websecure\n          websecure:\n            tls:\n              enabled: true\n  syncPolicy:\n    automated:\n      prune: true\n      selfHeal: true\n    syncOptions:\n      - CreateNamespace=true\n  ignoreDifferences: []\n```\n\nAnd that you want your KRM function to personalize the values of the Helm chart.\nWhat you would want is having your replacement path _follow inside_ the values\nproperty by specifying:\n\n```yaml\n- spec.source.helm.values.\u003cinside\u003e.ingressRoute.dashboard.enabled\n```\n\nThis is not possible with the standard `ReplacementTransformer`, but this is is\npossible with the one provided by `krmfnbuiltin`. Consider the following\nfunction configurations:\n\n```yaml\n# fn-traefik-customization.yaml\napiVersion: builtin\nkind: LocalConfiguration\nmetadata:\n  name: traefik-customization\n  annotations:\n    # This will inject this resource. like a ConfigMapGenerator, but with hierarchical\n    # properties\n    config.kaweezle.com/inject-local: \"true\"\n    config.kaweezle.com/local-config: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\ndata:\n  # kustomization\n  traefik:\n    dashboard_enabled: true\n    expose: true\n---\napiVersion: builtin\nkind: ReplacementTransformer\nmetadata:\n  name: replacement-transformer\n  annotations:\n    # remove LocalConfiguration after\n    config.kaweezle.com/prune-local: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\nreplacements:\n  - source:\n      kind: LocalConfiguration\n      fieldPath: data.traefik.dashboard_enabled\n    targets:\n      - select:\n          kind: Application\n          name: traefik\n        fieldPaths:\n          # !!yaml tells the transformer that the property contains YAML\n          - spec.source.helm.values.!!yaml.ingressRoute.dashboard.enabled\n  - source:\n      kind: LocalConfiguration\n      fieldPath: data.traefik.expose\n    targets:\n      - select:\n          kind: Application\n          name: traefik\n        fieldPaths:\n          - spec.source.helm.values.!!yaml.ports.traefik.expose\n```\n\nIf you apply this to the directory containing the application, you will obtain a\nnew application:\n\n```yaml\napiVersion: argoproj.io/v1alpha1\nkind: Application\nmetadata:\n  name: traefik\n  namespace: argocd\n  finalizers:\n    - resources-finalizer.argocd.argoproj.io\n  annotations:\n    config.kubernetes.io/path: traefik.yaml\n    internal.config.kubernetes.io/path: traefik.yaml\nspec:\n  destination:\n    namespace: traefik\n    server: https://kubernetes.default.svc\n  project: default\n  source:\n    chart: traefik\n    helm:\n      parameters: []\n      values: |\n        ...\n        ingressRoute:\n          dashboard:\n            enabled: true\n        ...\n        ports:\n          # BEWARE: use only for debugging\n          # traefik:\n          #   expose: false\n          web:\n            redirectTo: websecure\n          websecure:\n            tls:\n              enabled: true\n          traefik:\n            expose: true\n    repoURL: https://helm.traefik.io/traefik\n    targetRevision: \"10.19.5\"\n  syncPolicy:\n    automated:\n      prune: true\n      selfHeal: true\n    syncOptions:\n      - CreateNamespace=true\n  ignoreDifferences: []\n```\n\nAs you can see, inside the `values` property, the yaml has been modified.\n`ingressRoute.dashboard.enabled` is now `true` and `port.traefik.expose` is also\n`true`. Notice that this last property, also present as a comment, has been\ninserted at the end of the `ports` section.\n\nNow for a more _extreme_ use case involving regular expressions, imagine you\nhave the following configuration map defining two files:\n\n```yaml\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: sish-client\n  namespace: traefik\n  labels:\n    app.kubernetes.io/name: \"sish-client\"\n    app.kubernetes.io/component: edge\n    app.kubernetes.io/part-of: autocloud\ndata:\n  # ~/.ssh/config file\n  config: |\n    PubkeyAcceptedKeyTypes +ssh-rsa\n    Host sishserver\n      HostName holepunch.in\n      Port 2222\n      BatchMode yes\n      IdentityFile ~/.ssh_keys/id_rsa\n      IdentitiesOnly yes\n      LogLevel ERROR\n      ServerAliveInterval 10\n      ServerAliveCountMax 2\n      RemoteCommand sni-proxy=true\n      RemoteForward citest.holepunch.in:443 traefik.traefik.svc:443\n  # ~/.ssh/known_hosts with the server key\n  known_hosts: |\n    [holepunch.in]:2222 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAID+3abW2y3T5dodnI5O1Z/2KlIdH3bwnbGDvCFf13zlh\n```\n\nAnd imagine you want to modify it to access a different server on another domain\nname. You need to change:\n\n- `HostName` in `~/.ssh/config` from `holepunch.in` to the new server address.\n- `RemoteForward` in `~/.ssh/config` by changing the address forwarded from\n  `citest.holepunch.in` to the new address.\n- In `~/.ssh/known_hosts` the name of the host and the key fingerprint of the\n  new server.\n\nYou can do this by hand, but you may forget something now and the next time.\nThis is where the regexp transformer comes into play with the following\nconfiguration:\n\n```yaml\napiVersion: builtin\nkind: LocalConfiguration\nmetadata:\n  name: configuration-map\n  annotations:\n    config.kaweezle.com/inject-local: \"true\"\n    config.kaweezle.com/local-config: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\ndata:\n  sish:\n    # New properties\n    server: target.link\n    hostname: myhost.target.link\n    host_key: AAAAGnNrLXNzaC1lZDI1NTE5QG9wZW5zc2guY29tAAAAID+4/eqtPTLC18TE8ZP7NeF4ZP68/wnY2d7mhH/KVs79AAAABHNzaDo=\n---\napiVersion: builtin\nkind: ReplacementTransformer\nmetadata:\n  name: replacement-transformer\n  annotations:\n    config.kaweezle.com/prune-local: \"true\"\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\nreplacements:\n  - source:\n      kind: LocalConfiguration\n      fieldPath: data.sish.server\n    targets:\n      - select:\n          kind: ConfigMap\n          name: sish-client\n        fieldPaths:\n          - data.config.!!regex.^\\s+HostName\\s+(\\S+)\\s*$.1\n          - data.known_hosts.!!regex.^\\[(\\S+)\\].1\n  - source:\n      kind: LocalConfiguration\n      fieldPath: data.sish.hostname\n    targets:\n      - select:\n          kind: ConfigMap\n          name: sish-client\n        fieldPaths:\n          - data.config.!!regex.^\\s+RemoteForward\\s+(\\S+):.1\n  - source:\n      kind: LocalConfiguration\n      fieldPath: data.sish.host_key\n    targets:\n      - select:\n          kind: ConfigMap\n          name: sish-client\n        fieldPaths:\n          - data.known_hosts.!!regex.ssh-ed25519\\s(\\S+).1\n```\n\nThe _path_ after `!!regex` is composed of two elements. The first one is the\nregexp to match. The second one is the the capture group that needs to be\nreplaced with the source. In the first replacement, the regexp:\n\n```regexp\n^\\s+HostName\\s+(\\S+)\\s*$\n```\n\ncan be interpreted as:\n\n\u003e a line starting with one or more spaces followed by `HostName`, then one or\n\u003e more spaces and a sequence of non space characters, captured as a group; then\n\u003e optional spaces till the end of the line.\n\nThe second part of the path, `1`, tells to replace the first capturing group\nwith the source. With the above, the line:\n\n```sshconfig\n      HostName holepunch.in\n```\n\nwill become\n\n```sshconfig\n      HostName target.link\n```\n\n#### Replacements source reuse\n\nIn the above examples, the `ReplacementTransformer` gets the source data from a\ngenerator that is injected (`config.kaweezle.com/inject-local: \"true\"`) and then\nremoved (`config.kaweezle.com/prune-local: \"true\"`). The extended version of\n`ReplacementTransformer` allows specifying a `source:` that can either be a\nresource file or the path of a kustomization.\n\nWe can create a `properties.yaml` file:\n\n```yaml\n# properties.yaml\napiVersion: autocloud.config.kaweezle.com/v1alpha1\nkind: PlatformValues\nmetadata:\n  name: autocloud-values\ndata:\n  traefik:\n    dashboard_enabled: true\n    expose: true\n  sish:\n    hostname: mydomain.link\n    remote: argocd.mydomain.link\n    host_key: AAAAC3NzaC1lZDI1NTE5AAAAIEAfLUpTj0fn5sJFW6agmLMsvEacMBvXocyzHLW+AOSQ\n    # more configuration below...\n```\n\nAnd then reference it from our replacements:\n\n```yaml\n# fn-traefik-customization.yaml\napiVersion: builtin\nkind: ReplacementTransformer\nmetadata:\n  name: replacement-transformer\n  annotations:\n    config.kubernetes.io/function: |\n      exec:\n        path: krmfnbuiltin\n# Source of replacements\nsource: properties.yaml\nreplacements:\n  - source:\n      kind: LocalConfiguration\n      fieldPath: data.traefik.dashboard_enabled\n    targets:\n      - select:\n          kind: Application\n          name: traefik\n        fieldPaths:\n          # !!yaml tells the transformer that the property contains YAML\n          - spec.source.helm.values.!!yaml.ingressRoute.dashboard.enabled\n```\n\nAs the source of the replacement is _side loaded_, there no need to inject it\nnor remove it from the configuration. Also, as the `source` can be a\nkustomization, there is no need for it to be local.\n\n#### Replacement with encoding\n\nKustomize has an `encoding` option in `ReplacementTransformer` that is currently\nunused. We put it to the work and provide three encoding types:\n\n- base64\n- bcrypt\n- hex\n\nExample:\n\n```yaml\n- source:\n    name: autocloud-values\n    fieldPath: data.to_encode\n    options:\n      encoding: base64\n  targets:\n    - select:\n        kind: ConfigMap\n        name: argocd-cm\n      fieldPaths:\n        - data.b64encoded\n```\n\nThanks to this feature, you can keep some values in clear text inside your\nproperties files and encode them on kustomization. Be aware that the `bcrypt`\nencoding will generate a new value for each kustomization.\n\n## Installation\n\nWith each [Release](https://github.com/kaweezle/krmfnbuiltin/releases), we\nprovide binaries for most platforms as well as Alpine based packages.\n\nOn POSIX systems (Linux and Mac), you can install the last version with:\n\n```console\ncurl -sLS https://raw.githubusercontent.com/kaweezle/krmfnbuiltin/main/get.sh | /bin/sh\n```\n\nIf you don't want to pipe into shell, you can do:\n\n```console\n\u003e KRMFNBUILTIN_VERSION=\"v0.4.1\"\n\u003e curl -sLo /usr/local/bin/krmfnbuiltin https://github.com/kaweezle/krmfnbuiltin/releases/download/${KRMFNBUILTIN_VERSION}/krmfnbuiltin_${KRMFNBUILTIN_VERSION}_linux_amd64\n```\n\n## Argo CD integration\n\n`krmfnbuiltin` is **NOT** primarily meant to be used inside Argo CD, but instead\nto perform _structural_ modifications to the configuration **BEFORE** it's\ncommitted and provided to GitOps.\n\nAnyway, to use `krmfnbuiltin` with Argo CD, you need to:\n\n- Make the `krmfnbuiltin` binary available to the `argo-repo-server` pod.\n- Have Argo CD run kustomize with the `--enable-alpha-plugins --enable-exec`\n  parameters.\n\nTo add krmfnbuiltin on argo-repo-server, the\n[Argo CD documentation](https://argo-cd.readthedocs.io/en/stable/operator-manual/custom_tools/)\nprovides different methods to make custom tools available.\n\nIf you get serious about Argo CD, you will probably end up cooking your own\nimage. This\n[docker file](https://github.com/antoinemartin/autocloud/blob/deploy/citest/repo-server/Dockerfile#L45)\nshows how to use the above installation instructions in your image. To\nsummarize:\n\n```Dockerfile\nFROM argoproj/argocd:latest\n\nARG KRMFNBUILTIN_VERSION=v0.4.1\n\n# Switch to root for the ability to perform install\nUSER root\n\n# Install tools\nRUN apt-get update \u0026\u0026 \\\n    apt-get install -y curl \u0026\u0026 \\\n    apt-get clean \u0026\u0026 \\\n    rm -rf /var/lib/apt/lists/* \u0026\u0026 \\\n    curl -sLo /usr/local/bin/krmfnbuiltin https://github.com/kaweezle/krmfnbuiltin/releases/download/${KRMFNBUILTIN_VERSION}/krmfnbuiltin_${KRMFNBUILTIN_VERSION}_linux_amd64\n\nUSER argocd\n```\n\nYou also need to patch the `argo-cm` config map to add the parameters. The\nfollowing is a strategic merge patch for it:\n\n```yaml\n# argocd-cm.yaml\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: argocd-cm\ndata:\n  # Options to enable exec plugins (krmfnsops).\n  kustomize.buildOptions: \"--enable-alpha-plugins --enable-exec\"\n  ...\n```\n\n## Related projects\n\n[kpt], from Google, takes this in place transformation principle to another\nlevel by making resource configuration packages similar to docker images. In\nthis model, a generator or a transformer along its parameters in much like a\nline in a dockerfile. It takes a current configuration as source and generates a\nnew configuration after transformation.\n\nkrmfnbuiltin works with [kpt]. The `tests/test_krmfnbuiltin_kpt.sh` script\nperform the basic tests with kpt.\n\n[knot8] lenses have provided the idea of extended paths.\n\n\u003c!-- prettier-ignore-start --\u003e\n\n[KRM Functions]: https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/api-conventions/functions-spec.md\n[kpt]: https://kpt.dev/guides/rationale\n[functions tutorial]: https://github.com/kubernetes-sigs/kustomize/blob/master/cmd/config/docs/tutorials/function-basics.md\n\n[knot8]: https://knot8.io/\n[ConfigMapGenerator kustomize documentation]:\n  https://kubectl.docs.kubernetes.io/references/kustomize/builtins/#_configmapgenerator_\n[replacements kustomize documentation]: https://kubectl.docs.kubernetes.io/references/kustomize/kustomization/replacements/\n[sops]: https://github.com/mozilla/sops\n\u003c!-- prettier-ignore-end --\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaweezle%2Fkrmfnbuiltin","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkaweezle%2Fkrmfnbuiltin","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkaweezle%2Fkrmfnbuiltin/lists"}