{"id":13819322,"url":"https://github.com/stefanprodan/flux-local-dev","last_synced_at":"2025-04-28T12:07:26.057Z","repository":{"id":60044579,"uuid":"539399732","full_name":"stefanprodan/flux-local-dev","owner":"stefanprodan","description":"Flux local dev environment with Docker and Kubernetes KIND","archived":false,"fork":false,"pushed_at":"2023-12-30T22:18:53.000Z","size":1286,"stargazers_count":168,"open_issues_count":1,"forks_count":34,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-04-28T12:07:21.137Z","etag":null,"topics":["flux","gitops-toolkit","testing"],"latest_commit_sha":null,"homepage":"","language":"CUE","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/stefanprodan.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null},"funding":{"github":"stefanprodan"}},"created_at":"2022-09-21T09:10:40.000Z","updated_at":"2025-04-26T05:24:08.000Z","dependencies_parsed_at":"2023-12-30T23:22:14.133Z","dependency_job_id":"dd910d3a-3338-4ae1-b987-3cf358e9717c","html_url":"https://github.com/stefanprodan/flux-local-dev","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fflux-local-dev","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fflux-local-dev/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fflux-local-dev/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/stefanprodan%2Fflux-local-dev/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/stefanprodan","download_url":"https://codeload.github.com/stefanprodan/flux-local-dev/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251311330,"owners_count":21569009,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["flux","gitops-toolkit","testing"],"created_at":"2024-08-04T08:00:45.069Z","updated_at":"2025-04-28T12:07:26.030Z","avatar_url":"https://github.com/stefanprodan.png","language":"CUE","funding_links":["https://github.com/sponsors/stefanprodan"],"categories":["testing","CUE"],"sub_categories":[],"readme":"# flux-local-dev\n\n[![test](https://github.com/stefanprodan/flux-local-dev/workflows/test/badge.svg)](https://github.com/stefanprodan/flux-local-dev/actions)\n[![license](https://img.shields.io/github/license/stefanprodan/flux-local-dev.svg)](https://github.com/stefanprodan/flux-local-dev/blob/main/LICENSE)\n[![release](https://img.shields.io/github/release/stefanprodan/flux-local-dev/all.svg)](https://github.com/stefanprodan/flux-local-dev/releases)\n\nSpin up a local dev environment for [Flux](https://github.com/fluxcd/flux2)\nwith Docker and Kubernetes KIND in under five minutes.\n\n## Who is this for?\n\n- **Flux users** who want to test Flux configs locally, without having to push changes to a Git repository.\n  Config changes are pushed to a local registry and synced on the cluster by Flux automatically.\n- **Flux contributors** who want to test their changes to Flux controllers locally,\n  without having to push the container images to an external registry.\n- **Flux maintainers** who want to test Flux prereleases on various Kubernetes versions and configurations.\n\n## How does it work?\n\nThis project spins up a Docker Registry container named `kind-registry` and a Kubernetes Kind cluster\nnamed `flux` under the same Docker network. Then it installs Flux and configures it to upgrade itself\nfrom the latest OCI artifact published at `ghcr.io/fluxcd/flux-manifests`. Before an upgrade, Flux\nverifies that the OCI artifacts are signed by the Flux team with Cosign and GitHub OIDC.\n\n| Component                                                                                                | Role                            | Host                        |\n|----------------------------------------------------------------------------------------------------------|---------------------------------|-----------------------------|\n| [Kubernetes KIND](https://kind.sigs.k8s.io/)                                                             | Local cluster                   | Binds to port 80 and 443    |\n| [Docker Registry](https://docs.docker.com/registry/)                                                     | Local registry                  | Binds to port 5050          |\n| [Flux](https://fluxcd.io)                                                                                | Cluster reconciler              | -                           |\n| [ingress-nginx](https://github.com/kubernetes/ingress-nginx)                                             | Ingress for `*.flux.local`      | -                           |\n| [cert-manager](https://github.com/cert-manager/cert-manager)                                             | Self-signed ingress certs       | -                           |\n| [metrics-server](https://github.com/kubernetes-sigs/metrics-server)                                      | Container resource metrics      | -                           |\n| [kube-prometheus-stack](https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack) | Prometheus Operator and Grafana | Binds to grafana.flux.local |\n| [weave-gitops](https://github.com/weaveworks/weave-gitops)                                               | Flux UI                         | Binds to ui.flux.local      |\n| [podinfo](https://github.com/stefanprodan/podinfo)                                                       | Demo app                        | Binds to podinfo.flux.local |\n\nThe Docker registry is exposed on the local machine on `localhost:5050` and inside the cluster\non `kind-registry:5000`. The registry servers two purposes:\n- hosts container images e.g. `docker push localhost:5050/podinfo:test1`\n- hosts OCI artifacts e.g. `flux push artifact oci://localhost:5050/podinfo-manifests:test1`\n\nTo facilitate ingress access to the Flux UI and any other\napplication running inside the cluster, the Kubernetes Kind container\nbinds to port `80` and `443` on localhost.\nIngress is handled by Kubernetes ingress-nginx and self-signed TLS certs\nare provided by cert-manager.\n\nTo monitor how the deployed applications perform on the cluster,\nthe kube-prometheus-stack and metrics-server Helm charts are installed at\nbootstrap along with the Flux Grafana dashboards.\n\nTo monitor and debug Flux using a Web UI, the Weave GitOps Helm chart is\ninstalled at bootstrap.\n\n\n## How to get started?\n\n### Prerequisites\n\nStart by cloning the repository locally:\n\n```shell\ngit clone https://github.com/stefanprodan/flux-local-dev.git\ncd flux-local-dev\n```\n\nInstall Kubernetes kind, kubectl, flux and other CLI tools with Homebrew:\n\n```shell\nmake tools\n```\n\nThe complete list of tools can be found in the `Brewfile`.\n\nNote that the minimum required version of Flux is `v2.0.0-rc.1`.\n\n### Bootstrap\n\nStart the dev environment with:\n\n```shell\nmake up\n```\n\nThe `make up` command performs the following steps:\n- creates the Docker registry container if it's not already running\n- creates the Kubernetes Kind cluster if it's not already running\n- pushes the Kubernetes manifests as OCI artifacts to the local registry\n  - `locahost:5050/flux-cluster-sync` is generated from `kubernetes/clusters/local`\n  - `locahost:5050/flux-infra-sync` is generated from `kubernetes/infra`\n  - `locahost:5050/flux-apps-sync` is generated from `kubernetes/apps`\n- installs Flux on the clusters and configures it to self upgrade from `oci://ghcr.io/fluxcd/flux-manifests`\n- waits for Flux to reconcile the cluster addons from `oci://kind-registry:5000/flux-infra-sync`\n- waits for Flux to reconcile the demo apps from `oci://kind-registry:5000/flux-apps-sync`\n\n### Access Flux UI\n\n![flux-ui](docs/img/weave-gitops.png)\n\nAdd the following domains to `/etc/hosts`:\n\n```txt\n127.0.0.1 podinfo.flux.local\n127.0.0.1 grafana.flux.local\n127.0.0.1 ui.flux.local\n```\n\nVerify that the NGINX ingress self-signed TLS works:\n\n```shell\nmake check\n```\n\nAccess the Flux UI and Grafana using the username `admin` and password `flux`:\n\n- [http://ui.flux.local/applications](http://ui.flux.local/applications)\n- [http://grafana.flux.local/d/flux-control-plane](http://grafana.flux.local/d/flux-control-plane/flux-control-plane?orgId=1\u0026refresh=10s) (username: admin, password: flux)\n- [http://grafana.flux.local/d/flux-cluster](http://grafana.flux.local/d/flux-cluster/flux-cluster-stats?orgId=1\u0026refresh=10s)\n\nAccess the demo application on [http://podinfo.flux.local](http://podinfo.flux.local/).\n\n### Sync local changes\n\nAdd a label to the `apps` namespace definition:\n\n```shell\nyq eval '.metadata.labels.env=\"dev\"' -i ./kubernetes/apps/namespace.yaml\n```\n\nValidate the Kubernetes manifests and Flux custom resources:\n\n```shell\nmake validate\n```\n\nPush the changes to the local registry with:\n\n```shell\nmake sync\n```\n\nVerify that Flux has reconciled the namespace:\n\n```shell\nkubectl get ns apps --show-labels\n```\n\n### Teardown\n\nDelete the registry and the Kubernetes cluster with:\n\n```shell\nmake down\n```\n\n## How to use CUE for manifests generation?\n\nIn the [cue](/cue) directory you can find an example of how to use\n[cuelang](https://cuelang.org/)\nto define and generate Kubernetes resources.\n\nList the CUE generated resources with `make cue-ls`:\n\n```console\n$ make cue-ls\n\nRESOURCE                                   API VERSION\nNamespace/cue-apps                         v1\nServiceAccount/cue-apps/flux-cue-apps      v1\nService/cue-apps/podinfo                   v1\nServiceAccount/cue-apps/podinfo            v1\nDeployment/cue-apps/podinfo                apps/v1\nHorizontalPodAutoscaler/cue-apps/podinfo   autoscaling/v2beta2\nIngress/cue-apps/podinfo                   networking.k8s.io/v1\nServiceMonitor/cue-apps/podinfo            monitoring.coreos.com/v1\nRoleBinding/cue-apps/flux-cue-apps         rbac.authorization.k8s.io/v1\n```\n\nPush the generated resources to the local registry with `make cue-push`:\n\n```console\n$ make cue-push \n► pushing artifact to localhost:5050/flux-cue-apps-sync:local\n✔ artifact successfully pushed to localhost:5050/flux-cue-apps-sync@sha256:59676338abbb245a80345713fea24c2686c8c38cbd235691dd0af0fdc00fe116\n```\n\nTo reconcile the resources on the cluster, add a file called `cue-apps.yaml` to the `kubernetes/cluster/local` directory:\n\n```yaml\n---\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: cue-apps-source\n  namespace: flux-system\nspec:\n  insecure: true\n  interval: 1m\n  provider: generic\n  ref:\n    tag: local\n  url: oci://kind-registry:5000/flux-cue-apps-sync\n---\napiVersion: kustomize.toolkit.fluxcd.io/v1\nkind: Kustomization\nmetadata:\n  name: cue-apps-sync\n  namespace: flux-system\nspec:\n  dependsOn:\n    - name: infra-config\n  interval: 5m\n  retryInterval: 30s\n  timeout: 5m\n  path: ./\n  prune: true\n  sourceRef:\n    kind: OCIRepository\n    name: cue-apps-source\n```\n\nSync the changes on the cluster with:\n\n```shell\nmake sync\n```\n\nList the reconciled objects with:\n\n```console\n$ flux tree ks cue-apps-sync \nKustomization/flux-system/cue-apps-sync\n├── Namespace/cue-apps\n├── ServiceAccount/cue-apps/flux-cue-apps\n├── ServiceAccount/cue-apps/podinfo\n├── RoleBinding/cue-apps/flux-cue-apps\n├── Service/cue-apps/podinfo\n├── Deployment/cue-apps/podinfo\n├── HorizontalPodAutoscaler/cue-apps/podinfo\n├── ServiceMonitor/cue-apps/podinfo\n└── Ingress/cue-apps/podinfo\n```\n\nIf you make changes to the CUE definitions, run `make cue-push` and Flux will apply the changes on its own.\n\n## How to test Flux controllers?\n\nAssuming you are contributing a change to kustomize-controller,\nand you want to run a series of end-to-end tests before opening a PR.\nMost importantly, you want to make sure your changes don't break Flux\ncapability to upgrade itself.\n\n### Build and push the controller image\n\nFrom within the kustomize-controller local clone, run `make docker-build` to build the controller image.\nIf your local machine is an Apple M1, set the arch to `linux/arm64` and run:\n\n```shell\nIMG=localhost:5050/kustomize-controller:latest make docker-build docker-push\nBUILD_PLATFORMS=linux/arm64 make docker-build\n```\n\nTag and push the image to your local registry:\n\n```shell\ndocker tag fluxcd/kustomize-controller:latest localhost:5050/kustomize-controller:test1\ndocker push localhost:5050/kustomize-controller:test1\n```\n\n### Deploy the controller\n\nFrom within the flux-local-dev clone, open the `kubernetes/clusters/local/flux-system/flux-sync.yaml` file\nand add an image patch:\n\n```yaml\napiVersion: kustomize.toolkit.fluxcd.io/v1\nkind: Kustomization\nmetadata:\n  name: flux-sync\n  namespace: flux-system\nspec:\n  images:\n    - name: ghcr.io/fluxcd/kustomize-controller\n      newName: localhost:5050/kustomize-controller\n      newTag: test1\n```\n\nSync the changes on the cluster with `make sync` and verify that the image is being rolled out:\n\n```shell\nmake sync\nkubectl -n flux-system get deploy/kustomize-controller --watch\n```\n\nFinally, verify that the upgrade was successful with:\n\n```console\n$ flux check \n\n✔ kustomize-controller: deployment ready\n► localhost:5050/kustomize-controller:test1\n```\n\nIf you don't want to bump the image tag on every build, you can bypass the local registry\nand import the image directly in the cluster cache:\n\n```shell\nexport IMG=localhost:5050/kustomize-controller:test1\nIMG=${IMG} make docker-build \u0026\u0026\nkind import docker-image ${IMG} \u0026\u0026\nkubectl delete pod -n flux-system -l app=kustomize-controller\n```\n\n## How to test Flux prereleases?\n\nAssuming you are maintainer, and you want to test the Flux controller suite\nbefore a release.\n\n### Build and push the manifests\n\nFrom within the flux2 local clone, run `make build-dev` to build the Flux CLI\nbinary that embeds the install manifests.\n\nExtract the manifests to a directory with:\n\n```shell\nmkdir -p flux-vnext\n\n./bin/flux install --components-extra=image-reflector-controller,image-automation-controller \\\n--export \u003e ./flux-vnext/install.yaml\n```\n\nPush the manifests to your local registry:\n\n```shell\n./bin/flux push artifact oci://localhost:5050/flux:latest --path ./flux-vnext \\\n--source=\"$(git config --get remote.origin.url)\" \\\n--revision=\"$(git rev-parse HEAD)\"\n```\n\n### Deploy the controllers\n\nFrom within the flux-local-dev clone, open the `kubernetes/clusters/local/flux-system/flux-source.yaml` file,\nchange the URL to point to your local registry and enable the insecure flag:\n\n```yaml\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: flux-source\n  namespace: flux-system\nspec:\n  url: oci://kind-registry:5000/flux\n  insecure: true\n```\n\nSync the changes on the cluster with `make sync` and wait for the new version to rollout:\n\n```shell\nmake sync\nflux reconcile ks flux-sync --with-source\n```\n\nFinally, verify that the upgrade was successful with:\n\n```shell\nflux check \n```\n\n## How to test Flux CRDs?\n\nAssuming you are contributing an API change to source-controller,\nand you want to run a series of end-to-end tests before opening a PR.\n\n### Build and push the controller image\n\nFrom within the source-controller local clone, run `make docker-build` to build the controller image.\nIf your local machine is an Apple M1, set the arch to `linux/arm64` and run:\n\n```shell\nIMG=localhost:5050/source-controller \\\nTAG=oci1 \\\nBUILD_PLATFORMS=linux/arm64 \\\nBUILD_ARGS=--load \\\nmake docker-build docker-push\n```\n\n### Build and push the manifests\n\nFrom within the source-controller local clone, extract the Flux manifests to a directory:\n\n```shell\nmkdir -p flux-vnext\n\nflux install --components-extra=image-reflector-controller,image-automation-controller \\\n--export \u003e ./flux-vnext/install.yaml\n```\n\nReplace the CRD with the one from your branch:\n\n```shell\nexport CRD_NAME=\"ocirepositories.source.toolkit.fluxcd.io\"\nyq e 'select(.metadata.name != env(CRD_NAME))' -i ./flux-vnext/install.yaml\n\nCRD_BASE_PATH=\"./config/crd/bases\"\nCRD_FILE_NAME=\"source.toolkit.fluxcd.io_ocirepositories.yaml\"\ncp ${CRD_BASE_PATH}/${CRD_FILE_NAME} ./flux-vnext/\n```\n\nPush the manifests to your local registry:\n\n```shell\nflux push artifact oci://localhost:5050/flux:latest --path ./flux-vnext \\\n--source=\"$(git config --get remote.origin.url)\" \\\n--revision=\"$(git rev-parse HEAD)\"\n```\n\n### Deploy the controller\n\nFrom within the flux-local-dev clone, open the `kubernetes/clusters/local/flux-system/flux-source.yaml` file,\nchange the URL to point to your local registry and enable the insecure flag:\n\n```yaml\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: flux-source\n  namespace: flux-system\nspec:\n  url: oci://kind-registry:5000/flux\n  insecure: true\n```\n\nOpen the `kubernetes/clusters/local/flux-system/flux-sync.yaml` file\nand patch the controller deployment with the local image:\n\n```yaml\napiVersion: kustomize.toolkit.fluxcd.io/v1\nkind: Kustomization\nmetadata:\n  name: flux-sync\n  namespace: flux-system\nspec:\n  images:\n    - name: ghcr.io/fluxcd/source-controller\n      newName: localhost:5050/source-controller\n      newTag: oci1\n```\n\nSync the changes on the cluster with `make sync` and wait for the new version to rollout:\n\n```shell\nmake sync\nflux reconcile ks flux-sync --with-source\n```\n\n### Manual testing\n\nTest the new feature by adding a Flux resource to the `kubernetes/apps/source-test.yaml`:\n\n```yaml\napiVersion: source.toolkit.fluxcd.io/v1beta2\nkind: OCIRepository\nmetadata:\n  name: podinfo-keyless\n  namespace: apps\nspec:\n  interval: 5m\n  url: oci://ghcr.io/stefanprodan/manifests/podinfo\n  ref:\n    semver: \"*\"\n  verify:\n    provider: cosign\n```\n\nSync the changes on the cluster and see the reconciliation result:\n\n```shell\nmake sync\nflux get source oci podinfo-keyless -n apps\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanprodan%2Fflux-local-dev","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fstefanprodan%2Fflux-local-dev","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fstefanprodan%2Fflux-local-dev/lists"}