{"id":8251909,"url":"https://github.com/fluxcd/flux2-hub-spoke-example","last_synced_at":"2025-07-11T21:34:02.325Z","repository":{"id":232606321,"uuid":"784434209","full_name":"fluxcd/flux2-hub-spoke-example","owner":"fluxcd","description":"A hub-and-spoke example for multi-cluster continuous delivery with Flux","archived":false,"fork":false,"pushed_at":"2024-12-17T10:13:22.000Z","size":137,"stargazers_count":29,"open_issues_count":0,"forks_count":18,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-04-25T12:46:10.867Z","etag":null,"topics":["continuous-delivery","fluxcd","gitops","kubernetes"],"latest_commit_sha":null,"homepage":"https://fluxcd.io","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fluxcd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-04-09T21:03:53.000Z","updated_at":"2025-04-24T11:54:52.000Z","dependencies_parsed_at":"2024-04-10T17:10:56.074Z","dependency_job_id":"616c838d-f366-4848-8894-0f362f04747a","html_url":"https://github.com/fluxcd/flux2-hub-spoke-example","commit_stats":null,"previous_names":["fluxcd/flux2-hub-spoke-example"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/fluxcd/flux2-hub-spoke-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxcd%2Fflux2-hub-spoke-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxcd%2Fflux2-hub-spoke-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxcd%2Fflux2-hub-spoke-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxcd%2Fflux2-hub-spoke-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fluxcd","download_url":"https://codeload.github.com/fluxcd/flux2-hub-spoke-example/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fluxcd%2Fflux2-hub-spoke-example/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264902843,"owners_count":23681123,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["continuous-delivery","fluxcd","gitops","kubernetes"],"created_at":"2024-04-19T11:05:12.287Z","updated_at":"2025-07-11T21:34:02.288Z","avatar_url":"https://github.com/fluxcd.png","language":"Shell","funding_links":[],"categories":["kubernetes"],"sub_categories":[],"readme":"# flux2-hub-spoke-example\n\n[![test](https://github.com/fluxcd/flux2-hub-spoke-example/workflows/test/badge.svg)](https://github.com/fluxcd/flux2-hub-spoke-example/actions)\n[![e2e](https://github.com/fluxcd/flux2-hub-spoke-example/workflows/e2e/badge.svg)](https://github.com/fluxcd/flux2-hub-spoke-example/actions)\n[![license](https://img.shields.io/github/license/fluxcd/flux2-hub-spoke-example.svg)](https://github.com/fluxcd/flux2-hub-spoke-example/blob/main/LICENSE)\n\nThis repository showcases how to run Flux on a central Kubernetes cluster\nand have it manage the GitOps continuous delivery of apps and infrastructure\nworkloads on multiple clusters.\n\n![Flux Hub and Spoke](.github/imgs/flux-hub-spoke-diagram.png)\n\n## Prerequisites\n\nFor this example, you need to install the following tools:\n\n- [Flux CLI](https://fluxcd.io/flux/installation/#install-the-flux-cli)\n- [Kubernetes KinD](https://kind.sigs.k8s.io/docs/user/quick-start/#installation)\n- kubectl and kustomize\n\nOn macOS and Linux, you can install the tools using the following commands:\n\n```bash\nbrew install fluxcd/tap/flux\nbrew install kind kubectl kustomize\n```\n\nIn order to follow the guide you'll need a GitHub account and a\n[personal access token](https://help.github.com/en/github/authenticating-to-github/creating-a-personal-access-token-for-the-command-line)\nthat can create repositories (check all permissions under `repo`).\n\n## Repository structure\n\nThe Git repository contains the following top directories:\n\n- `deploy` dir contains the HelmRelease definitions for the apps and infrastructure workloads\n- `clusters` dir contains the apps and infrastructure Kustomize overlays for each target cluster\n- `hub` dir contains the Flux configuration for the central cluster and targets\n\n```shell\n├── deploy\n│   ├── apps\n│   │   ├── podinfo.yaml\n│   │   └── kustomization.yaml\n│   ├── infra-configs\n│   │   ├── cluster-issuers.yaml\n│   │   └── kustomization.yaml\n│   ├── infra-controllers\n│   │   ├── cert-manager.yaml\n│   │   ├── ingress-nginx.yaml\n│   │   └── kustomization.yaml\n│   └── tenants\n├── clusters\n│   ├── production\n│   └── staging\n│       ├── apps\n│       │   ├── kustomization.yaml\n│       │   └── podinfo-values.yaml\n│       ├── infra-configs\n│       ├── infra-controllers\n│       └── tenants\n└── hub\n    ├── flux-system\n    ├── production.yaml\n    └── staging.yaml\n```\n\n## Bootstrap the cluster fleet\n\nTo bootstrap the cluster fleet, first you need to create several Kubernetes KinD\nclusters by running the following command:\n\n```shell\nmake fleet-up\n```\n\nThe above command will create the following clusters:\n\n- `flux-hub` - the central cluster where Flux will run\n- `flux-staging` - the target cluster where Flux will deploy the `clusters/staging` workloads\n- `flux-production` - the target cluster where Flux will deploy the `clusters/production` workloads\n\nAfter the clusters are created, kubeconfig files for staging and production are generated and persisted\nin the `flux-hub` cluster, so that Flux can access the target clusters.\n\n```console\n$ kubectl get secrets -A\nNAMESPACE     NAME                     TYPE\nproduction    cluster-kubeconfig       Opaque\nstaging       cluster-kubeconfig       Opaque\n```\n\nFork this repository on your personal GitHub account and\nexport your GitHub access token, username and repo name:\n\n```shell\nexport GITHUB_TOKEN=\u003cyour-token\u003e\nexport GITHUB_USER=\u003cyour-username\u003e\nexport GITHUB_REPO=\u003crepository-name\u003e\n```\n\nThen, bootstrap Flux on the hub cluster:\n\n```shell\nflux bootstrap github \\\n    --context=kind-flux-hub \\\n    --owner=${GITHUB_USER} \\\n    --repository=${GITHUB_REPO} \\\n    --branch=main \\\n    --personal \\\n    --path=hub\n```\n\nThe bootstrap command commits the manifests for the Flux components in `hub/flux-system` dir\nand creates a deploy key with read-only access on GitHub, so it can pull changes inside the cluster.\n\nWait for the Flux to reconcile the infrastructure and apps workloads on the target clusters with:\n\n```shell\nwatch flux get kustomizations -A\n```\n\nOnce the Flux Kustomizations are ready, you can list the Helm releases deployed in the target clusters.\nFor example, in the staging cluster:\n\n```console\n$ helm --kube-context kind-flux-staging ls -A\nNAME            NAMESPACE       STATUS     CHART\ncert-manager    cert-manager    deployed   cert-manager-v1.14.4\ningress-nginx   ingress-nginx   deployed   ingress-nginx-4.10.0 \npodinfo         podinfo         deployed   podinfo-6.6.2  \n```\n\nFor each target cluster, there is a corresponding namespace in the hub cluster that contains the\nFlux HelmRelease objects for the apps and infrastructure workloads.\nFor example, in the staging namespace:\n\n```console\n$ flux --context kind-flux-hub -n staging get hr\nNAME            REVISION        SUSPENDED       READY                                                                                           \ncert-manager    v1.14.4         False           True        \ningress-nginx   4.10.0          False           True      \npodinfo         6.6.2           False           True\n```\n\n\u003e [!TIP]\n\u003e Note that Flux detects changes made directly in-cluster on the objects managed\n\u003e by a HelmRelease and automatically [corrects the drift](https://fluxcd.io/flux/components/helm/helmreleases/#drift-detection).\n\u003e During an incident or for debugging purposes, you can manually suspend the reconciliation\n\u003e of a HelmRelease with `flux suspend hr \u003cname\u003e -n \u003cnamespace\u003e`.\n\n## Customize the workloads\n\nAssuming you want to ship workloads to the production cluster with a different configuration,\nyou can employ Kustomize patches in the `clusters/production` overlay and change the Flux HelmRelease values.\n\nFor example, to change the number of replicas for `ingress-nginx` in the production cluster,\nyou can create a patch file in `clusters/production/infra-controllers/ingress-nginx-values.yaml`:\n\n```yaml\napiVersion: helm.toolkit.fluxcd.io/v2beta2\nkind: HelmRelease\nmetadata:\n  name: ingress-nginx\nspec:\n  chart:\n    spec:\n      version: \"\u003e=4.10.0\"\n  values:\n    controller:\n      replicaCount: 2\n```\n\nAnd then apply the patch to the `ingress-nginx` HelmRelease in the\n`clusters/production/infra-controllers/kustomization.yaml` file with:\n\n```yaml\napiVersion: kustomize.config.k8s.io/v1beta1\nkind: Kustomization\nresources:\n  - ../../../deploy/infra-controllers\npatches:\n  - target:\n      kind: HelmRelease\n      name: ingress-nginx\n    path: ingress-nginx-values.yaml\n```\n\nVerify that the patch is correctly applied with:\n\n```shell\nkustomize build ./clusters/production/infra-controllers/\n```\n\nAfter you commit the changes to the repository, Flux will automatically apply the changes.\n\nYou can trigger a manual reconciliation with:\n\n```shell\nflux -n production reconcile ks infra-controllers --with-source\n```\n\nTo verify the number of pods, you can list the deployments in the production cluster:\n\n```console\nkubectl --context kind-flux-production -n ingress-nginx get deploy\nNAME                       READY   UP-TO-DATE\ningress-nginx-controller   2/2     2\n```\n\n\u003e [!IMPORTANT]\n\u003e Note that on production clusters, it is recommended to pin the Helm chart to an exact\n\u003e version and to use a promotion workflow to test new versions on the staging cluster before\n\u003e deploying to production. For more information, see the guide\n\u003e [Promote Flux Helm Releases with GitHub Actions](https://fluxcd.io/flux/use-cases/gh-actions-helm-promotion/).\n\n## Security considerations and dependency management\n\nIn the `deploy/tenants` dir we provision the tenant namespaces and RBAC resources. There are two types of tenants:\n\n- cluster admins - have full access to the cluster resources and can deploy HelmReleases that contain CRD controllers\n- app operators - have restricted access to the app namespaces and can't manage cluster-wide resources like CRDs\n\nAt bootstrap, Flux provisions the tenant namespaces and RBAC resources in the target clusters.\nThe `deploy/apps` HelmReleases are deployed using the `flux-restricted` service account while the\n`deploy/infra-controllers` HelmReleases and the `deploy/infra-configs` custom resources\nare deployed using the `flux-cluster-admin` service account.\n\nTo enforce the RBAC restrictions, and to provision the controllers before the custom resources, we use the\n`dependsOn` feature in the `hub/staging.yaml` and `hub/production.yaml` to order the reconciliation like so:\n\n1. `tenants` (namespaces, service accounts and role bindings resources)\n2. `infra-controllers` (CRD controllers - depends on `tenants`)\n3. `infra-configs` (cluster-wide custom resources - depends on `infra-controllers`)\n4. `apps` (app workloads - depends on `infra-configs`)\n\n\u003e [!TIP]\n\u003e When managing a large number of tenants and clusters, it is recommended to use run a dedicated\n\u003e Flux instance for each group of clusters belonging to the same tenant. For more information\n\u003e on how to assign Flux instances to specific clusters, see the\n\u003e [Flux sharding and horizontal scaling guide](https://fluxcd.io/flux/installation/configuration/sharding/).\n\n## Cluster connectivity and access control\n\nFor the Flux kustomize-controller and helm-controller to be able to\nreconcile the remote clusters, the Kubernetes API servers\nneed to be accessible from the central cluster.\n\nThe Flux controllers authenticate with the target clusters using\nkubeconfig files stored as Kubernetes secrets in the central cluster.\n\nBoth the Flux `Kustomization` and `HelmRelease` objects take a reference to the\nKubernetes secret containing the kubeconfig file:\n\n```yaml\nkind: Kustomization | HelmRelease\nspec:\n  kubeConfig:\n    secretRef:\n      name: cluster-kubeconfig\n```\n\nThe secret defined in the `secretRef` must exist in the same namespace as the `Kustomization`\nor `HelmRelease` object, and the kubeconfig file must be stored in the `value` data key.\n\nIf the target clusters are accessible over a proxy, the proxy address must be set in the kubeconfig file.\nIf the target API servers use self-signed certificates, both controllers can be configured\nto skip the TLS verification by setting the `--insecure-kubeconfig-tls` flag in the controller container args.\n\n\u003e [!IMPORTANT]\n\u003e Note that kubeconfigs that rely on exec-based authentication plugins are not supported by default.\n\u003e You will need to build custom container images with the necessary binaries and configure\n\u003e the controllers with the `--insecure-kubeconfig-exec` flag. Another option is to generate kubeconfigs\n\u003e with bearer tokens and refresh them periodically with a CronJob that runs e.g. `aws eks get-token`.\n\n## Monitoring and alerting\n\nTo configure Prometheus, Loki and Grafana for monitoring the Flux controllers and the workloads reconciliation,\nsee the [monitoring example repository](https://github.com/fluxcd/flux2-monitoring-example).\n\nTo configure Flux to send events for Slack, Teams, Discord, Sentry and others external system,\nyou can follow the [alerting guide](https://fluxcd.io/flux/monitoring/alerts/).\n\n## Testing\n\nAfter making changes to the manifests, you can validate them locally with [kubeconform](https://github.com/yannh/kubeconform) by running:\n\n```shell\nmake validate\n```\n\nAny change to the Kubernetes manifests or to the repository structure should be validated in CI before\na pull requests is merged into the main branch and synced on the cluster.\n\nThis repository contains the following GitHub CI workflows:\n\n* the [test workflow](./.github/workflows/test.yaml) validates the Kubernetes manifests and Kustomize overlays are conformant with the Flux OpenAPI spec \n* the [e2e workflow](./.github/workflows/e2e.yaml) starts the Kubernetes cluster fleet in CI and tests the setup by running Flux in Kubernetes Kind\n\n## Teardown\n\nTo delete the cluster fleet, run the following command:\n\n```shell\nmake fleet-down\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluxcd%2Fflux2-hub-spoke-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffluxcd%2Fflux2-hub-spoke-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffluxcd%2Fflux2-hub-spoke-example/lists"}