{"id":48229063,"url":"https://github.com/controlplaneio-fluxcd/d1-fleet","last_synced_at":"2026-04-04T19:29:11.441Z","repository":{"id":223271736,"uuid":"759751721","full_name":"controlplaneio-fluxcd/d1-fleet","owner":"controlplaneio-fluxcd","description":"Example repository for GitOps-driven cluster fleet management with Flux","archived":false,"fork":false,"pushed_at":"2026-01-27T09:08:42.000Z","size":205,"stargazers_count":45,"open_issues_count":3,"forks_count":13,"subscribers_count":6,"default_branch":"main","last_synced_at":"2026-01-27T21:37:30.752Z","etag":null,"topics":["fluxcd","gitops"],"latest_commit_sha":null,"homepage":"https://control-plane.io/enterprise-for-flux-cd/","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"agpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/controlplaneio-fluxcd.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2024-02-19T09:14:06.000Z","updated_at":"2026-01-27T09:08:46.000Z","dependencies_parsed_at":"2024-03-01T13:28:25.543Z","dependency_job_id":"8cd9fbec-82ca-48f0-b417-19925633272d","html_url":"https://github.com/controlplaneio-fluxcd/d1-fleet","commit_stats":null,"previous_names":["controlplaneio-fluxcd/d1-fleet"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/controlplaneio-fluxcd/d1-fleet","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio-fluxcd%2Fd1-fleet","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio-fluxcd%2Fd1-fleet/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio-fluxcd%2Fd1-fleet/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio-fluxcd%2Fd1-fleet/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/controlplaneio-fluxcd","download_url":"https://codeload.github.com/controlplaneio-fluxcd/d1-fleet/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio-fluxcd%2Fd1-fleet/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":31410680,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-04-04T10:20:44.708Z","status":"ssl_error","status_checked_at":"2026-04-04T10:20:06.846Z","response_time":60,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["fluxcd","gitops"],"created_at":"2026-04-04T19:29:10.815Z","updated_at":"2026-04-04T19:29:11.432Z","avatar_url":"https://github.com/controlplaneio-fluxcd.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# d1-fleet\n\n\u003e [!NOTE]\n\u003e This repository is part of the reference architecture for the\n\u003e [ControlPlane Enterprise for Flux CD](https://github.com/controlplaneio-fluxcd/distribution).\n\u003e\n\u003e The `d1` reference architecture comprised of\n\u003e [d1-fleet](https://github.com/controlplaneio-fluxcd/d1-fleet),\n\u003e [d1-infra](https://github.com/controlplaneio-fluxcd/d1-infra) and\n\u003e [d1-apps](https://github.com/controlplaneio-fluxcd/d1-apps) \n\u003e is a set of best practices and production-ready examples for using Flux\n\u003e to manage the continuous delivery of Kubernetes infrastructure and\n\u003e applications on multi-cluster multi-tenant environments.\n\n## Scope and Access Control\n\nThis repository is managed by the platform team who are responsible for\nthe Kubernetes infrastructure and have direct access to the fleet of clusters.\n\nThe platform team that manages this repository must have **admin** rights to the `d1-fleet` repository\nand **cluster admin** rights to all clusters in the fleet to be able to perform the following tasks:\n\n- Bootstrap Flux with multi-tenancy restrictions on fleet clusters.\n- Configure the delivery of platform components (defined in [d1-infra repository](https://github.com/controlplaneio-fluxcd/d1-infra)).\n- Configure the delivery of applications (defined in [d1-apps repository](https://github.com/controlplaneio-fluxcd/d1-apps)).\n\n```mermaid\nflowchart LR\n\nA((Infra Repo)) --\u003e C\nB((Apps Repo)) --\u003e C(((Fleet Repo)))\nC --\u003e D(Flux \\nsync main branch)\nD --\u003e E[Staging cluster]\nC--\u003e G(Flux \\nsync prod branch)\nG --\u003e H[Production-A]\nG --\u003e I[Production-B]\nG --\u003e J[Production-C]\n```\n\n## Create a GitHub Account for Flux\n\nCreate a new GitHub account for the Flux bot. This account will be used by\nthe Flux controllers running on clusters to authenticate with GitHub.\n\nCreate a GitHub team under your organisation for the bot account and give it the following permissions:\n\n- Read-only access to the `d1-fleet` repository (required for cluster bootstrap)\n- Read and write access to the `d1-infra` and `d1-apps` repositories (required for cluster reconciliation and image automation)\n\n### Flux GitHub PAT for platform components\n\nCreate a GitHub fine-grained personal access token for the bot account with\nthe following permissions for the `d1-fleet` repository:\n\n- `Administration` -\u003e `Access: Read-only`\n- `Commit statuses` -\u003e `Access: Read and write`\n- `Contents` -\u003e `Access: Read-only`\n- `Metadata` -\u003e `Access: Read-only`\n\nThis token will be stored in all clusters to authenticate with GitHub to pull the fleet\ndesired state from the `d1-fleet` repository.\n\nCreate a second GitHub fine-grained personal access token for the bot account with\nthe following permissions for the `d1-infra` and `d1-apps` repositories:\n\n- `Administration` -\u003e `Access: Read-only`\n- `Commit statuses` -\u003e `Access: Read and write`\n- `Contents` -\u003e `Access: Read and write`\n- `Metadata` -\u003e `Access: Read-only`\n\nThis token will be stored in all clusters to authenticate with GitHub to pull the fleet\ndesired state from the `d1-infra` repository. The token is also used to automate the\nHelm chart updates in the `d1-infra` and `d1-apps` repositories, where the bot account\nhas push access to the main branch.\n\n## Bootstrap Procedure\n\nThe bootstrap procedure is a one-time operation that sets up Flux Operator and the Flux controllers\non the cluster, and configures the delivery of platform components and applications.\n\nAfter bootstrap, Flux will monitor the repository for changes and will reconcile Flux Operator\nfrom the Kubernetes manifest at `./clusters/\u003ccluster_name\u003e/flux-system/flux-operator.yaml` and\nthe FluxInstance resource from the Kubernetes manifest at\n`./clusters/\u003ccluster_name\u003e/flux-system/flux-instances.yaml`. Flux Operator, in turn, will\nreconcile Flux from the FluxInstance resource applied in the cluster by Flux. This results in\nFlux reconciling Flux Operator, and Flux Operator reconciling Flux, creating a self-sustaining\nreconciliation loop. Changes to Flux configuration and version upgrades are done by modifying\nthe repository and letting Flux Operator reconcile the changes, there is no need to run the\nbootstrap command ever again nor connect to the cluster.\n\n### Bootstrap the staging cluster\n\nMake sure to set the default context in your kubeconfig to your staging cluster, then run the following:\n\n```shell\nexport GITHUB_TOKEN=\u003cFlux Bot PAT\u003e\nexport GITHUB_OWNER=\u003cYour GitHub org or user\u003e\n\n# Install the Flux Operator Helm chart\nhelm install flux-operator oci://ghcr.io/controlplaneio-fluxcd/charts/flux-operator \\\n  --namespace flux-system \\\n  --create-namespace\n\n# Create the flux-system Git secret with the Flux bot PAT\nflux create secret git flux-system \\\n  --url=https://github.com/${GITHUB_OWNER}/d1-fleet \\\n  --username=git \\\n  --password=$GITHUB_TOKEN\n\n# Apply the FluxInstance custom resource to bootstrap Flux\nkubectl apply -f clusters/staging/flux-system/flux-instance.yaml\n\n# Onboard the apps tenant by creating the flux-apps secret with the apps GitHub PAT\nflux create secret git flux-apps \\\n  --namespace=flux-system \\\n  --label=toolkit.fluxcd.io/tenant=apps \\\n  --url=https://github.com \\\n  --username=git \\\n  --password=$GITHUB_TOKEN\n```\n\nFrom this point on, the Flux controllers will reconcile the cluster state with the desired state. Any changes\nto the `clusters/staging` directory in the `d1-fleet` repository will be automatically applied to the cluster.\n\n### Upgrade to the enterprise version\n\nWhen using the [ControlPlane enterprise](https://control-plane.io/enterprise-for-flux-cd/)\ndistribution for Flux, you need to create a \nKubernetes Image Pull Secret for the enterprise registry in the `flux-system` namespace:\n\n```shell\nflux create secret oci flux-enterprise-auth \\\n  --url=ghcr.io \\\n  --username=flux \\\n  --password=$FLUX_ENTERPRISE_TOKEN\n```\n\nThen you need to update the `.spec.distribution.registry` field of the FluxInstance to the\nFlux Enterprise distribution you prefer, such as `ghcr.io/controlplaneio-fluxcd/alpine` or\n`ghcr.io/controlplaneio-fluxcd/distroless`, and add the `.spec.distribution.imagePullSecret`\nfield to reference the image pull secret created above. Example:\n\n```yaml\nspec:\n  distribution:\n    version: 2.7.x\n    registry: ghcr.io/controlplaneio-fluxcd/distroless-fips\n    artifact: \"oci://ghcr.io/controlplaneio-fluxcd/flux-operator-manifests:latest\"\n    imagePullSecret: flux-enterprise-auth\n```\n\nAnother option is to copy the images from the ControlPlane registry to your organization's registry\nand update the `.spec.distribution.registry` field to point to your registry. You will also need to\nspecify the distribution variant in the field `.spec.distribution.variant`.\n\nCopying an image from the ControlPlane registry to your organization's registry can be done with the following commands:\n\n```shell\nFLUX_CONTROLLERS=(\n  \"source-controller\"\n  \"kustomize-controller\"\n  \"helm-controller\"\n  \"notification-controller\"\n  \"image-reflector-controller\"\n  \"image-automation-controller\"\n  \"source-watcher\"\n)\n\nfor controller in \"${FLUX_CONTROLLERS[@]}\"; do\n  crane copy --all-tags ghcr.io/controlplaneio-fluxcd/distroless/$controller  \u003cyour-registry\u003e/$controller\ndone\n```\n\nManifest example:\n\n```yaml\nspec:\n  distribution:\n    version: 2.7.x\n    registry: \u003cyour-registry\u003e\n    variant: enterprise-distroless-fips\n```\n\n### Rotate the Flux GitHub PAT\n\nIt is recommended to use GitHub fine-grained personal access tokens that expire. Before the Flux bot token expires,\nyou should rotate the token by creating a new one and updating the `flux-system` secret in the `flux-system` namespace:\n\n```shell\nflux create secret git flux-system \\\n  --namespace=flux-system \\\n  --url=https://github.com \\\n  --username=git \\\n  --password=$NEW_GITHUB_TOKEN\n```\n\n## Onboarding platform components\n\nThe platform team is responsible for onboarding the platform components defined as Flux HelmReleases in the\n[d1-infra repository](https://github.com/controlplaneio-fluxcd/d1-infra) and set the dependencies\nbetween the components.\n\nPlatform components are cluster add-ons such as CRD and admission controllers,\nand are reconciled by Flux as the **cluster admin**.\n\nTo onboard a component from the `d1-infra` repository, the platform team must add a\nFlux Kustomization to the `tenants/infra/components` directory in the `d1-fleet` repository.\n\nFor example, the `d1-fleet` repository contains the following definitions for the `infra` tenant:\n\n```shell\n./tenants/infra/components/\n├── admission.yaml\n└── monitoring.yaml\n```\n\nWhich configures the reconciliation the `infra` components defined in the `d1-infra` repository:\n\n```shell\n./components/\n├── admission\n│   ├── configs\n│   │   ├── base\n│   │   ├── production\n│   │   └── staging\n│   └── controllers\n│       ├── base\n│       ├── production\n│       └── staging\n└── monitoring\n    ├── configs\n    │   ├── base\n    │   ├── production\n    │   └── staging\n    └── controllers\n        ├── base\n        ├── production\n        └── staging\n```\n\n### Runtime configuration\n\nIn the `clusters/\u003ccluster-name\u003e/runtime-info.yaml` ConfigMaps, the platform team sets which\nconfiguration overlay to use for all components and from which branch to reconcile the changes.\n\nFor example, the `staging` cluster is configured to reconcile the `main` branch of the `d1-infra`\nand `d1-apps` repositories, and to use the `staging` overlay for all components:\n\n```yaml\napiVersion: v1\nkind: ConfigMap\nmetadata:\n  name: flux-runtime-info\n  namespace: flux-system\n  labels:\n    toolkit.fluxcd.io/runtime: \"true\"\n  annotations:\n    kustomize.toolkit.fluxcd.io/ssa: \"Merge\"\ndata:\n  ENVIRONMENT: \"staging\"\n  GIT_BRANCH: \"main\"\n  CLUSTER_NAME: \"staging-1\"\n  CLUSTER_DOMAIN: \"preview1.example.com\"\n```\n\nThe `flux-runtime-info` ConfigMap is propagated to all namespaces in a cluster by a Kyverno policy,\nand is used by all Flux Kustomizations to perform substitutions when reconciling the components.\n\nThe platform team can extend the `flux-runtime-info` ConfigMap with additional fields such as\ncluster region, cloud provider ID, etc. \n\n## Onboarding tenants\n\nThe platform team is responsible for onboarding the applications defined as Flux HelmReleases in the\n[d1-apps repository](https://github.com/controlplaneio-fluxcd/d1-apps) and restricting the access\nto predefined Kubernetes namespaces.\n\n### Flux GitHub PAT for tenant components\n\nCreate a GitHub fine-grained personal access token for the Flux bot account with\nthe following permissions for the `d1-apps` repository:\n\n- `Administration` -\u003e `Access: Read-only`\n- `Commit statuses` -\u003e `Access: Read and write`\n- `Contents` -\u003e `Access: Read and write`\n- `Metadata` -\u003e `Access: Read-only`\n\nAfter the cluster is bootstrapped, the platform team can onboard tenant applications by creating\na Kubernetes secret in the `flux-system` namespace with the tenant's GitHub PAT:\n\n```shell\nexport APPS_GITHUB_TOKEN=\u003cFlux bot apps PAT\u003e\n\nflux create secret git flux-apps \\\n  --namespace=flux-system \\\n  --label=toolkit.fluxcd.io/tenant=apps \\\n  --url=https://github.com \\\n  --username=git \\\n  --password=$APPS_GITHUB_TOKEN\n```\n\nThe tenant GitHub PAT secret is propagated from the `flux-system` namespace to all namespaces\nwhere the tenant applications are running, using a Kyverno policy. When rotating the tenant GitHub PAT,\nupdating the `flux-apps` secret in the `flux-system` namespace will automatically propagate the new token\nto all tenant namespaces labeled with `toolkit.fluxcd.io/tenant: apps`.\n\n### Continuous Delivery for tenant applications\n\nFor each namespace belonging to a tenant, the platform team must define the Kubernetes\nnamespace, RBAC, Flux GitRepository and Kustomization custom resources under the\ntenant's directory. \n\nThe directory structure under\n[tenants/apps](https://github.com/controlplaneio-fluxcd/d1-fleet/tree/main/tenants/apps)\nmatches the components defined in the\n[d1-apps repository](https://github.com/controlplaneio-fluxcd/d1-apps/components).\n\nFor example, the `d1-fleet` repository contains the following definitions for the `backend` namespace:\n\n```shell\n./tenants/apps/components/backend/\n├── kustomization.yaml\n├── namespace.yaml\n├── rbac.yaml\n└── sync.yaml\n```\n\nWhich configures the reconciliation under a restricted service account for the `backend` components defined in the\n[d1-apps repository](https://github.com/controlplaneio-fluxcd/d1-apps/components/backend):\n\n```shell\n./components/backend/\n├── base\n│   ├── bitnamicharts.yaml\n│   ├── kustomization.yaml\n│   ├── memcached.yaml\n│   └── redis.yaml\n├── production\n│   ├── kustomization.yaml\n│   ├── memcached-values.yaml\n│   └── redis-values.yaml\n└── staging\n    ├── kustomization.yaml\n    ├── memcached-values.yaml\n    └── redis-values.yaml\n```\n\nChanges made by the dev team to the `d1-apps` repository in the `main` branch will\nbe automatically reconciled by the Flux controllers running in the staging cluster.\n\nChanges made by the dev team to the `d1-apps` repository in the `production` branch will\nbe automatically reconciled by the Flux controllers running in the production cluster fleet.\n\nThe dev team can make any changes inside the namespaces assigned by the platform team, but they\ncannot change any cluster-wide resources or the namespace itself.\n\n### Helm release automation for tenant applications\n\nThe staging cluster runs the Flux image automation controllers which automatically\nupdate the HelmRelease definitions in the `main` branch of the `d1-apps` repository\nbased on Flux image polices defined by the dev team.\n\nWhen a new chart version is pushed to the container registry, and if it matches the semver policy,\nFlux will update the HelmRelease YAML definitions and will push the changes to the `main` branch.\n\nAfter the changes are reconciled on staging, the dev team can promote the changes\nto the production clusters by merging the `main` branch into the `production` branch of the `d1-apps` repository.\n\nThe platform team is responsible for configuring a dedicated Kubernetes namespace for \nthe image policies and defining the Flux image update automation custom resources in the `d1-fleet` repository:\n\n```shell\n./tenants/apps/update/\n├── automation.yaml\n├── kustomization.yaml\n├── namespace.yaml\n├── rbac.yaml\n└── sync.yaml\n```\n\nThe above configuration will reconcile the image polices define in the\n[d1-apps repository](https://github.com/controlplaneio-fluxcd/d1-apps/components):\n\n```shell\n./update/\n├── backend-memcached.yaml\n├── backend-redis.yaml\n├── frontend-podinfo.yaml\n└── kustomization.yaml\n```\n\nThe dev team has full control over the image policies, and they are responsible for\ndefining the image update automation rules for their applications.\nThe platform team is responsible for setting up the infrastructure for running the\nFlux image automation controllers and their access to the dev team repository.\n\n## Bootstrap the production clusters\n\nMake sure to set the default context in your kubeconfig to your production cluster, then run the following:\n\n```shell\nexport GITHUB_TOKEN=\u003cFlux Bot PAT\u003e\nexport GITHUB_OWNER=\u003cYour GitHub org or user\u003e\n\n# Install the Flux Operator Helm chart\nhelm install flux-operator oci://ghcr.io/controlplaneio-fluxcd/charts/flux-operator \\\n  --namespace flux-system \\\n  --create-namespace\n\n# Create the flux-system Git secret with the Flux bot PAT\nflux create secret git flux-system \\\n  --url=https://github.com/${GITHUB_OWNER}/d1-fleet \\\n  --username=git \\\n  --password=$GITHUB_TOKEN\n\n# Apply the FluxInstance custom resource to bootstrap Flux\nkubectl apply -f clusters/prod-eu/flux-system/flux-instance.yaml\n\n# Onboard the apps tenant by creating the flux-apps secret with the apps GitHub PAT\nflux create secret git flux-apps \\\n  --namespace=flux-system \\\n  --label=toolkit.fluxcd.io/tenant=apps \\\n  --url=https://github.com \\\n  --username=git \\\n  --password=$GITHUB_TOKEN\n```\n\nFrom this point on, the Flux controllers will reconcile the cluster state with the desired state. Any changes\nto the `clusters/prod-eu` directory in the `d1-fleet` repository in the `main` branch will be automatically\napplied to the cluster. The apps and infra components will be reconciled from the `production` branch of the\n`d1-infra` and `d1-apps` repositories.\n\n### Monitoring\n\nTo monitor the reconciliation process, run the following commands in different terminals:\n\n```shell\nwatch flux get kustomizations --all-namespaces\nwatch kubectl get pods --all-namespaces\n```\n\nTo list all the managed resources by Flux, run:\n\n```shell\nflux tree ks flux-system\n```\n\nTo view the Flux events with the reconciliation status, run:\n\n```shell\nflux events -A\n```\n\n### Grafana dashboards\n\nTo access Grafana, start port forward in a separate shell:\n\n```shell\nkubectl -n monitoring port-forward svc/kube-prometheus-stack-grafana  3000:80\n```\n\nNavigate to http://localhost:3000 in your browser and login with user `admin` and password `flux`.\n\nFlux dashboards:\n\n- Reconciliation stats: `http://localhost:3000/d/flux-cluster/flux-cluster-stats`\n- Controller stats: `http://localhost:3000/d/flux-control-plane/flux-control-plane`\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontrolplaneio-fluxcd%2Fd1-fleet","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontrolplaneio-fluxcd%2Fd1-fleet","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontrolplaneio-fluxcd%2Fd1-fleet/lists"}