{"id":19806897,"url":"https://github.com/rancher-sandbox/cluster-api-provider-harvester","last_synced_at":"2026-05-31T02:03:54.279Z","repository":{"id":197970753,"uuid":"699775892","full_name":"rancher-sandbox/cluster-api-provider-harvester","owner":"rancher-sandbox","description":"A Cluster API Infrastructure Provider for Harvester","archived":false,"fork":false,"pushed_at":"2024-04-29T14:57:14.000Z","size":717,"stargazers_count":14,"open_issues_count":12,"forks_count":2,"subscribers_count":5,"default_branch":"main","last_synced_at":"2024-04-30T11:38:52.705Z","etag":null,"topics":["cluster-api","harvester","infrastructure","kubernetes","kubevirt","provider"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rancher-sandbox.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-10-03T10:12:41.000Z","updated_at":"2024-05-06T16:31:12.126Z","dependencies_parsed_at":null,"dependency_job_id":"454339c8-801c-4315-9a8a-79d7ffd42e28","html_url":"https://github.com/rancher-sandbox/cluster-api-provider-harvester","commit_stats":null,"previous_names":["rancher-sandbox/cluster-api-provider-harvester"],"tags_count":2,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher-sandbox%2Fcluster-api-provider-harvester","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher-sandbox%2Fcluster-api-provider-harvester/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher-sandbox%2Fcluster-api-provider-harvester/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rancher-sandbox%2Fcluster-api-provider-harvester/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rancher-sandbox","download_url":"https://codeload.github.com/rancher-sandbox/cluster-api-provider-harvester/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224246011,"owners_count":17279649,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster-api","harvester","infrastructure","kubernetes","kubevirt","provider"],"created_at":"2024-11-12T09:08:50.081Z","updated_at":"2026-05-31T02:03:54.271Z","avatar_url":"https://github.com/rancher-sandbox.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Cluster API Provider Harvester (CAPHV)\n\n\u003e Fork of [rancher-sandbox/cluster-api-provider-harvester](https://github.com/rancher-sandbox/cluster-api-provider-harvester) with Harvester v1.7+/v1.8+ compatibility and production-ready features.\n\n## Overview\n\nCAPHV is a [Cluster API](https://cluster-api.sigs.k8s.io/) Infrastructure Provider for provisioning Kubernetes clusters on [Harvester HCI](https://harvesterhci.io/).\n\nThis fork adds significant enhancements over upstream v0.1.6:\n\n| Feature | Upstream v0.1.x | This fork (v0.3.x) |\n|---------|----------------|-------------------|\n| Harvester compatibility | v1.2.0 | v1.7.x + v1.8.x |\n| Multi-disk VMs | Single disk only | Multiple disks (image + storageClass) |\n| IP allocation | Manual / DHCP | Automatic from Harvester IPPool or DHCP |\n| Cloud-init | Basic | Network-config v1 (SLES), multi-NIC, static IP + DHCP |\n| Cloud provider bootstrap | Manual fixes needed | Automatic (hostNetwork, RBAC, tolerations) |\n| Node initialization | Manual providerID | Automatic from management cluster |\n| etcd cleanup | Manual | Automatic on CP machine deletion |\n| Validating webhooks | None | HarvesterMachine + HarvesterCluster |\n| Boot order | Not supported | Configurable per-disk |\n| VM runStrategy | Deprecated `spec.running` | `spec.runStrategy: Always` |\n| MachineHealthCheck | Untested | Tested, full auto-remediation |\n| Rolling K8s upgrade | Untested | Tested (CP + workers) |\n| E2E tests | Kubebuilder scaffold only | 18 integration tests (live cluster) |\n| ClusterClass | Generic example only | Production-ready with vmNetworkConfig, IPPool, sshUser |\n| CLI generator | None | `caphv-generate` script (~30-line clusters) |\n| Fleet/CAAPF addons | Not supported | CSI/CNI via Fleet GitOps with per-cluster CNI tuning |\n| Helm chart | None | Full chart with webhook + ClusterClass support |\n\n## Prerequisites\n\n- Harvester HCI v1.7.x or v1.8.x cluster\n- Management cluster (RKE2 recommended) with:\n  - Cluster API Core **v1.12.x** (serves `cluster.x-k8s.io/v1beta2`)\n  - RKE2 Bootstrap + ControlPlane providers **v0.24.x+** (validated against v0.25.0)\n  - Rancher Turtles **v0.26.x** (Rancher Manager 2.14.x ships Turtles by default,\n    but the RKE2 CAPIProviders must be enabled explicitly — see below)\n  - cert-manager (required for webhooks, default-on)\n- Harvester identity Secret (kubeconfig for the target Harvester cluster)\n- SSH KeyPair created on Harvester\n- VM image uploaded to Harvester (SLES 15 SP7 or openSUSE Leap 15.6 recommended)\n- IPPool configured on Harvester (for automatic IP allocation)\n\n\u003e **Upgrading from v0.2.x**: see [docs/migration-v0.2-to-v0.3.md](docs/migration-v0.2-to-v0.3.md)\n\u003e — v0.3.0 requires the CAPI v1.12 / v1beta2 ecosystem and is **not backward-compatible**\n\u003e with managers running CAPI v1.10.\n\n### Enabling RKE2 providers under Rancher Turtles\n\nRancher Turtles only deploys `cluster-api` core by default. To use CAPHV, enable\nthe RKE2 providers explicitly:\n\n```yaml\napiVersion: turtles-capi.cattle.io/v1alpha1\nkind: CAPIProvider\nmetadata:\n  name: rke2-bootstrap\n  namespace: rke2-bootstrap-system\nspec:\n  name: rke2\n  type: bootstrap\n---\napiVersion: turtles-capi.cattle.io/v1alpha1\nkind: CAPIProvider\nmetadata:\n  name: rke2-control-plane\n  namespace: rke2-control-plane-system\nspec:\n  name: rke2\n  type: controlPlane\n```\n- Harvester identity Secret (kubeconfig for the target Harvester cluster)\n- SSH KeyPair created on Harvester\n- VM image uploaded to Harvester (SLES 15 SP7 recommended)\n- IPPool configured on Harvester (for automatic IP allocation)\n\n## Installation\n\n### Option 1: CAPIProvider via Rancher Turtles (recommended for production)\n\n```yaml\napiVersion: turtles-capi.cattle.io/v1alpha1\nkind: CAPIProvider\nmetadata:\n  name: harvester\n  namespace: caphv-system\nspec:\n  name: harvester\n  type: infrastructure\n  version: v0.3.0\n  fetchConfig:\n    url: https://github.com/rancher-sandbox/cluster-api-provider-harvester/releases/download/v0.3.0/infrastructure-components.yaml\n  configSecret:\n    name: caphv-variables\n```\n\nSee [docs/operations.md](docs/operations.md) for full CAPIProvider deployment, upgrade, and migration instructions.\n\n### Option 2: Helm Chart (legacy)\n\n\u003e **Note**: The Helm chart is maintained for compatibility but is not the recommended\n\u003e installation method. Prefer CAPIProvider via Rancher Turtles (Option 1) or `clusterctl init`\n\u003e for production deployments. The chart may be removed in a future release.\n\n```bash\n# Without webhooks\nhelm install caphv chart/caphv/ \\\n  -n caphv-system --create-namespace \\\n  --set image.repository=ghcr.io/rancher-sandbox/cluster-api-provider-harvester \\\n  --set image.tag=v0.3.0\n\n# With webhooks (requires cert-manager)\nhelm install caphv chart/caphv/ \\\n  -n caphv-system --create-namespace \\\n  --set image.repository=ghcr.io/rancher-sandbox/cluster-api-provider-harvester \\\n  --set image.tag=v0.3.0 \\\n  --set webhooks.enabled=true \\\n  --set webhooks.certManager.enabled=true\n```\n\n### Option 3: Kustomize\n\n```bash\n# Build and push the image\nmake docker-build docker-push IMG=ghcr.io/rancher-sandbox/cluster-api-provider-harvester:v0.3.0\n\n# Deploy\nmake deploy IMG=ghcr.io/rancher-sandbox/cluster-api-provider-harvester:v0.3.0\n```\n\n### Option 4: Manual (standalone manifests)\n\n```bash\nkubectl apply -f out/infrastructure-components.yaml\n```\n\n## Quick Start (ClusterClass — recommended)\n\nUsing ClusterClass reduces cluster creation from ~200 lines to ~30 lines of YAML.\n\n### 1. Install the ClusterClass (once per management cluster)\n\n```bash\n# Via Helm (with controller)\nhelm install caphv chart/caphv/ \\\n  -n caphv-system --create-namespace \\\n  --set clusterClass.enabled=true\n\n# Or standalone\nkubectl apply -f templates/clusterclass/rke2/clusterclass-harvester-rke2.yaml\n```\n\n### 2. Generate cluster manifests with the CLI\n\n```bash\n# Generate all manifests\nbin/caphv-generate \\\n  --name my-cluster \\\n  --image \"default/my-vm-image.qcow2\" \\\n  --ssh-keypair \"default/my-ssh-key\" \\\n  --network \"default/my-vm-network\" \\\n  --gateway 10.0.0.1 \\\n  --subnet-mask 255.255.255.0 \\\n  --ip-pool my-ip-pool \\\n  --dns 10.0.0.53 \\\n  --harvester-kubeconfig ~/.kube/harvester.yaml \\\n  \u003e cluster.yaml\n\n# Or interactive mode\nbin/caphv-generate --interactive\n\n# Apply\nkubectl apply -f cluster.yaml\n# Or directly: bin/caphv-generate [...] --apply\n```\n\nThe CLI generates: Namespace, Secret, Cluster (topology), ConfigMaps (CCM/CSI/Calico), ClusterResourceSets, and MachineHealthCheck.\n\n### Fleet Mode (optional — GitOps addon management)\n\nWith CAAPF installed, addons can be managed via Fleet instead of CRS:\n\n```bash\nbin/caphv-generate \\\n  --name my-cluster \\\n  --cni calico --cni-mtu 1450 --cni-encapsulation VXLAN \\\n  --pod-cidr 10.244.0.0/16 \\\n  --fleet-addon-repo https://my-gitea/org/caphv-fleet-addons.git \\\n  --image \"default/my-vm-image.qcow2\" \\\n  --ssh-keypair \"default/my-ssh-key\" \\\n  --network \"default/my-vm-network\" \\\n  --gateway 10.0.0.1 --subnet-mask 255.255.255.0 \\\n  --ip-pool my-ip-pool \\\n  --harvester-kubeconfig ~/.kube/harvester.yaml \\\n  --apply\n```\n\nSee [docs/fleet-addons.md](docs/fleet-addons.md) for full documentation.\n\n### 3. Monitor cluster creation\n\n```bash\nkubectl get cluster,machine,harvestermachine -n my-cluster\n```\n\n## User Experience Summary\n\n### Prerequisites (one-time setup)\n\nOn the management cluster:\n- Rancher + Turtles installed\n- CAPHV deployed via Helm (`clusterClass.enabled=true`)\n- Rancher `cacerts` setting configured (required for Turtles strict TLS mode with external TLS termination)\n- Harvester kubeconfig available locally\n\n### Create a cluster\n\n**Interactive mode (guided):**\n```bash\ncaphv-generate --interactive\n```\nThe script asks ~15 questions with sensible defaults, then generates and applies everything.\n\n**Flags mode (scriptable):**\n```bash\ncaphv-generate \\\n  --name my-cluster \\\n  --cp-replicas 3 --worker-replicas 2 \\\n  --image \"default/my-vm-image.qcow2\" \\\n  --ssh-keypair \"default/my-ssh-key\" \\\n  --network \"default/my-vm-network\" \\\n  --gateway 10.0.0.1 --subnet-mask 255.255.255.0 \\\n  --ip-pool my-ip-pool --dns 10.0.0.53 \\\n  --harvester-kubeconfig ~/.kube/harvester.yaml \\\n  --apply\n```\n\n### What happens automatically (~16 min)\n\n1. **Namespace** created\n2. **Secret** with Harvester kubeconfig injected\n3. **ClusterClass** + templates deployed in the namespace\n4. **Cluster topology** created — CAPI orchestrates everything:\n   - VMs created on Harvester (IPs allocated from IPPool)\n   - RKE2 bootstrap (control plane then workers)\n   - Cloud-init with static IP, iptables, SSH\n   - Cloud provider + CSI Harvester installed via ClusterResourceSets\n   - MachineHealthCheck active (auto-remediation)\n5. **Rancher** detects the cluster (auto-import label) — deploys agent — cluster visible in the UI\n\n### Result\n\n- Fully functional Kubernetes cluster (RKE2)\n- Visible and manageable in Rancher UI\n- Auto-remediation: if a VM dies, it is automatically recreated (~9 min)\n- Rolling upgrade: change the K8s version in the Cluster spec — rolling update CP then workers\n\n### Day 2 Operations\n\n- **Scale**: modify `replicas` in the Cluster spec\n- **Upgrade K8s**: modify `version` in the Cluster spec\n- **Delete**: `kubectl delete cluster my-cluster -n my-namespace` — everything is cleaned up (VMs, PVCs, secrets)\n\n## Quick Start (manual — full control)\n\n### 1. Create the identity Secret\n\n```bash\nkubectl create secret generic hv-identity-secret \\\n  -n \u003cnamespace\u003e \\\n  --from-file=kubeconfig=\u003cpath-to-harvester-kubeconfig\u003e\n```\n\n### 2. Create a Cluster\n\n```yaml\napiVersion: cluster.x-k8s.io/v1beta1\nkind: Cluster\nmetadata:\n  name: my-cluster\n  namespace: my-namespace\nspec:\n  clusterNetwork:\n    pods:\n      cidrBlocks: [10.52.0.0/16]\n    services:\n      cidrBlocks: [10.53.0.0/16]\n  controlPlaneRef:\n    apiVersion: controlplane.cluster.x-k8s.io/v1beta1\n    kind: RKE2ControlPlane\n    name: my-cluster-cp\n  infrastructureRef:\n    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1\n    kind: HarvesterCluster\n    name: my-cluster-hv\n---\napiVersion: infrastructure.cluster.x-k8s.io/v1alpha1\nkind: HarvesterCluster\nmetadata:\n  name: my-cluster-hv\n  namespace: my-namespace\nspec:\n  targetNamespace: default\n  identitySecret:\n    name: hv-identity-secret\n    namespace: my-namespace\n  loadBalancerConfig:\n    ipamType: pool\n  vmNetworkConfig:\n    gateway: \"10.0.0.1\"\n    subnetMask: \"255.255.255.0\"\n    ipPoolRef: default/my-ip-pool\n```\n\n### 3. Define Machine Templates\n\n```yaml\napiVersion: infrastructure.cluster.x-k8s.io/v1alpha1\nkind: HarvesterMachineTemplate\nmetadata:\n  name: my-cluster-machine\n  namespace: my-namespace\nspec:\n  template:\n    spec:\n      cpu: 2\n      memory: \"4Gi\"\n      sshUser: sles\n      sshKeyPair: default/my-ssh-key\n      volumes:\n        - volumeType: image\n          imageName: default/my-vm-image.qcow2\n          volumeSize: \"40Gi\"\n          bootOrder: 1\n        - volumeType: storageClass     # optional: additional data disk\n          storageClass: longhorn\n          volumeSize: \"10Gi\"\n      networks:\n        - default/production\n```\n\n### 4. Create Control Plane + Workers\n\nSee [templates/](templates/) for complete RKE2 cluster template examples.\n\n## Architecture\n\n```\nManagement Cluster (RKE2)\n├── CAPI Core Controller\n├── RKE2 Bootstrap Controller\n├── RKE2 ControlPlane Controller\n├── CAPHV Controller  ◄── this project\n│   ├── HarvesterCluster reconciler\n│   ├── HarvesterMachine reconciler\n│   │   ├── IP allocation from IPPool\n│   │   ├── VM creation (multi-disk, cloud-init, static IP)\n│   │   ├── Cloud provider bootstrap (hostNetwork fix)\n│   │   ├── Node init (providerID + taint removal)\n│   │   └── etcd cleanup on CP deletion\n│   └── Validating webhooks (optional)\n└── Rancher Turtles (optional, auto-import)\n\nHarvester HCI (target)\n├── VMs (created by CAPHV)\n├── IPPool (IP allocation)\n├── VM Images (boot disks)\n└── Longhorn (storage)\n```\n\n## Configuration Reference\n\n### HarvesterCluster\n\n| Field | Type | Required | Description |\n|-------|------|----------|-------------|\n| `spec.targetNamespace` | string | Yes | Namespace on Harvester for VMs |\n| `spec.identitySecret.name` | string | Yes | Secret containing Harvester kubeconfig |\n| `spec.identitySecret.namespace` | string | Yes | Namespace of identity secret |\n| `spec.loadBalancerConfig.ipamType` | string | Yes | `pool` or `dhcp` |\n| `spec.vmNetworkConfig.gateway` | string | Yes* | Gateway IP (*required for pool IPAM) |\n| `spec.vmNetworkConfig.subnetMask` | string | Yes* | Subnet mask (e.g. \"255.255.0.0\") |\n| `spec.vmNetworkConfig.ipPoolRef` | string | No | Reference to Harvester IPPool |\n\n\u003e **DHCP mode**: If `vmNetworkConfig` is omitted and no machine-level `networkConfig` is set, all VM NICs will use DHCP automatically. No IPPool or static IP configuration is needed.\n\n### HarvesterMachine\n\n| Field | Type | Required | Description |\n|-------|------|----------|-------------|\n| `spec.cpu` | int | Yes | Number of CPU cores (must be \u003e 0) |\n| `spec.memory` | string | Yes | Memory (e.g. \"4Gi\") |\n| `spec.sshUser` | string | Yes | SSH user for cloud-init |\n| `spec.sshKeyPair` | string | Yes | Harvester SSH KeyPair reference |\n| `spec.volumes` | []Volume | Yes | At least one volume required |\n| `spec.networks` | []string | Yes | At least one network required |\n| `spec.volumes[].volumeType` | string | Yes | `image` or `storageClass` |\n| `spec.volumes[].imageName` | string | For image | Harvester VM image (namespace/name) |\n| `spec.volumes[].storageClass` | string | For SC | Storage class for blank disk |\n| `spec.volumes[].volumeSize` | string | Yes | Disk size (e.g. \"40Gi\") |\n| `spec.volumes[].bootOrder` | int | No | Boot priority (1 = first) |\n\n## Monitoring\n\nCAPHV exposes custom Prometheus metrics (`caphv_*` namespace) via the controller-runtime metrics endpoint (port 8080, protected by kube-rbac-proxy).\n\nA ServiceMonitor is included in the kustomize build. A ready-to-import Grafana dashboard is at `config/grafana/caphv-dashboard.json`.\n\nKey metrics: `caphv_machine_create_total`, `caphv_machine_creation_duration_seconds`, `caphv_machine_status`, `caphv_ippool_allocations_total`, `caphv_cluster_ready`, `caphv_etcd_member_remove_total`, `caphv_node_init_duration_seconds`.\n\nSee [docs/operations.md](docs/operations.md) for the full metrics list and alerting recommendations.\n\n## Documentation\n\n- [Operations Guide](docs/operations.md) — installation via CAPIProvider, cluster lifecycle, monitoring, backup/DR\n- [Fleet Addons Guide](docs/fleet-addons.md) — Fleet/CAAPF addon management for CSI and CNI\n- [Troubleshooting](docs/troubleshooting.md) — IPPool, cloud-init, DHCP, Turtles/Rancher, VM creation, etcd\n\n## E2E Tests\n\nIntegration tests run against a live Harvester + CAPI cluster:\n\n```bash\n./test/e2e/run-e2e.sh              # Run all (18 tests, ~30min)\n./test/e2e/run-e2e.sh webhook      # Validation tests (~10s)\n./test/e2e/run-e2e.sh scale        # Scale up/down (~7min)\n./test/e2e/run-e2e.sh multidisk    # Multi-disk VM (~7min)\n./test/e2e/run-e2e.sh remediation  # MHC auto-remediation (~14min)\n```\n\n## Building\n\n```bash\n# Build binary\nmake build\n\n# Build container image\nmake docker-build IMG=ghcr.io/rancher-sandbox/cluster-api-provider-harvester:v0.3.0\n\n# Run unit tests\nmake test\n```\n\n## Release History\n\n| Version | Date | Key changes |\n|---------|------|-------------|\n| v0.3.0 | 2026-05-30 | CAPI v1.12 / v1beta2 ecosystem migration: cluster-api v1.12.x, controller-runtime v0.22.5, k8s.io v0.34, metav1.Condition, RKE2 templates v1beta2. Validated against Harvester v1.8 + Rancher 2.14 + Turtles 0.26 |\n| v0.2.9 | 2026-04-15 | Supply chain hardening: cosign keyless signing, SLSA provenance, SBOM, GitHub Actions SHA-pinning, hadolint, least-privilege workflow permissions |\n| v0.2.8 | 2026-03-16 | CAPI contract compliance fixes, kustomize verify, v1beta2 readiness docs, Helm chart deprecation note |\n| v0.2.7 | 2026-03-10 | Code quality fixes for SURE-11421 review: kustomize modernization, finalizer naming conventions, context propagation |\n| v0.2.6 | 2026-03-09 | CSI decoupling, Fleet label automation, Fleet CSI bundle |\n| v0.2.5 | 2026-03-08 | Fleet/CAAPF addon management, CNI configuration flags |\n| v0.2.4 | 2026-03-08 | CAPIProvider in Turtles, P0 milestone complete |\n| v0.2.3 | 2026-03-07 | DHCP VM support, multi-NIC cloud-init |\n| v0.2.1 | 2026-03-06 | ClusterClass (harvester-rke2), CLI generator (caphv-generate), Helm ClusterClass option |\n| v0.2.0 | 2026-03-06 | Harvester v1.7.1, multi-disk, IPPool, webhooks, auto-remediation, e2e tests |\n| v0.1.6 | 2024-xx-xx | Upstream: initial CAPI contract, single disk, DHCP only |\n\n## License\n\nApache License 2.0 - See [LICENSE](LICENSE) for details.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Francher-sandbox%2Fcluster-api-provider-harvester","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Francher-sandbox%2Fcluster-api-provider-harvester","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Francher-sandbox%2Fcluster-api-provider-harvester/lists"}