Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/frank-at-suse/vsphere_cilium_kubeproxy-less

Terraform plan for creating a multi-node kube-proxy -less RKE2 cluster w/ Cilium CNI on VMware vSphere
https://github.com/frank-at-suse/vsphere_cilium_kubeproxy-less

cilium haproxy kube-vip kubernetes linux rancher rancher-server rancher2 rke2 terraform vmware-vsphere

Last synced: about 6 hours ago
JSON representation

Terraform plan for creating a multi-node kube-proxy -less RKE2 cluster w/ Cilium CNI on VMware vSphere

Host: GitHub
URL: https://github.com/frank-at-suse/vsphere_cilium_kubeproxy-less
Owner: frank-at-suse
License: mpl-2.0
Created: 2022-09-21T12:53:46.000Z (about 2 years ago)
Default Branch: master
Last Pushed: 2023-10-06T16:04:15.000Z (about 1 year ago)
Last Synced: 2023-10-07T16:34:55.940Z (about 1 year ago)
Topics: cilium, haproxy, kube-vip, kubernetes, linux, rancher, rancher-server, rancher2, rke2, terraform, vmware-vsphere
Language: HCL
Homepage:
Size: 39.1 KB
Stars: 1
Watchers: 1
Forks: 2
Open Issues: 0
Metadata Files:
- Readme: README.md
- License: LICENSE

Awesome Lists containing this project

README

        # RKE2 Cluster with kube-proxy -less Cilium CNI & BBR Pod Congestion Control

![Rancher](https://img.shields.io/badge/rancher-%230075A8.svg?style=for-the-badge&logo=rancher&logoColor=white) ![Terraform](https://img.shields.io/badge/terraform-%235835CC.svg?style=for-the-badge&logo=terraform&logoColor=white) ![Kubernetes](https://img.shields.io/badge/kubernetes-%23326ce5.svg?style=for-the-badge&logo=kubernetes&logoColor=white)

## Reason for Being

This Terraform plan is for creating a kube-proxy -less multi-node RKE2 cluster using [Cilium CNI](https://docs.cilium.io/en/latest/).  The Cilium chart enables [BBR Pod Congestion Control](https://docs.cilium.io/en/latest/operations/performance/tuning/#bbr-congestion-control-for-pods) for greatly increased network performance as well. Also installed is **Rancher's Prometheus Operator Chart (`cluster_monitoring.tf`)** along with an HAProxy Ingress Grafana Dashboard to gain visibility into your Ingress traffic (this plan disables the built-in RKE2 NGiNX Ingress in favor of HAProxy because...reasons  `¯\_(ツ)_/¯` ). kube-vip makes a token appearance here to advertise the HAProxy Ingress Controller via ARP.

_This is a nice boiler-plate Terraform plan for a high-performing cluster that includes a very capable monitoring stack._

## Environment Prerequisites

- Functional Rancher Management Server with vSphere Cloud Credential

- vCenter >= 7.x and credentials with appropriate permissions (see )

- Virtual Machine Hardware Compatibility at Version >= 15

- Linux Kernel >= 5.18 (required for enabling [BBR Pod Congestion Control](https://docs.cilium.io/en/latest/operations/performance/tuning/#bbr-congestion-control-for-pods))

- Create the following in the files/ directory:

    | NAME | PURPOSE |

    |:-----|:--------|

    | .rancher-api-url      | URL for Rancher Management Server |

    | .rancher-bearer-token | API bearer token generated via Rancher UI |

    | .ssh-public-key       | SSH public key for additional OS user |

## Caveats

- [Cilium's Hubble UI](https://docs.cilium.io/en/latest/gettingstarted/hubble/) is disabled as it can be a [drag on performance](https://docs.cilium.io/en/latest/operations/performance/tuning/#hubble).  However, if you enjoy looking at groupings of rectangles connected with lines and _do_ want to enable Hubble, reference the RKE2 Cilium Helm chart [HERE](https://github.com/rancher/rke2-charts/tree/main/charts/rke2-cilium/rke2-cilium).

- If you don't want to run the **Rancher Prometheus Operator**, it can be uninstalled at any time simply by removing `cluster_monitoring.tf` from your working directory and re-applying the plan.  It is here as a demonstration/value-add, not a requirement of any kind.

- kube-vip is operating via ARP mode, so services published via LoadBalancer _will have traffic directed to a single node_.

- Unlike RKE2's "baked-in" NGiNX Ingress Controller, HAProxy's ingress is **not** FIPS 140-2 compliant.

## To Run

```bash

terraform apply

```

## Tested Versions

| SOFTWARE | VERSION | DOCS |

|:---------|:--------|:-----|

| kube-vip                    | 0.6.2            |  |

| Rancher Prometheus Operator | 102.0.2+up40.1.2 |  |

| Rancher Server              | 2.7.6            |  |

| Rancher Terraform Provider  | 3.1.1            |  |

| RKE2                        | 1.26.8+rke2r1    |  |

| Terraform                   | 1.4.6            |  |

| vSphere                     | 8.0.1.00300      |  |