https://github.com/buroa/k8s-gitops
Fiancé-approved geeked homelab k8s cluster deployed on 🍏 Mac Minis with Talos Linux; automated via Flux, Renovate and GitHub Actions 🤖
https://github.com/buroa/k8s-gitops
apple flux gitops home-operations homelab k8s kubernetes kubesearch renovate selfhosted talos
Last synced: 5 months ago
JSON representation
Fiancé-approved geeked homelab k8s cluster deployed on 🍏 Mac Minis with Talos Linux; automated via Flux, Renovate and GitHub Actions 🤖
- Host: GitHub
- URL: https://github.com/buroa/k8s-gitops
- Owner: buroa
- License: wtfpl
- Created: 2023-01-09T14:29:27.000Z (over 2 years ago)
- Default Branch: master
- Last Pushed: 2024-04-13T23:48:20.000Z (about 1 year ago)
- Last Synced: 2024-04-14T00:29:18.106Z (about 1 year ago)
- Topics: apple, flux, gitops, home-operations, homelab, k8s, kubernetes, kubesearch, renovate, selfhosted, talos
- Language: YAML
- Homepage: https://ktwo.io
- Size: 37.9 MB
- Stars: 125
- Watchers: 4
- Forks: 10
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- Contributing: .github/CONTRIBUTING.md
- Funding: .github/FUNDING.yml
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Security: .github/SECURITY.md
Awesome Lists containing this project
README
### My _geeked_ homelab k8s cluster :wheel_of_dharma:
_... automated via [Flux](https://github.com/fluxcd/flux2), [Renovate](https://github.com/renovatebot/renovate) and [GitHub Actions](https://github.com/features/actions)_ :robot:
[](https://discord.gg/home-operations)
[](https://talos.dev)
[](https://kubernetes.io)
[](https://github.com/buroa/k8s-gitops/actions/workflows/renovate.yaml)
[](https://status.ktwo.io)
[](https://status.ktwo.io)
[](https://status.ktwo.io)
[](https://github.com/kashalls/kromgo)
[](https://github.com/kashalls/kromgo)
[](https://github.com/kashalls/kromgo)
[](https://github.com/kashalls/kromgo)
[](https://github.com/kashalls/kromgo)
[](https://github.com/kashalls/kromgo)
[](https://github.com/kashalls/kromgo)
---
## 📖 Overview
This is a repository for my home infrastructure and Kubernetes cluster. I try to adhere to Infrastructure as Code (IaC) and GitOps practices using tools like [Kubernetes](https://github.com/kubernetes/kubernetes), [Flux](https://github.com/fluxcd/flux2), [Renovate](https://github.com/renovatebot/renovate) and [GitHub Actions](https://github.com/features/actions).
---
## ⛵ Kubernetes
This semi hyper-converged cluster operates on [Talos Linux](https://github.com/siderolabs/talos), an immutable and ephemeral Linux distribution tailored for [Kubernetes](https://github.com/kubernetes/kubernetes), and is deployed on bare-metal [MS-01](https://store.minisforum.com/products/minisforum-ms-01) workstations. [Rook](https://github.com/rook/rook) supplies my workloads with persistent block, object, and file storage, while a separate server handles media file storage. The cluster is designed to enable a full teardown without any data loss.
There is a template at [onedr0p/cluster-template](https://github.com/onedr0p/cluster-template) if you want to follow along with some of the practices I use here.
### Core Components
- [actions-runner-controller](https://github.com/actions/actions-runner-controller): Self-hosted Github runners.
- [cert-manager](https://github.com/cert-manager/cert-manager): Creates SSL certificates for services in my cluster.
- [cilium](https://github.com/cilium/cilium): Internal Kubernetes container networking interface.
- [cloudflared](https://github.com/cloudflare/cloudflared): Enables Cloudflare secure access to my ingresses.
- [external-dns](https://github.com/kubernetes-sigs/external-dns): Automatically syncs ingress DNS records to a DNS provider.
- [external-secrets](https://github.com/external-secrets/external-secrets): Managed Kubernetes secrets using [1Password Connect](https://github.com/1Password/connect).
- [ingress-nginx](https://github.com/kubernetes/ingress-nginx): Kubernetes ingress controller using NGINX as a reverse proxy and load balancer.
- [multus](https://github.com/k8snetworkplumbingwg/multus-cni): Multi-homed pod networking.
- [rook](https://github.com/rook/rook): Distributed block storage for peristent storage.
- [sops](https://github.com/getsops/sops): Managed secrets for Kubernetes which are commited to Git.
- [spegel](https://github.com/spegel-org/spegel): Stateless cluster local OCI registry mirror.
- [tailscale](https://github.com/tailscale/tailscale): Private WireGuard based VPN.
- [volsync](https://github.com/backube/volsync): Backup and recovery of persistent volume claims.
### GitOps
[Flux](https://github.com/fluxcd/flux2) monitors my [kubernetes](./kubernetes) folder (see Directories below) and implements changes to my cluster based on the YAML manifests.
Flux operates by recursively searching the [kubernetes/apps](./kubernetes/apps) folder until it locates the top-level `kustomization.yaml` in each directory. It then applies all the resources listed in it. This `kustomization.yaml` typically contains a namespace resource and one or more Flux kustomizations. These Flux kustomizations usually include a `HelmRelease` or other application-related resources, which are then applied.
[Renovate](https://github.com/renovatebot/renovate) monitors my **entire** repository for dependency updates, automatically creating a PR when updates are found. When some PRs are merged, [Flux](https://github.com/fluxcd/flux2) applies the changes to my cluster.
### Directories
This Git repository contains the following directories under [kubernetes](./kubernetes).
```sh
📁 kubernetes # Kubernetes cluster defined as code
├─📁 apps # Apps deployed into my cluster grouped by namespace (see below)
├─📁 bootstrap # Flux installation
└─📁 flux # Main Flux configuration of repository
```
### Cluster layout
This is a high-level look how Flux deploys my applications with dependencies. Below there are 3 Flux kustomizations `postgres`, `postgres-cluster`, and `atuin`. `postgres` is the first app that needs to be running and healthy before `postgres-cluster` and once `postgres-cluster` is healthy `atuin` will be deployed.
```mermaid
graph TD;
id1>Kustomization: cluster] -->|Creates| id2>Kustomization: cluster-apps];
id2>Kustomization: cluster-apps] -->|Creates| id3>Kustomization: postgres];
id2>Kustomization: cluster-apps] -->|Creates| id5>Kustomization: postgres-cluster]
id2>Kustomization: cluster-apps] -->|Creates| id8>Kustomization: atuin]
id3>Kustomization: postgres] -->|Creates| id4[HelmRelease: postgres];
id5>Kustomization: postgres-cluster] -->|Depends on| id3>Kustomization: postgres];
id5>Kustomization: postgres-cluster] -->|Creates| id10[Postgres Cluster];
id8>Kustomization: atuin] -->|Creates| id9(HelmRelease: atuin);
id8>Kustomization: atuin] -->|Depends on| id5>Kustomization: postgres-cluster];
```
### Networking
Click to see a high-level network diagram
---
## 🌐 DNS
I have two instances of `external-dns` running in my cluster. The private DNS instance synchronizes DNS records with a `UDM Pro Max`, while the public DNS instance does the same with `Cloudflare`. This setup is managed by creating ingresses with specific ingress classes: `internal` for the private DNS and `external` for the public DNS. Both ingresses use the `external-dns.alpha.kubernetes.io/target` annotation to specify the target. The `external-dns` instances then syncs the DNS records to their respective platforms accordingly.
---
## 🔧 Hardware
Click to see my rack
| Device | Count | OS Disk Size | Data Disk Size | Ram | Operating System | Purpose |
|---------------------------|-------|-----------------|-----------------------------|------|------------------|-----------------|
| MS-01 (i9-13900H) | 3 | 1.92TB M.2 NVMe | 3.84TB U.2 NVMe (rook-ceph) | 96GB | Talos | Kubernetes |
| USW Pro Max 24 PoE | 1 | - | - | - | UniFi OS | 2.5G PoE Switch |
| USW Pro Aggregation | 1 | - | - | - | UniFi OS | 10G/25G Switch |
| USP PDU Pro | 1 | - | - | - | UniFi OS | PDU |
| UDM Pro Max | 1 | - | 2x16TB HDD | - | UniFi OS | Router & NVR |
| Synology NAS RS1221+ | 1 | - | 8x22TB HDD | 32GB | - | NFS |
| APC SMT15000RM2UNC | 1 | - | - | - | - | UPS |
| TESmart 8 Port KVM Switch | 1 | - | - | - | - | KVM |
| PiKVM (RasPi 4) | 1 | 64GB (SD) | - | 4GB | PiKVM (Arch) | KVM |
---
## ⭐ Stargazers
---
## 🤝 Gratitude and Thanks
Many thanks to my friend [@onedrop](https://github.com/onedr0p) and all the fantastic people who donate their time to the [Home Operations](https://discord.gg/home-operations) Discord community. Be sure to check out [kubesearch.dev](https://kubesearch.dev) for ideas on how to deploy applications or get ideas on what you may deploy.
---
## 📜 Changelog
See the latest [release](https://github.com/buroa/k8s-gitops/releases/latest) notes.
---
## 🔏 License
See [LICENSE](./LICENSE).