https://github.com/kashalls/home-cluster
๐ณ In-house kubernetes cluster built with k3s and managed by flux (gitops).
https://github.com/kashalls/home-cluster
ansible home-assistant home-automation k3s k8s-at-home kubernetes node-red renovate
Last synced: 2 months ago
JSON representation
๐ณ In-house kubernetes cluster built with k3s and managed by flux (gitops).
- Host: GitHub
- URL: https://github.com/kashalls/home-cluster
- Owner: kashalls
- License: wtfpl
- Created: 2021-08-24T00:44:55.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2024-05-01T17:05:54.000Z (about 1 year ago)
- Last Synced: 2024-05-01T17:21:33.424Z (about 1 year ago)
- Topics: ansible, home-assistant, home-automation, k3s, k8s-at-home, kubernetes, node-red, renovate
- Language: HCL
- Homepage: https://discord.gg/home-operations
- Size: 17.6 MB
- Stars: 65
- Watchers: 3
- Forks: 1
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
Awesome Lists containing this project
README
# Kashall's Home Operations
[](https://discord.gg/home-operations)ย ย
[](https://www.talos.dev/)ย ย
[](https://www.talos.dev/)ย ย
[](https://github.com/kashalls/home-cluster/actions/workflows/renovate.yaml)
[](https://github.com/kashalls/kromgo/)ย ย ย
[](https://github.com/kashalls/kromgo/)ย ย ย
[](https://github.com/kashalls/kromgo/)ย ย ย
[](https://github.com/kashalls/kromgo/)ย ย ย
[](https://github.com/kashalls/kromgo/)ย ย ย
[](https://github.com/kashalls/kromgo/)ย ย ย
[](https://github.com/kashalls/kromgo/)
## What is this?
This is the repository I use to version control my kubernetes cluster I deploy and maintain at home. I currently use [Talos](https://www.talos.dev/) to provide a secure, minimal and immutable environment for Kubernetes. Previous iterations of this repository relied on Debian-based Operating Systems which can lead unwanted changes in the base system.
## How did you do this?
Thanks to [onedr0p](https://github.com/onedr0p), there is the [cluster template](https://github.com/onedr0p/flux-cluster-template) that allows you to easily get started with your own kubernetes cluster at home. You don't need to have multiple computers or a fancy setup to get one working.
If you're interested, you can also join the community [Home Operations](https://discord.gg/home-operations). Several people are involved daily and it makes for some interesting conversations.
### Directory Helper
This repository uses the following layout for [Kubernetes](./kubernetes/).
```sh
๐ bootstrap
โโโ ๐ helmfile.yaml # Helmreleases required to run bootstrap flux.
โโโ ๐ secrets.yaml.tpl # Secrets required to bootstrap flux.
๐ kubernetes
โโโ ๐ apps # Per-cluster application-specific configurations.
โโโ ๐ components # Flux & Talos configurations for setting up the cluster.
โโโ ๐ flux # Flux configuration, application repositories and more.
๐ talos
โโโ ๐ nodes # Override configurations for each individual node.
โโโ ๐ machineconfig.yaml.j2 # Base configuration for all nodes.
โโโ ๐ talos.env # Kubernetes and Talos Version Variables
๐ unifi # Configuration files for UniFi
๐ kubeconfig
๐ talosconfig
```
## โ๏ธ Cloud Dependencies
While most of my infrastructure and workloads are self-hosted I do rely upon the cloud for certain key parts of my setup. This saves me from having to worry about two things. (1) Dealing with chicken/egg scenarios and (2) services I critically need whether my cluster is online or not.
| Service | Use | Cost |
|---------------------------------------------------------|----------------------------------------------------------------|----------------|
| [1Password](https://1password.com/) | Secrets with [External Secrets](https://external-secrets.io/) | ~$55/yr |
| [Cloudflare](https://www.cloudflare.com/) | Domains, Workers, Pages, and R2 | ~$30/yr |
| [Backblaze B2](https://www.backblaze.com/cloud-storage) | Backups | $0.50/m |
| [GCP](https://cloud.google.com/) | Voice interactions with Home Assistant over Google Assistant | Free |
| [GitHub](https://github.com/) | Hosting this repository and continuous integration/deployments | Free |
| [Let's Encrypt](https://letsencrypt.org/) | Issuing SSL Certificates with Cert Manager | Free |
| [Migadu](https://migadu.com/) | Email Hosting | ~$20/yr |
| [Pushover](https://pushover.net/) | Kubernetes Alerts and application notifications | Free |
| [UniFi Site Manager](https://unifi.ui.com) | UniFi External Access Management | Free |
| | | Total: ~$10/mo |
---
## ๐ป Networking
### Networking Diagram
```mermaid
flowchart LR
A[["#quot;The Internet#quot;"]] -- 2Gbps โ 350Mbps โ --> B("UDM Pro Max");
B -- 10Gbps โ --> C("USW Pro Max 16")
C -- 10Gbps โ --> D["1x MS-01 Main (Talos)"]
C -- 10Gbps โ --> E["1x Storage (TrueNAS)"]
C -- 1Gbps โ --> F["4x Rasbian (Talos)"]
```
### Networks & Vlans
| Name | VLAN | Description |
|---------------------|------|-------------------------------------------------------------------------------------|
| Management | 1 | Servers + Network Management |
| Devices | 2 | Wireless Devices and Workstations |
| IoT | 3 | Small devices that *could* be compromised, so they don't get to talk to each other. |
| Services | 4 | No DHCP, Simply a network for Cluster BGP |
| "I Don't Trust You" | 86 | Non-affiliated organization issued devices (school or work devices) |
### ๐ DNS
UniFi released a new feature update with UniFi routers that allow you to create custom dns records to be served to the whole network. I wrote [External DNS Unifi Webhook](https://github.com/kashalls/external-dns-unifi-webhook) to allow [External DNS](https://github.com/kubernetes-sigs/external-dns/) to gather service and ingress hosts from my clusters and deploy the records to my routers local dns server without any extra local resolvers or moving parts.
---
## ๐ง Hardware
Click to see the rack!
Updated 05/25/2024
| Device | Count | OS Disk Size | Data Disk Size | Ram | Operating System | Purpose |
|-------------------------|-------|--------------|----------------------|------|------------------|-------------------|
| UDM Pro | 1 | - | - | - | UniFi OS | Router |
| USW 16 Pro Max | 1 | - | - | - | UniFi OS | Switching |
| U6-LR | 1 | - | - | - | - | Office AP |
| UAP-AC-Pro | 1 | - | - | - | - | Dining Room AP |
| USP-PDU-Pro | 1 | - | - | - | - | Rack PDU |
| MS-01 | 1 | 1TB NVMe | 2x1TB NVMe | 32GB | Talos | Main Cluster |
| Fran | 1 | 2x1TB SSD | 5x8TB (raidz2) | 64GB | Debian | Storage Cluster |
| JetKVM | 1 | 16GB (Flash) | - | - | JetKVM | Network KVM |
| APC Back-Ups 1500 | 1 | - | - | - | - | UPS |
---
## โญ Stargazers
[](https://star-history.com/#kashalls/home-cluster&Date)
---
## Inspiration
Thanks to all the people who donate their time to the [Home Operations](https://discord.gg/home-operations) community.
Special thanks to: [แชัฮฝฮนฮท แทฯ
ะฝส](https://github.com/onedr0p/home-cluster), [Bแดสษดแด
Sแดสแดสษขแดสs](https://github.com/bjw-s/k8s-gitops), and [Toboshii Nakama](https://github.com/toboshii/home-cluster) for their assistance.
Check out [kubesearch.dev](https://kubesearch.dev) to see what other users are running in their kubernetes home labs!