{"id":18689022,"url":"https://github.com/nicklasfrahm/infrastructure","last_synced_at":"2025-04-12T05:38:39.903Z","repository":{"id":37556596,"uuid":"421646445","full_name":"nicklasfrahm/infrastructure","owner":"nicklasfrahm","description":"A repository to automate the configuration of my local infrastructure.","archived":false,"fork":false,"pushed_at":"2025-02-16T22:58:18.000Z","size":732,"stargazers_count":8,"open_issues_count":19,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-04-12T05:38:33.951Z","etag":null,"topics":["hybrid-cloud","infrastructure-as-code"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/nicklasfrahm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-27T02:08:29.000Z","updated_at":"2025-02-16T22:58:22.000Z","dependencies_parsed_at":"2023-10-22T23:20:28.773Z","dependency_job_id":"a838f275-cc2e-4602-a7ac-877878865574","html_url":"https://github.com/nicklasfrahm/infrastructure","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicklasfrahm%2Finfrastructure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicklasfrahm%2Finfrastructure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicklasfrahm%2Finfrastructure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/nicklasfrahm%2Finfrastructure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/nicklasfrahm","download_url":"https://codeload.github.com/nicklasfrahm/infrastructure/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248525168,"owners_count":21118616,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hybrid-cloud","infrastructure-as-code"],"created_at":"2024-11-07T10:39:43.516Z","updated_at":"2025-04-12T05:38:39.879Z","avatar_url":"https://github.com/nicklasfrahm.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Infrastructure\n\nA repository to automate the configuration of my local infrastructure.\n\nMy local infrastructure is my homelab, which I primarily use to learn about state-of-the-art technologies as well as to run applications for potential startup incubations.\n\nAs part of this repository you may find interesting learnings and insights that I came across when developing my homelab and its automation. Automating my infrastructure allows me to deploy workloads more efficiently and debug issues more quickly.\n\n## History 📜\n\nI have been running a homelab since 2014, where it started with a single [Cubietech Cubietruck][website-cubietruck]. It served as a NAS, LEMP server, and was a great introduction for me into Linux. Over time the setup grew to include other single board computers, such as the [Hardkernel Odroid XU4][website-odroidxu4].\n\nSince 2018 I started to make significant changes by automating the configuration of my router and my manually provisioned servers via [Ansible][website-ansible]. Using it allowed me to learn the _classic way_ of Baremetal provisioning. It provided a solid tool to get me started with the deployment and operation of [Kubernetes][website-kubernetes] clusters.\n\nNow, I am in the process of upgrading my infrastructure to become cloud-native and fully automated. The goal is to provide a RESTful API to provision baremetal machines from scratch, configure networking, and bootstrap and autoscale Kubernetes clusters.\n\n## Networking 🔌\n\nBelow, I describe the network setup of my Homelab. Some parts of it are already implemented, while others are still being conceptualized.\n\n### DNS 🌍\n\n**Status:** 🟢 Operational\n\nI register my domains with [Namecheap][website-namecheap], because they often provide good discounts on variety of domain names. Below, you may find a list of limitations that I ran into, which is the reason why I decided to only use them as a registrar, but not as a DNS provider.\n\n- **Limited usability in CI**  \n  Their API requires a manually administrated allow-list of trusted IPs. The allow list they offer does not allow CIDR notation IP ranges, e.g. `0.0.0.0/24`. The issue here is that [GitHub Actions'][website-github-actions] hosted runners do not have a static set of IPs. As I want to administrate my entire infrastructure via GitHub actions and pipelines, a self-host runner is not an option for me.\n\n- **Slow API development**  \n  The [Namecheap API][website-namecheap-api] is well-documented, but is not RESTful and uses XML. This setup is rather old-fashioned and makes it rather uncomfortable to use. Their customers have requested a more modern API surface, but so far the development of the API focuses on pure maintenance. Having a lightweight, easy-to-use API is critical when relying on it to build your products.\n\nTherefore I decided to use [Google Cloud DNS][website-gcp-dns] for the automated administration of my DNS records. The deployment of my DNS infrastructure is automated using [GitHub Actions][website-github-actions] and [Pulumi][website-pulumi], because it allows me to write my _Infrastructure as Code_ using my favourite programming language [Go][website-go].\n\nFor more information you may want to check out the following files and directories:\n\n- `main.go`: The entry point to my Pulumi program as a general overview.\n- `pkg/dns/`: The configuration of DNSSEC and my DNS records.\n\n### Routing 🔀\n\n**Status (v3):** 💡 Planning\n\n**Keywords:** BGP, Kubernetes Operator, [Kube-Router][website-kube-router], SDN  \n**Motivation:** Automation, ease-of-operation, client source IPs\n\n**Status (v2):** 🟢 Operational\n\nThis setup uses a [Seeed Dual Gigabit Ethernet NIC Carrier Board][shop-seeed-router] with a [Raspberry Pi Compute Module 4][shop-rpi-cm4] as a router. This setup allows it to run a Kubernetes cluster on my router and therefore on my network edge, which is not possible with my previous setup. This setup allows to build a custom orchestration layer for software defined networking via VLANs.\n\n**Status (v1):** 🔴 End of life\n\nIn this setup I was running a [Ubiquity Edgerouter-X][website-erx] (ER-X) with port-forwarding towards a single baremetal [k3s cluster][website-k3s] on a seperate machine. The ER-X is fine, but not great for automation and dynamic configuration.\n\nIn this setup I was had two subnets, **userspace** (`192.168.0.1/24`) and **homelab** (`172.16.0.1/22`), where DHCP was only enabled in the **userspace** subnet. This setup allowed me to maintain network connectivity for **userspace** traffic while performing testing or configuration changes in my **homelab**. The configuration was hardcoded to specific ports on my router.\n\n### VLANs 🛡️\n\n**Status:** 🟢 Operational\n\nThe table below describes the set of manually assigned VLANs. The configuration of the DHCPv4 server and my network interfaces via [netplan.io][docs-netplan] is based on the following resources:\n\n- [Ubuntu community article about `isc-dhcp-server`][article-ubuntu-dhcp]\n- [How to make a simple router on a Ubuntu server][medium-ubuntu-router]\n- [`netplan.io` examples][docs-netplan-examples]\n\n| VLAN ID | CIDR               | DHCP | Name       | Description                                                        |\n| ------- | ------------------ | ---- | ---------- | ------------------------------------------------------------------ |\n| 1       | `none`             | No   | default    | A network without any gateway to isolate unassigned hosts.         |\n| 10      | `172.16.0.0/22`    | Yes  | management | A network for the configuration and management of network devices. |\n| 4000    | `192.168.254.0/22` | No   | homelab    | A network for experimental deployments.                            |\n| 4090    | `192.168.255.0/24` | Yes  | userspace  | A home network for WiFi and other domestic traffic.                |\n\n### Firewall 🔥\n\n**Status (v1):** 🟢 Operational\n\nI chose to use `nftables` as it provides a modern and declarative API. In the future the configuration should be managed via a software layer that performs automatic reconciliation.\n\nFor anyone replicating this setup, it might be worth noting that there was one major roadblock during the implementation, which took me a week to figure out. Make sure to configure MTU clamping as described in the reference below or shown in `configs/cloud-init/seeed-rtcm4-0/user-data`. The symptoms of this missing configuration option are that small TCP and UDP packets will be forwarded correctly, such as the `client hello` of a TLS handshake or DNS queries, but regular TCP traffic will just be dropped. This in turn causes websites to fail loading. If you are able to open [`https://api.ipify.org`][api-ipify] in your browser, but not [`https://wikipedia.org`][wikipedia] it may very well be related to this.\n\n- [Quick reference for `nftables` from its wiki][wiki-nftables-quickstart]\n- [RHEL documentation `nftables` chapter][rhel-docs-nftables-quickstart]\n- [RHEL documentation explaining NAT configuration][rhel-docs-nftables-nat]\n- [Blog post for simple firewall setup][website-cryptsus-simple-firewall]\n- [Gento wiki showing `nftables` configuration examples][wiki-gentoo-nftables-examples]\n- [Arch Linux wiki explaining `nftables` basics][wiki-arch-nftables-basics]\n- [MTU clamping via packet header mangling][wiki-nftables-mtu-clamping]\n\n### Load balancing 🔁\n\n**Status:** 🔵 Limited availability\n\nCurrently, I am only running [Traefik][website-traefik] as an [Ingress Controller][docs-ingress], which does load balancing and TLS termination for my pods in Kubernetes. In the future I would like to have automatic load balancing for my Kubernetes Control Planes and `LoadBalancer` type services.\n\n**Status (v2):** 💡 Planning\n\n**Keywords:** BGP, [HAProxy][website-haproxy], [Gateway API][website-gateway-api]  \n**Motivation:** Automation, ease-of-operation, client source IPs\n\n## GitOps 🔧\n\nTo automate deployments, I chose to implement GitOps via Argo CD. The initial installation procedure of Argo CD is described in [this blog post][website-argo-installation].\n\n## Limitations❗\n\nBelow you may find a list of limitations with my current infrastructure setup.\n\n- **Manual NS and DS records**  \n  Due to the limitations of the [Namecheap API][website-namecheap-api], I decided to completely administrate my NS and DS records by hand. If I can't automate it for one registrar, I will not automate it for any of them. This is subject to change based on the amount of domains and the frequency of changes. Usually however NS records rarely change.\n\n- **Manual administration of VLANs**  \n  Because my network contains a variety of network devices with different management protocols, there is a currently little value in automating the management of VLANs. If this should prove to be a valuable investment at some point, the links below include information on how to interface with some of my hardware via protocols such as the _Easy Smart Configuration Protocol (ESCP)_.\n  - [How I can gain control of your TP-LINK home switch](https://www.pentestpartners.com/security-blog/how-i-can-gain-control-of-your-tp-link-home-switch/)\n  - [Information disclosure vulnerability in TP-Link Easy Smart switches](https://www.chrisdcmoore.co.uk/post/tplink-easy-smart-switch-vulnerabilities/)\n  - [Ansible Collection - rgl.tp_link_easy_smart_switch](https://github.com/rgl/ansible-collection-tp-link-easy-smart-switch)\n\n## Hidden gems 💎\n\n- [`kboot`: A utility script to kexec into a new Kernel without rebooting](./configs/armbian-build/userpatches/overlay/kboot)\n\n## License 📄\n\nThis project is licensed under the terms of the [MIT license][file-license].\n\n[website-cubietruck]: https://linux-sunxi.org/Cubietech_Cubietruck\n[website-namecheap]: https://namecheap.com\n[website-namecheap-api]: https://www.namecheap.com/support/api/intro/\n[website-github-actions]: https://github.com/features/actions\n[website-gcp-dns]: https://cloud.google.com/dns\n[website-pulumi]: https://www.pulumi.com/\n[website-go]: https://golang.org\n[website-odroidxu4]: https://wiki.odroid.com/odroid-xu4/odroid-xu4\n[website-ansible]: https://www.ansible.com/\n[website-kubernetes]: https://kubernetes.io/\n[website-erx]: https://www.ui.com/edgemax/edgerouter-x/\n[website-k3s]: https://k3s.io\n[docs-ingress]: https://kubernetes.io/docs/concepts/services-networking/ingress/\n[website-traefik]: https://traefik.io/traefik/\n[website-kube-router]: https://www.kube-router.io/\n[website-haproxy]: http://www.haproxy.org/\n[website-gateway-api]: https://gateway-api.sigs.k8s.io/\n[file-license]: ./LICENSE.md\n[article-ubuntu-dhcp]: https://help.ubuntu.com/community/isc-dhcp-server\n[medium-ubuntu-router]: https://medium.com/@exesse/how-to-make-a-simple-router-gateway-from-ubuntu-server-18-04-lts-fd40b7bfec9\n[shop-seeed-router]: https://www.seeedstudio.com/Rapberry-Pi-CM4-Dual-GbE-Carrier-Board-p-4874.html\n[shop-rpi-cm4]: https://www.seeedstudio.com/Raspberry-Pi-Compute-Module-CM4104032-p-4722.html\n[wiki-nftables-quickstart]: https://wiki.nftables.org/wiki-nftables/index.php/Quick_reference-nftables_in_10_minutes\n[rhel-docs-nftables-quickstart]: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/chap-getting_started_with_nftables\n[rhel-docs-nftables-nat]: https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/security_guide/sec-configuring_nat_using_nftables\n[website-cryptsus-simple-firewall]: https://cryptsus.com/blog/setting-up-nftables-firewall.html\n[wiki-gentoo-nftables-examples]: https://wiki.gentoo.org/wiki/Nftables/Examples\n[wiki-arch-nftables-basics]: https://wiki.archlinux.org/title/nftables\n[wiki-nftables-mtu-clamping]: https://wiki.nftables.org/wiki-nftables/index.php/Mangling_packet_headers\n[wikipedia]: https://wikipedia.org\n[api-ipify]: https://api.ipify.org\n[docs-netplan-examples]: https://netplan.io/examples/\n[docs-netplan]: https://netplan.io\n[website-argo-installation]: https://www.arthurkoziel.com/setting-up-argocd-with-helm/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicklasfrahm%2Finfrastructure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnicklasfrahm%2Finfrastructure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnicklasfrahm%2Finfrastructure/lists"}