{"id":21516534,"url":"https://github.com/vinetos/infrastructure","last_synced_at":"2025-04-09T20:50:48.796Z","repository":{"id":187484275,"uuid":"677058835","full_name":"Vinetos/infrastructure","owner":"Vinetos","description":"My HomeLab with Proxmox, Terraform, Ansible, k3s, OPNsense, kubeseal ...","archived":false,"fork":false,"pushed_at":"2024-09-03T08:03:42.000Z","size":179,"stargazers_count":17,"open_issues_count":0,"forks_count":4,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-23T22:42:24.333Z","etag":null,"topics":["ansible","k3s","kubeseal","opnsense","proxmox","terraform","traefik"],"latest_commit_sha":null,"homepage":"","language":"Nix","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Vinetos.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-10T16:30:19.000Z","updated_at":"2025-03-19T22:31:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"bd8d511e-897b-4378-bd48-a768ea8a8362","html_url":"https://github.com/Vinetos/infrastructure","commit_stats":null,"previous_names":["vinetos/infrastructure"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vinetos%2Finfrastructure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vinetos%2Finfrastructure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vinetos%2Finfrastructure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Vinetos%2Finfrastructure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Vinetos","download_url":"https://codeload.github.com/Vinetos/infrastructure/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248111957,"owners_count":21049576,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["ansible","k3s","kubeseal","opnsense","proxmox","terraform","traefik"],"created_at":"2024-11-24T00:22:08.556Z","updated_at":"2025-04-09T20:50:48.761Z","avatar_url":"https://github.com/Vinetos.png","language":"Nix","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Infrastructure\r\n\r\nA little HomeLab with Proxmox, Terraform, Ansible and RKE2.  \r\nNo production use case, just for fun and learning.\r\n\r\n# What technologies are used ?\r\n\r\nProxmox as hypervisor.  \r\nOpnsense as firewall that redistribute network to VLAN.  \r\nOpenTofu to provision VMs.  \r\nAnsible to configure VMs.  \r\nRKE2 as undercloud.  \r\n\r\n\r\n\u003e Made with ~~pain~~ love :heart:\r\n\r\n# Getting started\r\n\r\n## On the Proxmox host\r\n\r\nFirst, we will create a user with the right permissions for Terraform.  \r\nNote that privileges has to be adapted to your needs.\r\n\r\n```shell\r\npveum role add TerraformProv -privs \"Datastore.Allocate Datastore.AllocateSpace Datastore.Audit Datastore.AllocateTemplate Pool.Allocate SDN.Use Sys.Audit Sys.Console Sys.Modify VM.Allocate VM.Audit VM.Clone VM.Config.CDROM VM.Config.Cloudinit VM.Config.CPU VM.Config.Disk VM.Config.HWType VM.Config.Memory VM.Config.Network VM.Config.Options VM.Migrate VM.Monitor VM.PowerMgmt\"\r\npveum user add terraform-prov@pve --password \u003cpassword\u003e\r\npveum aclmod / -user terraform-prov@pve -role TerraformProv\r\n```\r\n\r\n## Terraform\r\n\r\nIn order to use Terraform, you need to create a `terraform.tfvars`. You can use `terraform.tfvars.example` as a\r\ntemplate.\r\n\r\n### Variables\r\n\r\n```shell\r\ncd terraform/\r\ncp terraform.tfvars.example terraform.tfvars\r\n# Edit terraform.tfvars\r\n```\r\n\r\n### Import vmbr0 config to terraform state\r\n\r\nTerraform can manages interface, bridges and VLANs. In order to be able to manage vmbr0,\r\nwe must import it to the state. Otherwise, Terraform will try to create the resource (and fail because it already\r\nexist).\r\n\r\nIn this example, `var.pm_node_name = dell` and network is defined in `terraform/host/network.tf`.\r\n```shell\r\n# Importing vmbr0 configuration\r\nterraform import module.host.proxmox_virtual_environment_network_linux_bridge.vmbr0 dell:vmbr0\r\n# Importing a VLAN on vmbr0 configuration\r\nterraform import module.host.proxmox_virtual_environment_network_linux_vlan.vlan200 dell:vmbr0.200\r\n\r\n```\r\n\r\n### Cloud-init\r\n\r\nVMs are configured with cloud-init. You can find the configuration in `terraform/templates/cloud-init.yml`.\r\n\u003e **Do not forget to add your SSH key in the `ssh_authorized_keys` section.**\r\n\r\nBy default, VMs are configured with :\r\n\r\n\u003e - User `vinetos` with sudo privileges\r\n\u003e - fish as default shell\r\n\u003e - root login disabled\r\n\u003e - Europe/Paris as timezone\r\n\u003e - quemu-guest-agent installed\r\n\u003e - IP will be configured with DHCP\r\n\r\n### Terraform content\r\n\r\nA lot of things are configured in Terraform but no configurable with variable (in my TODO list). Make sure to check the\r\ncontent of all terraform files before running Terraform.\r\n\r\nThen, you can run Terraform.\r\n\r\n```shell\r\nterraform init\r\nterraform plan --var-file=terraform.tfvars # Note that if you don't use the default name (terraform.tfvars), you need to specify it with -var-file\r\nterraform apply\r\n```\r\n\r\nTerraform will create :\r\n\r\n- 1 VM for the firewall (OPNsense)\r\n- 1 VM for the Kubernetes master\r\n- 3 VMs for the Kubernetes workers\r\n- 1 VM for the Minecraft server\r\n- 1 VM for the Minecraft server manager\r\n\r\n## Ansible\r\n\r\nNow that VMs are created, we can configure them with Ansible.\r\n\r\nThe inventory is generated by Terraform. You can find it in `ansible/inventory.yml`.  \r\nIf you want to add more servers, you need to add them in `terraform/templates/inventory.yml.tpl` and then\r\nrun `terraform apply` to update the inventory.\r\n\r\n```shell\r\ncd ansible/\r\n\r\n# Setup servers with a common configuration for all servers\r\nansible-playbook playbook/initial_setup.yml -i inventory.yml\r\n\r\n# Install K3S cluster\r\nansible-playbook playbook/k3s_site.yml -i inventory.yml\r\n# Upgrade K3S cluster (not needed for the first install)\r\nansible-playbook playbook/k3s_upgrade.yml -i inventory.yml\r\n\r\n# Install Minecraft Servers\r\nansible-playbook playbook/minecraft.yml -i inventory.yml\r\n```\r\n\r\nNow, you have a working Kubernetes cluster and a Minecraft server (not covered by this guide).\r\n\r\n## Kubernetes\r\n\r\n### Cluster access\r\n\r\nYou may need to get the kubeconfig to access it locally. If so, You can retrieve it from the master node with the\r\nfollowing command:\r\n\r\n```shell\r\n# Get kubeconfig from k3s master\r\nrsync --rsync-path=\"sudo rsync\" \u003cuser\u003e@\u003cmaster_ip\u003e:/etc/rancher/k3s/k3s.yaml ~/.kube/config\r\n# Replace the value of server with the master IP\r\nsed -i 's/127\\.0\\.0\\.1/\u003cmaster_ip\u003e/g' ~/.kube/config\r\n# Test access\r\nkubectl get nodes\r\n```\r\n\r\n### Add applications\r\n\r\nTime to add some applications to the cluster.  \r\nEverything about the cluster can be found in `k8s/` directory. You may want to edit some files before applying them to\r\nthe cluster as it depends on my needs (eg. IngressRoutes, Credentials...).\r\n\r\nIn todo: Automate this part with Ansible.\r\n\r\n```shell\r\ncd k8s/\r\n\r\n# Install ArgoCD for GitOps\r\nkubectl apply -k argocd/\r\n\r\n# Install Kubernetes Sealed Secrets\r\nkubectl apply -f cluster/kubeseal/application.yml\r\n\r\n# Install Traefik as IngressController\r\nkubectl apply -f traefik/application.yml\r\n```\r\n\r\nYou way see that Traefik is not working. It's normal, we need to add the Cloudflare to the cluster.\r\n\r\n# Managing Secrets\r\n\r\n```shell\r\ncd k8s/traefik/\r\ncp cloudflare-credentials.yml.example cloudflare-credentials.yml\r\n```\r\n\r\n```shell\r\n# Encrypt credentials\r\nkubeseal --controller-name=sealed-secrets --controller-namespace=sealed-secrets -o yaml \u003c cloudflare-credentials.yml \u003e cloudflare-credentials-sealed.yml\r\n```\r\n\r\nAs the application is deployed with the GitOps pattern, you need to commit and push the changes to the repository.\r\n\r\nThat's it ! Everything should be working now.\r\n\r\n## What's next ?\r\n\r\n- Use docker to run Ansible and Terraform (instead of using the host)\r\n- Remove default token\r\n- Add more applications\r\n- Add monitoring\r\n- Add variables to make it more generic and more configurable\r\n- Suggestions ?\r\n\r\n## Credits\r\n\r\nSome links I have used to build this project.  \r\nhttps://github.com/NatiSayada/k3s-proxmox-terraform-ansible  \r\nhttps://github.com/k3s-io/k3s-ansible\r\n\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvinetos%2Finfrastructure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvinetos%2Finfrastructure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvinetos%2Finfrastructure/lists"}