{"id":13510376,"url":"https://github.com/techno-tim/k3s-ansible","last_synced_at":"2025-05-13T21:09:40.762Z","repository":{"id":37722099,"uuid":"474238072","full_name":"techno-tim/k3s-ansible","owner":"techno-tim","description":"The easiest way to bootstrap a self-hosted High Availability Kubernetes cluster.  A fully automated HA k3s etcd install with kube-vip, MetalLB, and more.  Build. Destroy. Repeat.","archived":false,"fork":false,"pushed_at":"2025-02-20T02:43:47.000Z","size":481,"stargazers_count":2702,"open_issues_count":30,"forks_count":1147,"subscribers_count":52,"default_branch":"master","last_synced_at":"2025-04-10T00:07:15.182Z","etag":null,"topics":["etcd","high-availability","k3s","k3s-cluster","k8s","kube-vip","kubernetes","metallb","rancher"],"latest_commit_sha":null,"homepage":"https://technotim.live/posts/k3s-etcd-ansible/","language":"Jinja","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/techno-tim.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":"timothystewart6","patreon":"technotim"}},"created_at":"2022-03-26T04:11:10.000Z","updated_at":"2025-04-09T16:48:34.000Z","dependencies_parsed_at":"2023-10-21T02:30:21.936Z","dependency_job_id":"2441aabb-4403-48dd-b64c-bbe108056c83","html_url":"https://github.com/techno-tim/k3s-ansible","commit_stats":{"total_commits":265,"total_committers":49,"mean_commits":5.408163265306122,"dds":0.6150943396226415,"last_synced_commit":"d136fa4486af05e7252dda8f92fd85663cec4935"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techno-tim%2Fk3s-ansible","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techno-tim%2Fk3s-ansible/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techno-tim%2Fk3s-ansible/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/techno-tim%2Fk3s-ansible/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/techno-tim","download_url":"https://codeload.github.com/techno-tim/k3s-ansible/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":251520476,"owners_count":21602508,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["etcd","high-availability","k3s","k3s-cluster","k8s","kube-vip","kubernetes","metallb","rancher"],"created_at":"2024-08-01T02:01:36.539Z","updated_at":"2025-04-29T14:39:34.529Z","avatar_url":"https://github.com/techno-tim.png","language":"Jinja","funding_links":["https://github.com/sponsors/timothystewart6","https://patreon.com/technotim"],"categories":["Jinja","Tools for Self-Hosting","kubernetes"],"sub_categories":["Development"],"readme":"# Automated build of HA k3s Cluster with `kube-vip` and MetalLB\n\n![Fully Automated K3S etcd High Availability Install](https://img.youtube.com/vi/CbkEWcUZ7zM/0.jpg)\n\nThis playbook will build an HA Kubernetes cluster with `k3s`, `kube-vip` and MetalLB via `ansible`.\n\nThis is based on the work from [this fork](https://github.com/212850a/k3s-ansible) which is based on the work from [k3s-io/k3s-ansible](https://github.com/k3s-io/k3s-ansible). It uses [kube-vip](https://kube-vip.io/) to create a load balancer for control plane, and [metal-lb](https://metallb.universe.tf/installation/) for its service `LoadBalancer`.\n\nIf you want more context on how this works, see:\n\n📄 [Documentation](https://technotim.live/posts/k3s-etcd-ansible/) (including example commands)\n\n📺 [Watch the Video](https://www.youtube.com/watch?v=CbkEWcUZ7zM)\n\n## 📖 k3s Ansible Playbook\n\nBuild a Kubernetes cluster using Ansible with k3s. The goal is easily install a HA Kubernetes cluster on machines running:\n\n- [x] Debian (tested on version 11)\n- [x] Ubuntu (tested on version 22.04)\n- [x] Rocky (tested on version 9)\n\non processor architecture:\n\n- [X] x64\n- [X] arm64\n- [X] armhf\n\n## ✅ System requirements\n\n- Control Node (the machine you are running `ansible` commands) must have Ansible 2.11+ If you need a quick primer on Ansible [you can check out my docs and setting up Ansible](https://technotim.live/posts/ansible-automation/).\n\n- You will also need to install collections that this playbook uses by running `ansible-galaxy collection install -r ./collections/requirements.yml` (important❗)\n\n- [`netaddr` package](https://pypi.org/project/netaddr/) must be available to Ansible. If you have installed Ansible via apt, this is already taken care of. If you have installed Ansible via `pip`, make sure to install `netaddr` into the respective virtual environment.\n\n- `server` and `agent` nodes should have passwordless SSH access, if not you can supply arguments to provide credentials `--ask-pass --ask-become-pass` to each command.\n\n## 🚀 Getting Started\n\n### 🍴 Preparation\n\nFirst create a new directory based on the `sample` directory within the `inventory` directory:\n\n```bash\ncp -R inventory/sample inventory/my-cluster\n```\n\nSecond, edit `inventory/my-cluster/hosts.ini` to match the system information gathered above\n\nFor example:\n\n```ini\n[master]\n192.168.30.38\n192.168.30.39\n192.168.30.40\n\n[node]\n192.168.30.41\n192.168.30.42\n\n[k3s_cluster:children]\nmaster\nnode\n```\n\nIf multiple hosts are in the master group, the playbook will automatically set up k3s in [HA mode with etcd](https://rancher.com/docs/k3s/latest/en/installation/ha-embedded/).\n\nFinally, copy `ansible.example.cfg` to `ansible.cfg` and adapt the inventory path to match the files that you just created.\n\nThis requires at least k3s version `1.19.1` however the version is configurable by using the `k3s_version` variable.\n\nIf needed, you can also edit `inventory/my-cluster/group_vars/all.yml` to match your environment.\n\n### ☸️ Create Cluster\n\nStart provisioning of the cluster using the following command:\n\n```bash\nansible-playbook site.yml -i inventory/my-cluster/hosts.ini\n```\n\nAfter deployment control plane will be accessible via virtual ip-address which is defined in inventory/group_vars/all.yml as `apiserver_endpoint`\n\n### 🔥 Remove k3s cluster\n\n```bash\nansible-playbook reset.yml -i inventory/my-cluster/hosts.ini\n```\n\n\u003eYou should also reboot these nodes due to the VIP not being destroyed\n\n## ⚙️ Kube Config\n\nTo copy your `kube config` locally so that you can access your **Kubernetes** cluster run:\n\n```bash\nscp debian@master_ip:/etc/rancher/k3s/k3s.yaml ~/.kube/config\n```\nIf you get file Permission denied, go into the node and temporarly run:\n```bash\nsudo chmod 777 /etc/rancher/k3s/k3s.yaml\n```\nThen copy with the scp command and reset the permissions back to:\n```bash\nsudo chmod 600 /etc/rancher/k3s/k3s.yaml\n```\n\nYou'll then want to modify the config to point to master IP by running:\n```bash\nsudo nano ~/.kube/config\n```\nThen change `server: https://127.0.0.1:6443` to match your master IP: `server: https://192.168.1.222:6443`\n\n### 🔨 Testing your cluster\n\nSee the commands [here](https://technotim.live/posts/k3s-etcd-ansible/#testing-your-cluster).\n\n### Variables\n\n| Role(s) | Variable | Type | Default | Required | Description |\n|---|---|---|---|---|---|\n| `download` | `k3s_version` | string | ❌ | Required | K3s binaries version |\n| `k3s_agent`, `k3s_server`, `k3s_server_post` | `apiserver_endpoint` | string | ❌ | Required | Virtual ip-address configured on each master |\n| `k3s_agent` | `extra_agent_args` | string | `null` | Not required | Extra arguments for agents nodes |\n| `k3s_agent`, `k3s_server` | `group_name_master` | string | `null` | Not required | Name othe master group |\n| `k3s_agent` | `k3s_token` | string | `null` | Not required | Token used to communicate between masters |\n| `k3s_agent`, `k3s_server` | `proxy_env` | dict | `null` | Not required | Internet proxy configurations |\n| `k3s_agent`, `k3s_server` | `proxy_env.HTTP_PROXY` | string | ❌ | Required | HTTP internet proxy |\n| `k3s_agent`, `k3s_server` | `proxy_env.HTTPS_PROXY` | string | ❌ | Required | HTTP internet proxy |\n| `k3s_agent`, `k3s_server` | `proxy_env.NO_PROXY` | string | ❌ | Required | Addresses that will not use the proxies |\n| `k3s_agent`, `k3s_server`, `reset` | `systemd_dir` | string | `/etc/systemd/system` | Not required | Path to systemd services |\n| `k3s_custom_registries` | `custom_registries_yaml` | string | ❌ | Required | YAML block defining custom registries. The following is an example that pulls all images used in this playbook through your private registries. It also allows you to pull your own images from your private registry, without having to use imagePullSecrets in your deployments. If all you need is your own images and you don't care about caching the docker/quay/ghcr.io images, you can just remove those from the mirrors: section. |\n| `k3s_server`, `k3s_server_post` | `cilium_bgp` | bool | `~` | Not required | Enable cilium BGP control plane for LB services and pod cidrs. Disables the use of MetalLB. |\n| `k3s_server`, `k3s_server_post` | `cilium_iface` | string | ❌ | Not required | The network interface used for when Cilium is enabled |\n| `k3s_server` | `extra_server_args` | string | `\"\"` | Not required | Extra arguments for server nodes |\n| `k3s_server` | `k3s_create_kubectl_symlink` | bool | `false` | Not required | Create the kubectl -\u003e k3s symlink |\n| `k3s_server` | `k3s_create_crictl_symlink` | bool | `true` | Not required | Create the crictl -\u003e k3s symlink |\n| `k3s_server` | `kube_vip_arp` | bool | `true` | Not required | Enables kube-vip ARP broadcasts |\n| `k3s_server` | `kube_vip_bgp` | bool | `false` | Not required | Enables kube-vip BGP peering |\n| `k3s_server` | `kube_vip_bgp_routerid` | string | `\"127.0.0.1\"` | Not required | Defines the router ID for the kube-vip BGP server |\n| `k3s_server` | `kube_vip_bgp_as` | string | `\"64513\"` | Not required | Defines the AS for the kube-vip BGP server |\n| `k3s_server` | `kube_vip_bgp_peeraddress` | string | `\"192.168.30.1\"` | Not required | Defines the address for the kube-vip BGP peer |\n| `k3s_server` | `kube_vip_bgp_peeras` | string | `\"64512\"` | Not required | Defines the AS for the kube-vip BGP peer |\n| `k3s_server` | `kube_vip_bgp_peers` | list | `[]` | Not required | List of BGP peer ASN \u0026 address pairs |\n| `k3s_server` | `kube_vip_bgp_peers_groups` | list | `['k3s_master']` | Not required | Inventory group in which to search for additional `kube_vip_bgp_peers` parameters to merge. |\n| `k3s_server` | `kube_vip_iface` | string | `~` | Not required | Explicitly define an interface that ALL control nodes should use to propagate the VIP, define it here. Otherwise, kube-vip will determine the right interface automatically at runtime. |\n| `k3s_server` | `kube_vip_tag_version` | string | `v0.7.2` | Not required | Image tag for kube-vip |\n| `k3s_server` | `kube_vip_cloud_provider_tag_version` | string | `main` | Not required | Tag for kube-vip-cloud-provider manifest when enable |\n| `k3s_server`, `k3_server_post` | `kube_vip_lb_ip_range` | string | `~` | Not required | IP range for kube-vip load balancer |\n| `k3s_server`, `k3s_server_post` | `metal_lb_controller_tag_version` | string | `v0.14.3` | Not required | Image tag for MetalLB |\n| `k3s_server` | `metal_lb_speaker_tag_version` | string | `v0.14.3` | Not required | Image tag for MetalLB |\n| `k3s_server` | `metal_lb_type` | string | `native` | Not required | Use FRR mode or native. Valid values are `frr` and `native` |\n| `k3s_server` | `retry_count` | int | `20` | Not required | Amount of retries when verifying that nodes joined |\n| `k3s_server` | `server_init_args` | string | ❌ | Not required | Arguments for server nodes |\n| `k3s_server_post` | `bpf_lb_algorithm` | string | `maglev` | Not required | BPF lb algorithm |\n| `k3s_server_post` | `bpf_lb_mode` | string | `hybrid` | Not required | BPF lb mode |\n| `k3s_server_post` | `calico_blocksize` | int | `26` | Not required | IP pool block size |\n| `k3s_server_post` | `calico_ebpf` | bool | `false` | Not required | Use eBPF dataplane instead of iptables |\n| `k3s_server_post` | `calico_encapsulation` | string | `VXLANCrossSubnet` | Not required | IP pool encapsulation |\n| `k3s_server_post` | `calico_natOutgoing` | string | `Enabled` | Not required | IP pool NAT outgoing |\n| `k3s_server_post` | `calico_nodeSelector` | string | `all()` | Not required | IP pool node selector |\n| `k3s_server_post` | `calico_iface` | string | `~` | Not required | The network interface used for when Calico is enabled |\n| `k3s_server_post` | `calico_tag` | string | `v3.27.2` | Not required | Calico version tag |\n| `k3s_server_post` | `cilium_bgp_my_asn` | int | `64513` | Not required | Local ASN for BGP peer |\n| `k3s_server_post` | `cilium_bgp_peer_asn` | int | `64512` | Not required | BGP peer ASN |\n| `k3s_server_post` | `cilium_bgp_peer_address` | string | `~` | Not required | BGP peer address |\n| `k3s_server_post` | `cilium_bgp_neighbors` | list | `[]` | Not required | List of BGP peer ASN \u0026 address pairs |\n| `k3s_server_post` | `cilium_bgp_neighbors_groups` | list | `['k3s_all']` | Not required | Inventory group in which to search for additional `cilium_bgp_neighbors` parameters to merge. |\n| `k3s_server_post` | `cilium_bgp_lb_cidr` | string | `192.168.31.0/24` | Not required | BGP load balancer IP range |\n| `k3s_server_post` | `cilium_exportPodCIDR` | bool | `true` | Not required | Export pod CIDR |\n| `k3s_server_post` | `cilium_hubble` | bool | `true` | Not required | Enable Cilium Hubble |\n| `k3s_server_post` | `cilium_hubble` | bool | `true` | Not required | Enable Cilium Hubble |\n| `k3s_server_post` | `cilium_mode` | string | `native` | Not required | Inner-node communication mode (choices are `native` and `routed`) |\n| `k3s_server_post` | `cluster_cidr` | string | `10.52.0.0/16` | Not required | Inner-cluster IP range |\n| `k3s_server_post` | `enable_bpf_masquerade` | bool | `true` | Not required | Use IP masquerading |\n| `k3s_server_post` | `kube_proxy_replacement` | bool | `true` | Not required | Replace the native kube-proxy with Cilium |\n| `k3s_server_post` | `metal_lb_available_timeout` | string | `240s` | Not required | Wait for MetalLB resources |\n| `k3s_server_post` | `metal_lb_ip_range` | string | `192.168.30.80-192.168.30.90` | Not required | MetalLB ip range for load balancer |\n| `k3s_server_post` | `metal_lb_controller_tag_version` | string | `v0.14.3` | Not required | Image tag for MetalLB |\n| `k3s_server_post` | `metal_lb_mode` | string | `layer2` | Not required | Metallb mode (choices are `bgp` and `layer2`) |\n| `k3s_server_post` | `metal_lb_bgp_my_asn` | string | `~` | Not required | BGP ASN configurations |\n| `k3s_server_post` | `metal_lb_bgp_peer_asn` | string | `~` | Not required | BGP peer ASN configurations |\n| `k3s_server_post` | `metal_lb_bgp_peer_address` | string | `~` | Not required | BGP peer address |\n| `lxc` | `custom_reboot_command` | string | `~` | Not required | Command to run on reboot |\n| `prereq` | `system_timezone` | string | `null` | Not required | Timezone to be set on all nodes |\n| `proxmox_lxc`, `reset_proxmox_lxc` | `proxmox_lxc_ct_ids` | list | ❌ | Required | Proxmox container ID list |\n| `raspberrypi` | `state` | string | `present` | Not required | Indicates whether the k3s prerequisites for Raspberry Pi should be set up (possible values are `present` and `absent`) |\n\n\n### Troubleshooting\n\nBe sure to see [this post](https://github.com/techno-tim/k3s-ansible/discussions/20) on how to troubleshoot common problems\n\n### Testing the playbook using molecule\n\nThis playbook includes a [molecule](https://molecule.rtfd.io/)-based test setup.\nIt is run automatically in CI, but you can also run the tests locally.\nThis might be helpful for quick feedback in a few cases.\nYou can find more information about it [here](molecule/README.md).\n\n### Pre-commit Hooks\n\nThis repo uses `pre-commit` and `pre-commit-hooks` to lint and fix common style and syntax errors.  Be sure to install python packages and then run `pre-commit install`.  For more information, see [pre-commit](https://pre-commit.com/)\n\n## 🌌 Ansible Galaxy\n\nThis collection can now be used in larger ansible projects.\n\nInstructions:\n\n- create or modify a file `collections/requirements.yml` in your project\n\n```yml\ncollections:\n  - name: ansible.utils\n  - name: community.general\n  - name: ansible.posix\n  - name: kubernetes.core\n  - name: https://github.com/techno-tim/k3s-ansible.git\n    type: git\n    version: master\n```\n\n- install via `ansible-galaxy collection install -r ./collections/requirements.yml`\n- every role is now available via the prefix `techno_tim.k3s_ansible.` e.g. `techno_tim.k3s_ansible.lxc`\n\n## Thanks 🤝\n\nThis repo is really standing on the shoulders of giants. Thank you to all those who have contributed and thanks to these repos for code and ideas:\n\n- [k3s-io/k3s-ansible](https://github.com/k3s-io/k3s-ansible)\n- [geerlingguy/turing-pi-cluster](https://github.com/geerlingguy/turing-pi-cluster)\n- [212850a/k3s-ansible](https://github.com/212850a/k3s-ansible)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechno-tim%2Fk3s-ansible","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftechno-tim%2Fk3s-ansible","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftechno-tim%2Fk3s-ansible/lists"}