{"id":13449161,"url":"https://github.com/fjudith/saltstack-kubernetes","last_synced_at":"2025-08-16T12:25:15.466Z","repository":{"id":55606484,"uuid":"187357467","full_name":"fjudith/saltstack-kubernetes","owner":"fjudith","description":"Deploy the lowest-cost production ready Kubernetes cluster using terraform and saltstack.","archived":false,"fork":false,"pushed_at":"2024-02-01T23:08:19.000Z","size":16890,"stargazers_count":71,"open_issues_count":0,"forks_count":20,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-04-07T06:41:25.485Z","etag":null,"topics":["cluster","cncf","hyperconverged","kubernetes","saltstack","terraform"],"latest_commit_sha":null,"homepage":"","language":"Jinja","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/fjudith.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2019-05-18T12:51:19.000Z","updated_at":"2024-12-20T13:49:47.000Z","dependencies_parsed_at":"2024-01-18T17:08:36.628Z","dependency_job_id":null,"html_url":"https://github.com/fjudith/saltstack-kubernetes","commit_stats":null,"previous_names":[],"tags_count":15,"template":false,"template_full_name":null,"purl":"pkg:github/fjudith/saltstack-kubernetes","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjudith%2Fsaltstack-kubernetes","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjudith%2Fsaltstack-kubernetes/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjudith%2Fsaltstack-kubernetes/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjudith%2Fsaltstack-kubernetes/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/fjudith","download_url":"https://codeload.github.com/fjudith/saltstack-kubernetes/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/fjudith%2Fsaltstack-kubernetes/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265497967,"owners_count":23777092,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cluster","cncf","hyperconverged","kubernetes","saltstack","terraform"],"created_at":"2024-07-31T06:00:32.481Z","updated_at":"2025-07-16T08:42:21.778Z","avatar_url":"https://github.com/fjudith.png","language":"Jinja","funding_links":[],"categories":["Jinja"],"sub_categories":[],"readme":"[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/2188/badge)](https://bestpractices.coreinfrastructure.org/projects/2188)\r\n[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Ffjudith%2Fsaltstack-kubernetes.svg?type=shield)](https://app.fossa.io/projects/git%2Bgithub.com%2Ffjudith%2Fsaltstack-kubernetes?ref=badge_shield)\r\n\r\n\u003cimg src=\"https://i.imgur.com/SJAtDZk.png\" width=\"460\" height=\"125\" \u003e\r\n\r\n---\r\n\r\nSaltstack-Kubernetes is an open source Kubernetes cluster deployment platform which aims to evaluate and run Cloud Native Applications like those registered in the [CNCF landscape](https://landscape.cncf.io).\r\nServer provisionning is managed using [Terraform](https://www.terraform.io) with a primarly target on low-cost Cloud providers like [Scaleway](https://scaleway.com) and [Hetzner](https://hetzner.com/cloud).\r\nKubernetes cluster deployment is managed using [Saltstack](https://saltstack.com) to deploy the various software binaries, configuration files and cloud native applications required to operate.\r\n\r\n---\r\n\r\n## Solution design\r\n\r\n\u003cimg src=\"./docs/diagram/kubernetes-cluster.svg\" \u003e\r\n\r\nThe solution design carries the following requirements:\r\n\r\n1. **Cloud provider agnostic**: Works similarly on any clouds\r\n2. **Networking privacy**: All intra-cluster communications are TLS encrypted, pod network is encrypted, Firewall is enabled by default.\r\n2. **Cluster security**: Node security and RBAC are enabled by default\r\n4. **Public endpoint**: Leverage two servers stanting as edge gateway and allow the use of a single redudant Public IP address\r\n5. **Secure admin network**: Establish a private Mesh VPN between all servers\r\n6. **Composable CRI**: Support various Container Runtime Interface plugins (see: [Features](./docs/features.md))\r\n7. **Composable CNI**: Support various Container Network Interface plugins (see: [Features](./docs/features.md))\r\n8. **Converged Storage**: Persistent storage provided by cluster nodes\r\n9. **API driven DNS**: DNS records are managed just-in-time during the deployment\r\n10. **Stable**: Only leverage stable versions of software components\r\n\r\n---\r\n\r\n## Major components versions\r\n\r\n\u003ctable\u003e\r\n  \u003ctr\u003e\r\n    \u003cth\u003eCloud provider\u003c/th\u003e\r\n    \u003cth\u003eDNS provider\u003c/th\u003e\r\n    \u003cth\u003eKubernetes version\u003c/th\u003e\r\n    \u003cth\u003eContainer runtime\u003c/th\u003e\r\n    \u003cth\u003eContainer network\u003c/th\u003e\r\n  \u003c/tr\u003e\r\n  \u003ctr\u003e\r\n    \u003ctd\u003e\u003cul\u003e\u003cli\u003e\u003cb\u003ehetzner\u003c/b\u003e\u003c/li\u003e\u003cli\u003escaleway\u003c/li\u003e\u003c/ul\u003e\u003c/td\u003e\r\n    \u003ctd\u003e\u003cul\u003e\u003cli\u003e\u003cb\u003ecloudflare\u003c/b\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/td\u003e\r\n    \u003ctd\u003e\u003cul\u003e\u003cli\u003e\u003cb\u003e1.26.4\u003c/b\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/td\u003e\r\n    \u003ctd\u003e\u003cul\u003e\u003cli\u003edocker 19.03.13\u003c/li\u003e\u003cli\u003e\u003cb\u003econtainerd 1.6.20\u003c/b\u003e\u003c/li\u003e\u003cli\u003ecri-o 1.27.0\u003c/li\u003e\u003c/ul\u003e\u003c/td\u003e\r\n    \u003ctd\u003e\u003cul\u003e\u003cli\u003e\u003cb\u003ecni 1.2.0\u003c/b\u003e\u003c/li\u003e\u003cli\u003ecalico 3.25.1\u003c/li\u003e\u003cli\u003eflannel 0.1.0\u003c/li\u003e\u003cli\u003eweave 2.6.5\u003c/li\u003e\u003cli\u003e\u003cb\u003ecillium 1.13.2\u003c/b\u003e\u003c/li\u003e\u003c/ul\u003e\u003c/td\u003e\r\n  \u003c/tr\u003e\r\n\u003c/table\u003e\r\n\r\n* Default: **bold**\r\n\r\n---\r\n\r\n## Quick start\r\n\r\n### Pre-requisits\r\n\r\nBefore starting check that following requirements are met:\r\n\r\n* [ ] Register a public domain name\r\n* [ ] Associate the domain name with [Cloudflare (Free)](https://www.cloudflare.com)\r\n* [ ] Register with the cloud provider of your choice. Expect 100$ for a full month (i.e [Scaleway](https://scaleway.com), [Hetzner](https://hetzner.com/cloud))\r\n* [ ] Setup the `terraform/terraform.tfvars` with your appropriate credentials and configuration using this [Example](./terraform/terraform.tfvars.example)\r\n* [ ] Setup the `srv/pillar/cluster_config.sls` with your appropriate credentials and configuration using this [Example](./srv/pillar/cluster_config.sls.example)\r\n  * Use this [guide](./docs/password.md) to customize the various credentials.\r\n* [ ] Install the [required tools](./docs/prerequisits.md) (i.e. terraform, jq, wireguard-tools, etc.)\r\n* [ ] Create the SSH key required to send commands to the servers.\r\n\r\n\r\n\u003e **Notice**: The configuration files are recorded in the `.gitignore` file to avoid the accidental uploads on the Web.\r\n\r\n### Server creation \r\n\r\nOnce the requirements are met, use the following command lines instanciate the server and the appropriate dns records.\r\n\r\n```bash\r\ncd terrafrom/\r\nterraform init\r\nterraform plan\r\nterraform apply\r\n```\r\n\r\n14 servers are instanciated by default. Terraform task parallelism is constrained in order to contraint the load on the cloud provider API. \r\n\r\nAt the end of the process a similar output should be displayed, listing all the generated servers and associated IP adresses.\r\n\r\n```text\r\nOutputs:\r\n\r\nhostnames = [\r\n    edge01,\r\n    edge02,\r\n    etcd01,\r\n    etcd02,\r\n    etcd03,\r\n    master01,\r\n    master02,\r\n    master03,\r\n    node01,\r\n    node02,\r\n    node03,\r\n    node04,\r\n    node05,\r\n    node06\r\n]\r\n\r\n...\r\n\r\nvpn_ips = [\r\n    172.17.4.251,\r\n    172.17.4.252,\r\n    172.17.4.51,\r\n    172.17.4.52,\r\n    172.17.4.53,\r\n    172.17.4.101,\r\n    172.17.4.102,\r\n    172.17.4.103,\r\n    172.17.4.201,\r\n    172.17.4.202,\r\n    172.17.4.203,\r\n    172.17.4.204,\r\n    172.17.4.205,\r\n    172.17.4.206\r\n]\r\n```\r\n\r\n### Kubernetes cluster deployment\r\n\r\nThe Kubernetes cluster deployment is acheived by connecting to the **salt-master** server (i.e edge01) to execute the salt states.\r\n\r\nThis can be acheived using the following one-liner...\r\n\r\n```bash\r\nssh root@edge01.example.com -C \"salt-run state.orchestrate _orchestrate\"\r\n```\r\n\r\n... Or by opening first a SSH session to get benefit of the salt state output coloring.\r\n\r\n```bash\r\nssh root@edge01.example.com\r\n\r\nroot@edge01 ~ # salt-run state.orchestrate _orchestrate\r\n```\r\n\r\n---\r\n\r\n## Accessing\r\n\r\n\u003e **Replace** example.com\" with the \"public-domain\" value from the salt pillar.\r\n\r\nRetrieve the admin user token stored in the salt pillar (i.e /srv/pillar/cluster_config.sls).\r\n\r\nInstall [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl).\r\n\r\nDownload the Kubernetes cluster CA certificate.\r\n\r\n```bash\r\nexport CLUSTER_DOMAIN=\"example.com\"\r\n\r\nmkdir -p ~/.kube/ssl/${CLUSTER_DOMAIN}\r\nscp root@edge01.${CLUSTER_DOMAIN}:/etc/kubernetes/ssl/ca.pem ~/.kube/ssl/${CLUSTER_DOMAIN}/\r\n```\r\n\r\nCreate the kubectl configuration file.\r\n\r\n```bash\r\nexport CLUSTER_TOKEN=mykubernetestoken\r\nexport CLUSTER_NAME=\"example\"\r\nexport KUBECONFIG=\"~/.kube/config\"\r\n\r\nkubectl config set-cluster ${CLUSTER_NAME} \\\r\n--server=https://kubernetes.${CLUSTER_DOMAIN}:6443 \\\r\n--certificate-authority=~/.kube/ssl/${CLUSTER_DOMAIN}/ca.pem\r\n\r\nkubectl config set-credentials admin-${CLUSTER_NAME} \\\r\n--token=${CLUSTER_TOKEN}\r\n\r\nkubectl config set-context ${CLUSTER_NAME} \\\r\n--cluster=${CLUSTER_NAME} \\\r\n--user=admin-${CLUSTER_NAME}\r\n\r\nkubectl config use-context ${CLUSTER_NAME}\r\n```\r\n\r\n### Kubernetes cluster access\r\n\r\nCheck the Kubernetes cluster component health.\r\n\r\n```bash\r\nkubectl get componentstatus\r\n\r\nNAME                 STATUS    MESSAGE              ERROR\r\netcd-2               Healthy   {\"health\": \"true\"}\r\netcd-1               Healthy   {\"health\": \"true\"}\r\ncontroller-manager   Healthy   ok\r\nscheduler            Healthy   ok\r\netcd-0               Healthy   {\"health\": \"true\"}\r\n```\r\n\r\nCheck the Kubernetes cluster nodes status.\r\n\r\n```bash\r\nkubectl get nodes\r\n\r\nNAME       STATUS   ROLES          AGE   VERSION\r\nedge01     Ready    ingress,node   32d   v1.26.4\r\nedge02     Ready    ingress,node   32d   v1.26.4\r\nmaster01   Ready    master         32d   v1.26.4\r\nmaster02   Ready    master         32d   v1.26.4\r\nmaster03   Ready    master         32d   v1.26.4\r\nnode01     Ready    node           32d   v1.26.4\r\nnode02     Ready    node           32d   v1.26.4\r\nnode03     Ready    node           32d   v1.26.4\r\nnode04     Ready    node           32d   v1.26.4\r\nnode05     Ready    node           32d   v1.26.4\r\nnode06     Ready    node           32d   v1.26.4\r\n```\r\n\r\nRetreive the URLs protected by the Kube-APIserver.\r\n\r\n```bash\r\nkubectl cluster-info\r\n\r\nKubernetes control plane is running at https://kubernetes.example.com:6443\r\nElasticsearch is running at https://kubernetes.example.com:6443/api/v1/namespaces/kube-system/services/elasticsearch-logging:db/proxy\r\nKibana is running at https://kubernetes.example.com:6443/api/v1/namespaces/kube-system/services/kibana-logging/proxy\r\nCoreDNS is running at https://kubernetes.example.com:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy\r\n\r\nTo further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.\r\n```\r\n\r\n## Kubectl Proxy\r\n\r\nThe URLs returned by `kubectl cluster-info` are protected by a mutual TLS authentification. Meaning that direct access from your Web Browser is denied until you register the appropriate certificate and private key in it.\r\n\r\nPrefer the `kubectl proxy` command which enables the access to URL protected by the Kube-APIServer.\r\nOnce launched. URLs are available from the **localhost** on the HTTP port 8001.\r\n\r\n\u003e e.g. http://localhost:8001/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy\r\n\r\n![Kubernetes Dashboard](./docs/media/kubernetes_dashboard.png)\r\n\r\n---\r\n\r\n## Credits\r\n\r\nThis project is vastly inspired by the following projects:\r\n\r\n* [Kubernetes-Saltstack](https://github.com/valentin2105/Kubernetes-Saltstack) from [@valentin2105](https://github.com/valentin2105)\r\n* [hobby-kube](https://github.com/hobby-kube/provisionning)  from [@pstadler](https://github.com/pstadler)\r\n* [Kubernetes The Hard Way](https://github.com/kelseyhightower/kubernetes-the-hard-way) from [@kelseyhightower](https://github.com/kelseyhightower)\r\n* [Saltformula-Kubernetes](https://github.com/salt-formulas/salt-formula-kubernetes)\r\n* [Kubernetes Icons](https://github.com/octo-technology/kubernetes-icons)\r\n\r\n## License\r\n[![FOSSA Status](https://app.fossa.io/api/projects/git%2Bgithub.com%2Ffjudith%2Fsaltstack-kubernetes.svg?type=large)](https://app.fossa.io/projects/git%2Bgithub.com%2Ffjudith%2Fsaltstack-kubernetes?ref=badge_large)\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffjudith%2Fsaltstack-kubernetes","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffjudith%2Fsaltstack-kubernetes","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffjudith%2Fsaltstack-kubernetes/lists"}