{"id":13449434,"url":"https://github.com/valentin2105/Kubernetes-Saltstack","last_synced_at":"2025-03-22T22:33:03.252Z","repository":{"id":39525655,"uuid":"114044555","full_name":"valentin2105/Kubernetes-Saltstack","owner":"valentin2105","description":"Recipe to deploy a production Kubernetes cluster.","archived":false,"fork":false,"pushed_at":"2023-08-22T21:21:45.000Z","size":209,"stargazers_count":205,"open_issues_count":2,"forks_count":46,"subscribers_count":21,"default_branch":"master","last_synced_at":"2024-10-11T19:41:35.518Z","etag":null,"topics":["kubernetes","kubernetes-cluster","kubernetes-setup","salt-master","saltstack"],"latest_commit_sha":null,"homepage":"","language":"SaltStack","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"gpl-3.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/valentin2105.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":null,"patreon":"ValentinOuvrard","open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2017-12-12T22:13:09.000Z","updated_at":"2024-08-02T02:21:10.000Z","dependencies_parsed_at":"2024-09-02T17:41:01.982Z","dependency_job_id":null,"html_url":"https://github.com/valentin2105/Kubernetes-Saltstack","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valentin2105%2FKubernetes-Saltstack","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valentin2105%2FKubernetes-Saltstack/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valentin2105%2FKubernetes-Saltstack/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/valentin2105%2FKubernetes-Saltstack/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/valentin2105","download_url":"https://codeload.github.com/valentin2105/Kubernetes-Saltstack/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":221840644,"owners_count":16889834,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubernetes","kubernetes-cluster","kubernetes-setup","salt-master","saltstack"],"created_at":"2024-07-31T06:00:38.561Z","updated_at":"2024-10-28T14:30:56.969Z","avatar_url":"https://github.com/valentin2105.png","language":"SaltStack","funding_links":["https://patreon.com/ValentinOuvrard","https://www.patreon.com/ValentinOuvrard"],"categories":["SaltStack","Roadmap","Featured On","Code"],"sub_categories":[],"readme":"\u003cimg src=\"https://i.imgur.com/SJAtDZk.png\" width=\"460\" height=\"125\" \u003e\n\nKubernetes-Saltstack provide an easy way to deploy H/A **Kubernetes Cluster** using Salt.\n\n## Features\n\n- Cloud-provider **agnostic**\n- Support **high-available** clusters\n- Use the power of **`Saltstack`**\n- Made for **`systemd`** based Linux systems\n- **Layer 3** networking by default (**`Calico`**)\n- **`CoreDNS`** as internal DNS resolver\n- Highly **Composable** (CNI, CRI, Add-ons)\n- Integrated **add-ons** (Helm, CoreDNS, MetalLB, Dashboard, Nginx-Ingress, ...)\n- **RBAC** \u0026 **TLS** by default\n- Support **IPv6**\n\n## Getting started\n\n### I. Generate CA and TLS certificates using CfSSL\n\nLet's clone the git repo on Salt-master and create CA \u0026 certificates on the `k8s-certs/` directory using **`CfSSL`** tools:\n\n```bash\ngit clone https://github.com/valentin2105/Kubernetes-Saltstack.git /srv/salt\nln -s /srv/salt/pillar /srv/pillar\n\nwget -q --show-progress --https-only --timestamping \\\n   https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 \\\n   https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64\n\nchmod +x cfssl_linux-amd64 cfssljson_linux-amd64\nsudo mv cfssl_linux-amd64 /usr/local/bin/cfssl\nsudo mv cfssljson_linux-amd64 /usr/local/bin/cfssljson\n```\n\n##### IMPORTANT Point\n\nBecause we generate our own CA and certificates for the cluster, \n\nYou MUST put **every hostnames and IPs of the Kubernetes cluster** (master \u0026 workers) in the `certs/kubernetes-csr.json` (**`hosts`** field). \n\nYou can also modify the `certs/*json` files to match your cluster-name / country. (optional)  \n\nYou can use **either public or private names**, but they must be registered somewhere (DNS provider, internal DNS server, `/etc/hosts` file) or use **IP records instead of names**.\n\n```bash\ncd /srv/salt/k8s-certs\ncfssl gencert -initca ca-csr.json | cfssljson -bare ca\n\n# !!!!!!!!!\n# Don't forget to edit kubernetes-csr.json before this point !\n# !!!!!!!!!\n\ncfssl gencert \\\n  -ca=ca.pem \\\n  -ca-key=ca-key.pem \\\n  -config=ca-config.json \\\n  -profile=kubernetes \\\n  kubernetes-csr.json | cfssljson -bare kubernetes\n\nchown salt: /srv/salt/k8s-certs/ -R\n```\n\nAfter that, edit the `pillar/cluster_config.sls` to tweak your future Kubernetes cluster :\n\n```yaml\nkubernetes:\n  version: v1.16.1\n  domain: cluster.local\n\n  master:\n    count: 1\n    hostname: \u003cValidHostname-or-IP\u003e\n    ipaddr: 10.240.0.10\n\n    etcd:\n      version: v3.3.12\n    encryption-key: '0Wh+uekJUj3SzaKt+BcHUEJX/9Vo2PLGiCoIsND9GyY='\n\n  pki:\n    enable: false\n    host: master01.domain.tld\n    wildcard: '*.domain.tld'\n\n  worker:\n    runtime:\n      provider: docker\n      docker:\n        version: 18.09.9\n        data-dir: /dockerFS\n    networking:\n      cni-version: v0.7.1\n      provider: calico\n      calico:\n        version: v3.9.0\n        cni-version: v3.9.0\n        calicoctl-version: v3.9.0\n        controller-version: 3.9-release\n        as-number: 64512\n        token: hu0daeHais3a--CHANGEME--hu0daeHais3a\n        ipv4:\n          range: 192.168.0.0/16\n          nat: true\n          ip-in-ip: true\n        ipv6:\n          enable: false\n          nat: true\n          interface: eth0\n          range: fd80:24e2:f998:72d6::/64\n\n  global:\n    clusterIP-range: 10.32.0.0/16\n    helm-version: v2.14.3\n    dashboard-version: v2.0.0-beta4\n    coredns-version: 1.6.4 \n    admin-token: Haim8kay1rar--CHANGEME--Haim8kay11ra\n    kubelet-token: ahT1eipae1wi--CHANGEME--ahT1eipa1e1w\n    metallb: \n      enable: false\n      version: v0.8.1\n      protocol: layer2\n      addresses: 10.100.0.0/24\n    nginx-ingress:\n      enable: false \n      version: 0.26.1\n      service-type: LoadBalancer\n    cert-manager:\n      enable: false\n      version: v0.11.0\n```\n\n###### Don't forget to change Master's hostname \u0026 Tokens  using `pwgen` for example !\n\nIf you want to enable IPv6 on pod's side, you need to change `kubernetes.worker.networking.calico.ipv6.enable` to `true`.\n\n### II. Cluster deployment\n\nTo deploy your Kubernetes cluster using this formula, you first need to setup your Saltstack master/Minion.\n\nYou can use [Salt-Bootstrap](https://docs.saltstack.com/en/stage/topics/tutorials/salt_bootstrap.html) or [Salt-Cloud](https://docs.saltstack.com/en/latest/topics/cloud/) to enhance the process. \n\nThe configuration is done to use the Salt-master as the Kubernetes master. \n\nYou can have them as different nodes if needed but the `post_install/script.sh` require `kubectl` and access to the `pillar` files.\n\n#### The recommended configuration is :\n\n- one or three Kubernetes-master (Salt-master \u0026 minion)\n\n- one or more Kubernetes-workers (Salt-minion)\n\nThe Minion's roles are matched with `Salt Grains` (kind of inventory), so you need to define theses grains on your servers :\n\nIf you want a small cluster, a master can be a worker too. \n\n```bash\n# Kubernetes masters\ncat \u003c\u003c EOF \u003e /etc/salt/grains\nrole: k8s-master\nEOF\n\n# Kubernetes workers\ncat \u003c\u003c EOF \u003e /etc/salt/grains\nrole: k8s-worker\nEOF\n\n# Kubernetes master \u0026 workers\ncat \u003c\u003c EOF \u003e /etc/salt/grains\nrole: \n  - k8s-master\n  - k8s-worker\nEOF\n\nservice salt-minion restart \n```\n\nAfter that, you can apply your configuration with a (`highstate`) :\n\n```bash\n# Apply Kubernetes master configurations :\n~ salt -G 'role:k8s-master' state.highstate \n\n~ kubectl get componentstatuses\nNAME                 STATUS    MESSAGE              ERROR\nscheduler            Healthy   ok\ncontroller-manager   Healthy   ok\netcd-0               Healthy   {\"health\": \"true\"}\netcd-1               Healthy   {\"health\": \"true\"}\netcd-2               Healthy   {\"health\": \"true\"}\n\n# Apply Kubernetes worker configurations :\n~ salt -G 'role:k8s-worker' state.highstate\n\n~ kubectl get nodes\nNAME           STATUS   ROLES    AGE     VERSION   OS-IMAGE                       KERNEL-VERSION           CONTAINER-RUNTIME\nk8s-worker01   Ready    \u003cnone\u003e   3h56m   v1.16.1   Ubuntu 18.04.3 LTS             4.15.0-58-generic        docker://18.9.9\nk8s-worker02   Ready    \u003cnone\u003e   3h56m   v1.16.1   Ubuntu 18.04.3 LTS             4.15.0-58-generic        docker://18.9.9\nk8s-worker03   Ready    \u003cnone\u003e   91m     v1.16.1   Debian GNU/Linux 10 (buster)   4.19.0-6-cloud-amd64     docker://18.9.9\nk8s-worker04   Ready    \u003cnone\u003e   67m     v1.16.1   Fedora 30 (Cloud Edition)      5.2.18-200.fc30.x86_64   docker://18.9.9\n\n# Deploy Calico and Add-ons :\n~  /opt/kubernetes/post_install/setup.sh\n\n~# kubectl get pod --all-namespaces\ncert-manager           pod/cert-manager-55c44f98f-vmpcm                     1/1     Running     \ncert-manager           pod/cert-manager-cainjector-576978ffc8-w7m5j         1/1     Running     \ncert-manager           pod/cert-manager-webhook-c67fbc858-tjcvm             1/1     Running     \ndefault                pod/debug-85d7f9799-dtc6c                            1/1     Running\nkube-system            pod/calico-kube-controllers-5979855b8-vdpvw          1/1     Running\nkube-system            pod/calico-node-h7n58                                1/1     Running\nkube-system            pod/calico-node-jl4fc                                1/1     Running\nkube-system            pod/calico-node-tv5cq                                1/1     Running\nkube-system            pod/calico-node-xxbgh                                1/1     Running\nkube-system            pod/coredns-7c7c6c44bf-4lxn4                         1/1     Running\nkube-system            pod/coredns-7c7c6c44bf-t9g7v                         1/1     Running\nkube-system            pod/tiller-deploy-6966cf57d8-jpf5k                   1/1     Running\nkubernetes-dashboard   pod/dashboard-metrics-scraper-566cddb686-mf8xn       1/1     Running\nkubernetes-dashboard   pod/kubernetes-dashboard-7b5bf5d559-25cdb            1/1     Running\nmetallb-system         pod/controller-6bcfdfd677-g9s6f                      1/1     Running\nmetallb-system         pod/speaker-bmx5p                                    1/1     Running\nmetallb-system         pod/speaker-g8cqr                                    1/1     Running\nmetallb-system         pod/speaker-mklzd                                    1/1     Running\nmetallb-system         pod/speaker-xmhkm                                    1/1     Running\nnginx-ingress          pod/nginx-ingress-controller-5dcb7b4488-b68zj        1/1     Running\nnginx-ingress          pod/nginx-ingress-controller-5dcb7b4488-n7kwc        1/1     Running\nnginx-ingress          pod/nginx-ingress-default-backend-659bd647bd-5l2km   1/1     Running\n```\n\n### III. Add nodes afterwards \n\nIf you want add a node on your Kubernetes cluster, just add the new **Hostname**  and *IPs* on `kubernetes-csr.json` and run theses commands to regenerate your cluster certificates :\n\n```bash\ncd /srv/salt/k8s-certs\n\ncfssl gencert \\\n  -ca=ca.pem \\\n  -ca-key=ca-key.pem \\\n  -config=ca-config.json \\\n  -profile=kubernetes \\\n  kubernetes-csr.json | cfssljson -bare kubernetes\n\n# Reload k8s components on Master and Workers.\nsalt -G 'role:k8s-master' state.highstate\nsalt -G 'role:k8s-worker' state.highstate\n```\n\nThe `highstate` configure automatically new workers (if it match the k8s-worker role in Grains).\n\n- Tested on Debian, Ubuntu and Fedora.\n- You can easily upgrade software version on your cluster by changing values in `pillar/cluster_config.sls` and apply a `highstate`.\n- This configuration use ECDSA certificates (you can switch to `rsa` in `certs/*.json`).\n- You can change IPv4 IPPool, enable IPv6, change IPv6 IPPool, enable IPv6 NAT (for no-public networks), change BGP AS number, Enable IPinIP (to allow routes sharing between subnets).\n- If you use `salt-ssh` or `salt-cloud` you can quickly scale new workers.\n\n## Support me on Patreon\nHelp me out for a couple of :beers:!\n\nhttps://www.patreon.com/ValentinOuvrard\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalentin2105%2FKubernetes-Saltstack","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fvalentin2105%2FKubernetes-Saltstack","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fvalentin2105%2FKubernetes-Saltstack/lists"}