{"id":14965326,"url":"https://github.com/unixhot/salt-kubeadm","last_synced_at":"2025-04-05T20:06:19.882Z","repository":{"id":113041940,"uuid":"201775093","full_name":"unixhot/salt-kubeadm","owner":"unixhot","description":"SaltStack使用kubeadm方式自动化部署Kubernetes集群。","archived":false,"fork":false,"pushed_at":"2024-10-07T05:50:20.000Z","size":399,"stargazers_count":70,"open_issues_count":0,"forks_count":48,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-03-29T18:06:54.160Z","etag":null,"topics":["kubernetes","saltstack"],"latest_commit_sha":null,"homepage":"http://k8s.unixhot.com/","language":"SaltStack","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/unixhot.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-08-11T14:28:32.000Z","updated_at":"2024-10-07T05:50:24.000Z","dependencies_parsed_at":"2023-11-22T13:31:10.625Z","dependency_job_id":"dbe441a8-fe7d-4bef-8916-528ee767eb29","html_url":"https://github.com/unixhot/salt-kubeadm","commit_stats":{"total_commits":216,"total_committers":6,"mean_commits":36.0,"dds":0.06481481481481477,"last_synced_commit":"4fc05fdca88e61c6338289d7bd3557172031c444"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixhot%2Fsalt-kubeadm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixhot%2Fsalt-kubeadm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixhot%2Fsalt-kubeadm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/unixhot%2Fsalt-kubeadm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/unixhot","download_url":"https://codeload.github.com/unixhot/salt-kubeadm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247393569,"owners_count":20931812,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubernetes","saltstack"],"created_at":"2024-09-24T13:34:35.416Z","updated_at":"2025-04-05T20:06:19.859Z","avatar_url":"https://github.com/unixhot.png","language":"SaltStack","funding_links":[],"categories":[],"sub_categories":[],"readme":"# SaltStack自动化部署Kubernetes(kubeadm HA版)\n\n- 在Kubernetes v1.13版本开始，kubeadm正式可以生产使用，但是kubeadm手动操作依然很繁琐，这里使用SaltStack进行自动化部署。\n\n## 版本明细：Release-v1.30.2\n\n- 支持高可用HA\n- 测试通过系统： CentOS 8.x（不再支持CentOS7）\n- salt-ssh:    3002.2\n\n### 架构介绍\n建议部署节点：最少三个节点，请配置好主机名解析（必备）\n1. 使用Salt Grains进行角色定义，增加灵活性。\n2. 使用Salt Pillar进行配置项管理，保证安全性。\n3. 使用Salt SSH执行状态，不需要安装Agent，保证通用性。\n4. 使用Kubernetes当前稳定版本v1.30.2，保证稳定性。\n\n# 部署手册\n\n请参考开源书籍：[Docker和Kubernetes实践指南](http://k8s.unixhot.com) 第五章节内容。\n\n\n## 1.系统初始化(必备，所有节点都需操作)\n\n**1.1 设置主机名！！！**\n\n```\n[root@linux-node1 ~]# hostnamectl set-hostname linux-node1.example.com\n[root@linux-node2 ~]# hostnamectl set-hostname linux-node2.example.com\n[root@linux-node3 ~]# hostnamectl set-hostname linux-node3.example.com\n\n```\n**1.2 设置/etc/hosts保证主机名能够解析**\n\n```\n[root@linux-node1 ~]# vim /etc/hosts\n127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4\n::1         localhost localhost.localdomain localhost6 localhost6.localdomain6\n192.168.56.11 linux-node1 linux-node1.example.com\n192.168.56.12 linux-node2 linux-node2.example.com\n192.168.56.13 linux-node3 linux-node3.example.com\n```\n**1.3 关闭SELinux**\n\n```\n[root@linux-node1 ~]# vim /etc/sysconfig/selinux\nSELINUX=disabled #修改为disabled\n```\n\n**1.4 关闭NetworkManager和防火墙开启自启动**\n\n```\n[root@linux-node1 ~]# systemctl stop firewalld \u0026\u0026 systemctl disable firewalld\n```\n\n**1.5 彻底关闭交换分区**\n```\n[root@linux-node1 ~]# vim /etc/fstab\n#删除掉交换分区配置\n```\n\n**1.6 更新到最新版本并重启**\n\n```\n[root@linux-node1 ~]# yum update -y \u0026\u0026 reboot\n```\n\n\u003e 注意：以上初始化操作需要所有节点都执行，缺少步骤会导致无法安装。Kubernetes要求集群的时间同步，并且主机名不能相同，而且保证可以解析。\n\n## 2.安装Salt-SSH并克隆本项目代码。\n\n**2.1 设置部署节点到其它所有节点的SSH免密码登录（包括本机）**\n\n```\n[root@linux-node1 ~]# ssh-keygen -t rsa -q -N ''\n[root@linux-node1 ~]# ssh-copy-id linux-node1\n[root@linux-node1 ~]# ssh-copy-id linux-node2\n[root@linux-node1 ~]# ssh-copy-id linux-node3\n```\n\n**2.2 安装Salt SSH（注意：老版本的Salt SSH不支持Roster定义Grains，需要2017.7.4以上版本）**\n\n# For CentOS 8\n```\nrpm --import https://repo.saltproject.io/py3/redhat/8/x86_64/latest/SALTSTACK-GPG-KEY.pub\ncurl -fsSL https://repo.saltproject.io/py3/redhat/8/x86_64/latest.repo | sudo tee /etc/yum.repos.d/salt.repo\nyum install -y salt-ssh git unzip\n```\n\n**2.3 获取本项目代码，并放置在/srv目录**\n\n```\n# 克隆项目\ngit clone https://github.com/unixhot/salt-kubeadm.git\n\n# 放置文件\ncd salt-kubeadm/\ncp -r * /srv/\n/bin/cp /srv/roster /etc/salt/roster\n/bin/cp /srv/master /etc/salt/master\n```\n\n## 3.Salt SSH管理的机器以及角色分配\n\n\u003e 注意：下方单Master部署和多Master部署，选择其中之一执行。\n\n### Kubernetes单Master部署 \n\n```\n[root@linux-node1 ~]# vim /etc/salt/roster \nlinux-node1:\n  host: 192.168.56.11\n  user: root\n  priv: /root/.ssh/id_rsa\n  minion_opts:\n    grains:\n      k8s-role: master\n\nlinux-node2:\n  host: 192.168.56.12\n  user: root\n  priv: /root/.ssh/id_rsa\n  minion_opts:\n    grains:\n      k8s-role: node\n\nlinux-node3:\n  host: 192.168.56.13\n  user: root\n  priv: /root/.ssh/id_rsa\n  minion_opts:\n    grains:\n      k8s-role: node\n```\n\n\u003e k8s-role: 用来设置K8S的角色\n\n## 4.修改对应的配置参数，本项目使用Salt Pillar保存配置\n```\n[root@linux-node1 ~]# vim /srv/pillar/k8s.sls\n#设置需要安装的Kubernetes版本\nK8S_VERSION: \"1.30.2\"\n\n#设置软件包的版本，和安装版本有区别\nK8S_PKG_VERSION: \"1.30.2-150500.1.1\"\n\n#设置高可用集群VIP地址（部署高可用必须修改）\nMASTER_VIP: \"192.168.56.10\"\n\n#设置Master的IP地址(必须修改)\nMASTER_IP: \"192.168.56.11\"\n\n#通过Grains FQDN自动获取本机IP地址，请注意保证主机名解析到本机IP地址\nNODE_IP: {{ grains['fqdn_ip4'][0] }}\n\n#配置Service IP地址段\nSERVICE_CIDR: \"10.1.0.0/16\"\n\n#Kubernetes服务 IP (从 SERVICE_CIDR 中预分配)\nCLUSTER_KUBERNETES_SVC_IP: \"10.1.0.1\"\n\n#Kubernetes DNS 服务 IP (从 SERVICE_CIDR 中预分配)\nCLUSTER_DNS_SVC_IP: \"10.1.0.2\"\n\n#设置Node Port的端口范围\nNODE_PORT_RANGE: \"20000-40000\"\n\n#设置POD的IP地址段\nPOD_CIDR: \"10.2.0.0/16\"\n\n#设置集群的DNS域名\nCLUSTER_DNS_DOMAIN: \"cluster.local.\"\n```\n\n## 5.集群部署\n\n### 5.1 测试Salt SSH联通性\n\n```\n[root@linux-node1 ~]# salt-ssh -i '*' -r 'yum install -y python3 \u0026\u0026 swapoff -a'\n[root@linux-node1 ~]# salt-ssh -i '*' test.ping\nlinux-node2:\n    True\nlinux-node3:\n    True\nlinux-node1:\n    True\n```\n\u003e 此步骤是测试salt-ssh可以联通待部署的节点，保证没有问题，都返回True方可继续，如果有异常请先解决异常。保证机器没有SWAP分区，如果存在需要关闭，如果不是全新的系统，请谨慎执行关闭交换分区操作！\n\n### 5.2 部署K8S集群基础组件\n\n执行高级状态，会根据定义的角色再对应的机器部署对应的服务，例如安装kubeadm、kubelet、docker，加载IPVS内核模板，调整内核参数，生成kubeadm的配置文件等。\n```\nsalt-ssh '*' state.highstate\n```\n\n\u003e 喝杯咖啡休息一下，根据网络环境的不同，该步骤一般时长在5分钟以内，如果执行有失败可以再次执行即可！执行该操作会部署基本的环境，包括初始化需要用到的YAML。执行完毕之后请查看结果，需要保证所有的Failed：为0，说明初始化成功。\n```\nSummary for linux-node3\n-------------\nSucceeded: 19 (changed=19)\nFailed:     0\n-------------\nTotal states run:     19\nTotal run time:  733.939 s\n```\n\n### 5.3 Master初始化\n\n\u003e 注意：下方单Master部署和多Master部署，选择其中之一执行。\n\n1. 单Master初始化\n\n在上面的操作中，是自动化安装了Kubeadm、kubelet、docker进行了系统初始化，并生成了后续需要的yaml文件，下面的操作手工操作用于了解kubeadm的基本知识。\n如果是在实验环境，只有1个CPU，在执行初始化的时候需要增加--ignore-preflight-errors=NumCPU。\n\u003e 你可以对kubeadm.yml进行定制，kubeadm会读取该文件进行初始化操作，这里我修改了负载均衡的配置使用IPVS,存放在/etc/sysconfig/kubeadm.yml\n\n```\nkubeadm init --config /etc/sysconfig/kubeadm.yml --ignore-preflight-errors=NumCPU \n```\n\u003e 需要下载Kubernetes所有应用服务镜像，根据网络情况，时间可能较长，请等待。可以在新窗口，docker images查看下载镜像进度。\n\n### 5.4 为kubectl准备配置文件\n\nkubectl默认会在用户的家目录寻找.kube/config配置文件，下面使用管理员的配置\n\n```\nmkdir -p $HOME/.kube\ncp -i /etc/kubernetes/admin.conf $HOME/.kube/config\nchown $(id -u):$(id -g) $HOME/.kube/config\n```\n\n### 5.6 部署网络插件Flannel\n\n\u003e 需要保证所有Node的网卡名称一直，如果你的网卡名称不是eth0，请修改对应参数。 - --iface=eth0，修改为对应的网卡名称。\n\n```\nkubectl create -f /etc/sysconfig/kube-flannel.yml \n```\n\n### 5.7 节点加入集群\n\n1. 在Master节点上输出加入集群的命令：\n```\n[root@linux-node1 ~]# kubeadm token create --print-join-command\nkubeadm join 192.168.56.11:6443 --token qnlyhw.cr9n8jbpbkg94szj     --discovery-token-ca-cert-hash sha256:cca103afc0ad374093f3f76b2f91963ac72eabea3d379571e88d403fc7670611 \n```\n\n2. 在Node节点上执行上面输出的命令，进行部署并加入集群。\n\n\u003e 如果执行的过程中，一直卡着无进度，请检查三台主机的时间是否同步，时间不同步会造成集群不正常，例如证书过期等。\n\n**在linux-node2.example.com上执行**\n\n```\n[root@linux-node2 ~]# kubeadm join 192.168.56.11:6443 --token qnlyhw.cr9n8jbpbkg94szj     --discovery-token-ca-cert-hash sha256:cca103afc0ad374093f3f76b2f91963ac72eabea3d379571e88d403fc7670611\n```\n\n**在linux-node3.example.com上执行**\n```\n[root@linux-node3 ~]# kubeadm join 192.168.56.11:6443 --token qnlyhw.cr9n8jbpbkg94szj     --discovery-token-ca-cert-hash sha256:cca103afc0ad374093f3f76b2f91963ac72eabea3d379571e88d403fc7670611\n```\n\n## 6.测试Kubernetes安装\n\n### 查看节点状态\n```\n[root@linux-node1 ~]# kubectl get node\nNAME            STATUS    ROLES     AGE       VERSION\n192.168.56.11   Ready     master    1m        v1.30.2\n192.168.56.12   Ready     \u003cnone\u003e    1m        v1.30.2\n192.168.56.13   Ready     \u003cnone\u003e    1m        v1.30.2\n```\n\u003e 安装时，默认给linux-node1这个node设置了污点，默认不会调度非关键组件的Pod，如需取消污点，请执行kubectl taint  node linux-node1.example.com node-role.kubernetes.io/master:NoSchedule-\n\n## 7.测试Kubernetes集群和Flannel网络\n\n1. 创建Pod进行测试\n```\n[root@linux-node1 ~]# kubectl run nginx-test --image=registry.cn-beijing.aliyuncs.com/opsany/nginx:1.26-perl\npod/nginx-test created\n需要等待拉取镜像，可能稍有的慢，请等待。\n```\n\n2. 查看创建状态\n```\n[root@linux-node1 ~]# kubectl get pod -o wide\nNAME       READY   STATUS    RESTARTS   AGE   IP         NODE                      NOMINATED NODE   READINESS GATES\nnginx-test   1/1     Running   0          22s   10.2.12.2  linux-node2.example.com   \u003cnone\u003e           \u003cnone\u003e\n```\n\n3. 测试联通性，如果都能ping通，说明Kubernetes集群部署完毕。\n```\n[root@linux-node1 ~]# ping 10.2.12.2\nPING 10.2.12.2 (10.2.12.2) 56(84) bytes of data.\n64 bytes from 10.2.12.2: icmp_seq=1 ttl=61 time=8.72 ms\n\n--- 10.2.12.2 ping statistics ---\n1 packets transmitted, 1 received, 0% packet loss, time 0ms\nrtt min/avg/max/mdev = 8.729/8.729/8.729/0.000 ms\n```\n\n4. 测试访问，如果访问正常。说明Kubernetes集群运行正常。\n```\n[root@linux-node1 ~]# curl --head http://10.2.1.3\nHTTP/1.1 200 OK\nServer: nginx/1.26.1\nDate: Mon, 17 Jun 2024 12:54:23 GMT\nContent-Type: text/html\nContent-Length: 615\nLast-Modified: Tue, 28 May 2024 13:28:07 GMT\nConnection: keep-alive\nETag: \"6655dbe7-267\"\nAccept-Ranges: bytes\n```\n\n# 必备插件\n\n## 1. 部署Ingress-Control\n\n```\nkubectl label nodes linux-node2.example.com edgenode=true\nkubectl create -f /srv/addons/nginx-ingress/nginx-ingress.yaml\nkubectl get pod -n ingress-nginx\n```\n\n## 2.部署Helm3\n\n\u003e HELM是Kubernetes的包管理工具。使用Helm可以快速的安装和部署应用到Kubernetes上。\n\n1.部署Helm\n```\ncd /usr/local/src\n# 官方包\nwget https://get.helm.sh/helm-v3.15.2-linux-amd64.tar.gz\n# 国内访问\nwget https://opsany.oss-cn-beijing.aliyuncs.com/helm-v3.15.2-linux-amd64.tar.gz\ntar zxf helm-v3.15.2-linux-amd64.tar.gz\nmv linux-amd64/helm /usr/local/bin/\n```\n\n2.验证安装是否成功\n```\n[root@linux-node1 ~]# helm version\nversion.BuildInfo{Version:\"v3.15.2\", GitCommit:\"1a500d5625419a524fdae4b33de351cc4f58ec35\", GitTreeState:\"clean\", GoVersion:\"go1.22.4\"}\n```\n\n\u003e ------------------------------------------------------------------------------\n\n## 如何新增Kubernetes Node节点\n\n1.设置SSH无密码登录\n```\n[root@linux-node1 ~]# ssh-copy-id linux-node4\n```\n\n2.在/etc/salt/roster里面，增加对应的机器\n```\n[root@linux-node1 ~]# vim /etc/salt/roster \nlinux-node4:\n  host: 192.168.56.14\n  user: root\n  priv: /root/.ssh/id_rsa\n  minion_opts:\n    grains:\n      k8s-role: node\n```\n\n3.执行SaltStack状态salt-ssh '*' state.highstate。\n```\n[root@linux-node1 ~]# salt-ssh 'linux-node4' state.highstate\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funixhot%2Fsalt-kubeadm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Funixhot%2Fsalt-kubeadm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Funixhot%2Fsalt-kubeadm/lists"}