{"id":15321049,"url":"https://github.com/eryajf/magic-of-kubernetes-scripts","last_synced_at":"2025-04-15T02:35:31.944Z","repository":{"id":104310779,"uuid":"159678081","full_name":"eryajf/magic-of-kubernetes-scripts","owner":"eryajf","description":"使用小脚本一键部署k8s-1.10.4高可用集群，更多交流可以访问我的个人博客：www.eryajf.net","archived":false,"fork":false,"pushed_at":"2018-11-29T14:39:58.000Z","size":17,"stargazers_count":16,"open_issues_count":0,"forks_count":5,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-03-28T14:21:53.605Z","etag":null,"topics":["eryajf","k8s-cluster","kubernetes","script"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/eryajf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-11-29T14:24:14.000Z","updated_at":"2024-01-17T00:38:46.000Z","dependencies_parsed_at":null,"dependency_job_id":"136bda96-cf3b-45e6-bec0-647582cab881","html_url":"https://github.com/eryajf/magic-of-kubernetes-scripts","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eryajf%2Fmagic-of-kubernetes-scripts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eryajf%2Fmagic-of-kubernetes-scripts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eryajf%2Fmagic-of-kubernetes-scripts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/eryajf%2Fmagic-of-kubernetes-scripts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/eryajf","download_url":"https://codeload.github.com/eryajf/magic-of-kubernetes-scripts/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248994936,"owners_count":21195472,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["eryajf","k8s-cluster","kubernetes","script"],"created_at":"2024-10-01T09:09:47.410Z","updated_at":"2025-04-15T02:35:31.908Z","avatar_url":"https://github.com/eryajf.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"\n## 1，简单说明。\n\n此脚本所能够成形于今日，完全是拜大神分享的https://github.com/opsnull/follow-me-install-kubernetes-cluster 项目所依托而成。之前也曾想过对k8s熟悉之后做一下部署脚本，但那时候并没有什么多么好的思路，直到上周看到了如上开源项目的部署思路，让我有种拨云见日，豁然开朗的感觉，当我跟随项目学习的时候，就已经打算了要写一下部署小脚本了。\n\n因此，这个脚本基本上可以说是大神项目流程的一个堆叠，自己则只不过是做了一点点小小的整理与调试罢了，**再一次，郑重的，对此表示感谢！**\n\n当然啦，事实上当自己来整理这个脚本的时候发现，事情也并没有那么的简单，而写脚本的不简单，则是为了以后每次部署的更简单。\n\n这里简单说明一下我使用的服务器情况：\n\n服务器均采用CentOS7.3版本，未在其他系统版本中进行测试。\n\n| 主机  | 主机名  |  组件 |\n| :------------: | :------------: | :------------: |\n| 192.168.111.3  | kube-node1  | Kubernetes 1.10.4,Docker 18.03.1-ce,Etcd 3.3.7,Flanneld 0.10.0,kube-apiserver,kube-controller-manager,kube-scheduler,kubelet,kube-proxy  |\n| 192.168.111.4  | kube-node2  | 同上  |\n| 192.168.111.5  | kube-node3  | 同上  |\n\n## 2，准备工作。\n\n首先将整个部署文件上传到部署服务器，进行解压，然后做以下准备工作。\n\n整个安装包我已打包并上传百度云，可自行下载。\n\n链接: https://pan.baidu.com/s/1JbICafwEdIwHnsDlGvPIMw 提取码: 4iaq\n\n### 1，修改以下内容。\n\n```\nconfig/environment.sh #修改ip为自己将要部署的机器ip\n\nconfig/Kcsh/hosts  #修改ip为自己将要部署的机器ip\n\nconfig/Ketcd/etcd-csr.json #修改ip为自己将要部署的机器ip\n\nconfig/Kmaster/Kha/haproxy.cfg #修改ip为自己将要部署的机器ip\n\nconfig/Kmaster/Kapi/kubernetes-csr.json #修改ip为自己将要部署的机器ip\n\nconfig/Kmaster/Kmanage/kube-controller-manager-csr.json #修改ip为自己将要部署的机器ip\n\nconfig/Kmaster/Kscheduler/kube-scheduler-csr.json #修改ip为自己将要部署的机器ip\n```\n\n\n### 2，基础配置。\n\n这些操作均在kube-node1主机上执行。\n\n`注意：`请严格按照如下这几步操作进行，否则可能导致下边部署脚本无法正常走完。\n\n```\nssh-keygen\nssh-copy-id 192.168.111.3\nssh-copy-id 192.168.111.4\nssh-copy-id 192.168.111.5\n\nscp config/Kcsh/hosts root@192.168.111.3:/etc/hosts\nscp config/Kcsh/hosts root@192.168.111.4:/etc/hosts\nscp config/Kcsh/hosts root@192.168.111.5:/etc/hosts\n\n\nssh root@kube-node1 \"hostnamectl set-hostname kube-node1\"\nssh root@kube-node2 \"hostnamectl set-hostname kube-node2\"\nssh root@kube-node3 \"hostnamectl set-hostname kube-node3\"\n```\n\n## 3，正式部署。\n\n部署非常简单，直接执行`magic.sh`脚本即可。\n\n不过有几点需要做一下简单说明：\n\n- 1，启动正式部署之前，务必仔细认真检查各处配置是否与所需求的相匹配了，若不匹配，应当调整。\n- 2，部署过程中如果有卡壳，或者未正常部署而退出，请根据对应的部署阶段进行排查，然后重新执行部署脚本，即可进行接续部署。\n- 3，如对脚本中一些不足地方有任何建议，欢迎与我提出，一起维护，共同进步！\n\n## 4，简单验证。\n\n部署完成之后，可使用如下方式进行一些对集群可用性的初步检验：\n\n### 1，检查服务是否均已正常启动。\n\n```\n#!/bin/bash\n#\n#author:eryajf\n#blog:www.eryajf.net\n#time:2018-11\n#\nset -e\nsource /opt/k8s/bin/environment.sh\n\n#\n##set color##\nechoRed() { echo $'\\e[0;31m'\"$1\"$'\\e[0m'; }\nechoGreen() { echo $'\\e[0;32m'\"$1\"$'\\e[0m'; }\nechoYellow() { echo $'\\e[0;33m'\"$1\"$'\\e[0m'; }\n##set color##\n#\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echoGreen \"\u003e\u003e\u003e ${node_ip}\"\n\tssh root@${node_ip} \"systemctl status etcd|grep Active\"\n\tssh root@${node_ip} \"systemctl status flanneld|grep Active\"\n\tssh root@${node_ip} \"systemctl status haproxy|grep Active\"\n\tssh root@${node_ip} \"systemctl status keepalived|grep Active\"\n\tssh root@${node_ip} \"systemctl status kube-apiserver |grep 'Active:'\"\n\tssh root@${node_ip} \"systemctl status kube-controller-manager|grep Active\"\n\tssh root@${node_ip} \"systemctl status kube-scheduler|grep Active\"\n\tssh root@${node_ip} \"systemctl status docker|grep Active\"\n\tssh root@${node_ip} \"systemctl status kubelet | grep Active\"\n\tssh root@${node_ip} \"systemctl status kube-proxy|grep Active\"\ndone\n```\n\n### 2，查看相关服务可用性。\n\n#### 1，验证etcd集群可用性。\n\n```\ncat \u003e magic.sh \u003c\u003c \"EOF\"\n#!/bin/bash\n\nsource /opt/k8s/bin/environment.sh\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echo \"\u003e\u003e\u003e ${node_ip}\" \n    ETCDCTL_API=3 /opt/k8s/bin/etcdctl \\\n    --endpoints=https://${node_ip}:2379 \\\n    --cacert=/etc/kubernetes/cert/ca.pem \\\n    --cert=/etc/etcd/cert/etcd.pem \\\n    --key=/etc/etcd/cert/etcd-key.pem endpoint health\ndone\nEOF\n```\n\n#### 2，验证flannel网络。\n\n查看已分配的 Pod 子网段列表：\n\n```\nsource /opt/k8s/bin/environment.sh\n\netcdctl \\\n  --endpoints=${ETCD_ENDPOINTS} \\\n  --ca-file=/etc/kubernetes/cert/ca.pem \\\n  --cert-file=/etc/flanneld/cert/flanneld.pem \\\n  --key-file=/etc/flanneld/cert/flanneld-key.pem \\\n  ls ${FLANNEL_ETCD_PREFIX}/subnets\n```\n\n输出：\n\n```\n/kubernetes/network/subnets/172.30.84.0-24\n/kubernetes/network/subnets/172.30.8.0-24\n/kubernetes/network/subnets/172.30.29.0-24\n```\n\n验证各节点能通过 Pod 网段互通：\n\n`注意其中的IP段换成自己的。`\n\n```\ncat \u003e magic.sh \u003c\u003c \"EOF\"\n#!/bin/bash\n\nsource /opt/k8s/bin/environment.sh\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echo \"\u003e\u003e\u003e ${node_ip}\" \n    ssh ${node_ip} \"ping -c 1 172.30.8.0\"\n    ssh ${node_ip} \"ping -c 1 172.30.29.0\"\n    ssh ${node_ip} \"ping -c 1 172.30.84.0\"\ndone\nEOF\n```\n\n#### 3，高可用组件验证。\n\n查看 VIP 所在的节点，确保可以 ping 通 VIP：\n\n```\ncat \u003e magic.sh \u003c\u003c \"EOF\"\n#!/bin/bash\n\nsource /opt/k8s/bin/environment.sh\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echo \"\u003e\u003e\u003e ${node_ip}\" \n    ssh ${node_ip} \"/usr/sbin/ip addr show ${VIP_IF}\"\n    ssh ${node_ip} \"ping -c 1 ${MASTER_VIP}\"\ndone\nEOF\n```\n\n#### 4，高可用性试验。\n\n查看当前的 leader：\n\n```\n$kubectl get endpoints kube-controller-manager --namespace=kube-system  -o yaml\napiVersion: v1\nkind: Endpoints\nmetadata:\n  annotations:\n    control-plane.alpha.kubernetes.io/leader: '{\"holderIdentity\":\"kube-node1_444fbc06-f3d8-11e8-8ca8-0050568f514f\",\"leaseDurationSeconds\":15,\"acquireTime\":\"2018-11-29T13:11:21Z\",\"renewTime\":\"2018-11-29T13:48:10Z\",\"leaderTransitions\":0}'\n  creationTimestamp: 2018-11-29T13:11:21Z\n  name: kube-controller-manager\n  namespace: kube-system\n  resourceVersion: \"3134\"\n  selfLink: /api/v1/namespaces/kube-system/endpoints/kube-controller-manager\n  uid: 4452bff1-f3d8-11e8-a5a6-0050568fef9b\n```\n\n可见，当前的 leader 为 kube-node1 节点。\n\n现在停掉kube-node1上的kube-controller-manager。\n\n```\n$systemctl stop kube-controller-manager\n$systemctl status kube-controller-manager |grep Active\n   Active: inactive (dead) since Sat 2018-11-24 00:52:53 CST; 44s ago\n```\n\n大概一分钟后，再查看一下当前的leader：\n\n```\n$kubectl get endpoints kube-controller-manager --namespace=kube-system  -o yaml\napiVersion: v1\nkind: Endpoints\nmetadata:\n  annotations:\n    control-plane.alpha.kubernetes.io/leader: '{\"holderIdentity\":\"kube-node3_45525ae6-f3d8-11e8-a2b8-0050568fbcaa\",\"leaseDurationSeconds\":15,\"acquireTime\":\"2018-11-29T13:49:28Z\",\"renewTime\":\"2018-11-29T13:49:28Z\",\"leaderTransitions\":1}'\n  creationTimestamp: 2018-11-29T13:11:21Z\n  name: kube-controller-manager\n  namespace: kube-system\n  resourceVersion: \"3227\"\n  selfLink: /api/v1/namespaces/kube-system/endpoints/kube-controller-manager\n  uid: 4452bff1-f3d8-11e8-a5a6-0050568fef9b\n```\n\n可以看到已经自动漂移到kube-node3上去了。\n\n#### 5，查验kube-proxy功能。\n\n查看 ipvs 路由规则\n\n```\ncat \u003e magic.sh \u003c\u003c \"EOF\"\n#!/bin/bash\n\nsource /opt/k8s/bin/environment.sh\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echo \"\u003e\u003e\u003e ${node_ip}\" \n    ssh root@${node_ip} \"/usr/sbin/ipvsadm -ln\"\ndone\nEOF\n```\n\n输出：\n\n```\n$bash magic.sh\n\u003e\u003e\u003e 192.168.111.120\nIP Virtual Server version 1.2.1 (size=4096)\nProt LocalAddress:Port Scheduler Flags\n  -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\nTCP  10.254.0.1:443 rr persistent 10800\n  -\u003e 192.168.111.120:6443         Masq    1      0          0\n  -\u003e 192.168.111.121:6443         Masq    1      0          0\n  -\u003e 192.168.111.122:6443         Masq    1      0          0\n\u003e\u003e\u003e 192.168.111.121\nIP Virtual Server version 1.2.1 (size=4096)\nProt LocalAddress:Port Scheduler Flags\n  -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\nTCP  10.254.0.1:443 rr persistent 10800\n  -\u003e 192.168.111.120:6443         Masq    1      0          0\n  -\u003e 192.168.111.121:6443         Masq    1      0          0\n  -\u003e 192.168.111.122:6443         Masq    1      0          0\n\u003e\u003e\u003e 192.168.111.122\nIP Virtual Server version 1.2.1 (size=4096)\nProt LocalAddress:Port Scheduler Flags\n  -\u003e RemoteAddress:Port           Forward Weight ActiveConn InActConn\nTCP  10.254.0.1:443 rr persistent 10800\n  -\u003e 192.168.111.120:6443         Masq    1      0          0\n  -\u003e 192.168.111.121:6443         Masq    1      0          0\n  -\u003e 192.168.111.122:6443         Masq    1      0          0\n```\n\n#### 6，创建一个应用。\n\n查看集群节点：\n\n```\n$kubectl get node\nNAME         STATUS    ROLES     AGE       VERSION\nkube-node1   Ready     \u003cnone\u003e    45m       v1.10.4\nkube-node2   Ready     \u003cnone\u003e    45m       v1.10.4\nkube-node3   Ready     \u003cnone\u003e    45m       v1.10.4\n```\n\n创建测试应用：\n\n```\ncat \u003e nginx-ds.yml \u003c\u003cEOF\napiVersion: v1\nkind: Service\nmetadata:\n  name: nginx-ds\n  labels:\n    app: nginx-ds\nspec:\n  type: NodePort\n  selector:\n    app: nginx-ds\n  ports:\n  - name: http\n    port: 80\n    targetPort: 80\n---\napiVersion: extensions/v1beta1\nkind: DaemonSet\nmetadata:\n  name: nginx-ds\n  labels:\n    addonmanager.kubernetes.io/mode: Reconcile\nspec:\n  template:\n    metadata:\n      labels:\n        app: nginx-ds\n    spec:\n      containers:\n      - name: my-nginx\n        image: nginx:1.7.9\n        ports:\n        - containerPort: 80\nEOF\n```\n\n执行定义文件，启动之前，可以先将上边定义的镜像pull下来。\n\n```\n$ kubectl create -f nginx-ds.yml\nservice \"nginx-ds\" created\ndaemonset.extensions \"nginx-ds\" created\n```\n\n检查各 Node 上的 Pod IP 连通性\n\n```\n$kubectl get pods  -o wide|grep nginx-ds\nnginx-ds-78lqz   1/1       Running   0          2m        172.30.87.2   kube-node3\nnginx-ds-j45zf   1/1       Running   0          2m        172.30.99.2   kube-node2\nnginx-ds-xhttt   1/1       Running   0          2m        172.30.55.2   kube-node1\n```\n\n可见，nginx-ds 的 Pod IP 分别是 172.30.84.2、172.30.8.2、172.30.29.2，在所有 Node 上分别 ping 这三个 IP，看是否连通：\n\n```\ncat \u003e magic.sh \u003c\u003c \"EOF\"\n#!/bin/bash\n\nsource /opt/k8s/bin/environment.sh\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echo \"\u003e\u003e\u003e ${node_ip}\"\n    ssh ${node_ip} \"ping -c 1 172.30.87.2\"\n    ssh ${node_ip} \"ping -c 1 172.30.99.2\"\n    ssh ${node_ip} \"ping -c 1 172.30.55.2\"\ndone\nEOF\n```\n\n检查服务 IP 和端口可达性\n\n```\n$kubectl get svc |grep nginx-ds\nnginx-ds           NodePort    10.254.110.153   \u003cnone\u003e        80:8781/TCP      6h\n```\n\n在所有 Node 上 curl Service IP：\n\n```\ncat \u003e magic.sh \u003c\u003c \"EOF\"\n#!/bin/bash\n\nsource /opt/k8s/bin/environment.sh\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echo \"\u003e\u003e\u003e ${node_ip}\" \n    ssh ${node_ip} \"curl 10.254.128.98\"\ndone\nEOF\n```\n\n检查服务的 NodePort 可达性\n\n```\ncat \u003e magic.sh \u003c\u003c \"EOF\"\n#!/bin/bash\n\nsource /opt/k8s/bin/environment.sh\n\nfor node_ip in ${NODE_IPS[@]}\ndo\n    echo \"\u003e\u003e\u003e ${node_ip}\" \n    ssh ${node_ip} \"curl ${node_ip}:8996\"\ndone\nEOF\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feryajf%2Fmagic-of-kubernetes-scripts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Feryajf%2Fmagic-of-kubernetes-scripts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Feryajf%2Fmagic-of-kubernetes-scripts/lists"}