{"id":13577379,"url":"https://github.com/cocowool/k8s-go","last_synced_at":"2025-04-05T11:32:47.238Z","repository":{"id":69097345,"uuid":"142007470","full_name":"cocowool/k8s-go","owner":"cocowool","description":"K8S-Go 收集整理了自己在学习实践Kubernetes过程中的一些文档和脚本，公布出来供大家参考、交流。","archived":false,"fork":false,"pushed_at":"2018-11-06T07:47:38.000Z","size":8447,"stargazers_count":135,"open_issues_count":1,"forks_count":94,"subscribers_count":7,"default_branch":"master","last_synced_at":"2024-11-05T14:44:58.860Z","etag":null,"topics":["go","kubernetes"],"latest_commit_sha":null,"homepage":"http://edulinks.cn","language":"Roff","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cocowool.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2018-07-23T11:53:30.000Z","updated_at":"2024-03-28T07:57:18.000Z","dependencies_parsed_at":"2023-02-24T10:00:59.793Z","dependency_job_id":null,"html_url":"https://github.com/cocowool/k8s-go","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cocowool%2Fk8s-go","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cocowool%2Fk8s-go/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cocowool%2Fk8s-go/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cocowool%2Fk8s-go/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cocowool","download_url":"https://codeload.github.com/cocowool/k8s-go/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247331760,"owners_count":20921844,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["go","kubernetes"],"created_at":"2024-08-01T15:01:20.938Z","updated_at":"2025-04-05T11:32:45.685Z","avatar_url":"https://github.com/cocowool.png","language":"Roff","funding_links":[],"categories":["Roff"],"sub_categories":[],"readme":"# 目录\n\n这里整理了我学习Kubernetes的资料，供大家参考交流\n- 安装\n\t- [离线环境二进制方式安装Kubernetes集群](https://github.com/cocowool/k8s-go/blob/master/install/offline-binary.md)) \n- 概念\n\t- [Kubernetes的Controllers](https://github.com/cocowool/k8s-go/blob/master/controller/README.md)\n\t- [Kubernetes的命名空间](https://github.com/cocowool/k8s-go/blob/master/learnrecord/namespace.MD)\n\t- [Kubernetes Pod详细介绍](https://github.com/cocowool/k8s-go/tree/master/pod)\n\t- [Kubernetes 存储系统介绍](https://github.com/cocowool/k8s-go/tree/master/storage)\n\t- [Kubernetes 中的ConfigMap和Secret](https://github.com/cocowool/k8s-go/tree/master/storage/configmap.md)\n- 案例\n\t- [Kubernetes部署ELK并且使用Filebeat收集集群容器日志](https://github.com/cocowool/k8s-go/tree/master/elk)\n\n# kubeadm安装kubernetes V1.11.1 集群\n\n\u003e 之前测试了[离线环境下使用二进制方法安装配置Kubernetes集群](https://www.cnblogs.com/cocowool/p/install_k8s_offline.html)的方法，安装的过程中听说 kubeadm 安装配置集群更加方便，因此试着折腾了一下。安装过程中，也有一些坑，相对来说操作上要比二进制方便一点，毕竟不用手工创建那么多的配置文件，但是对于了解Kubernetes的运作方式，可能不如二进制方式好。同时，因为kubeadm方式，很多集群依赖的组件都是以容器方式运行在Master节点上，感觉对于虚拟机资源的消耗要比二进制方式厉害。\n\n## 0. kubeadm 介绍与准备工作\n\u003e kubeadm is designed to be a simple way for new users to start trying Kubernetes out, possibly for the first time, a way for existing users to test their application on and stitch together a cluster easily, and also to be a building block in other ecosystem and/or installer tool with a larger scope.\nkubeadm是一个python写的项目，代码在[这里](https://github.com/kubernetes/kubeadm)，用来帮助快速部署Kubernetes集群环境，但是目前仅仅是作为测试环境使用，如果你想在生产环境使用，可是要三思。\n\n本文所用的环境：\n- 虚拟机软件：VirtualBox\n- 操作系统：Centos 7.3 minimal 安装\n- 网卡：两块网卡，一块 Host-Only方式，一块 Nat 方式。\n- 网络规划：\n    - Master:192.168.0.101\n    - Node:192.168.0.102-104\n\n### 0.1 关掉 selinux\n```sh\n$ setenforce  0 \n$ sed -i \"s/^SELINUX=enforcing/SELINUX=disabled/g\" /etc/sysconfig/selinux \n```\n\n### 0.2 关掉防火墙\n```sh\n$ systemctl stop firewalld\n$ systemctl disable firewalld\n```\n\n### 0.3 关闭 swap\n```sh\n$ swapoff -a \n$ sed -i 's/.*swap.*/#\u0026/' /etc/fstab\n```\n\n### 0.4 配置转发参数\n```sh\n$ cat \u003c\u003cEOF \u003e  /etc/sysctl.d/k8s.conf\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.bridge.bridge-nf-call-iptables = 1\nEOF\n$ sysctl --system\n```\n\n### 0.5 设置国内 yum 源\n```sh\n$ cat \u003c\u003cEOF \u003e /etc/yum.repos.d/kubernetes.repo\n[kubernetes]\nname=Kubernetes\nbaseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/\nenabled=1\ngpgcheck=1\nrepo_gpgcheck=1\ngpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg\nEOF\n```\n\n### 0.6 安装一些必备的工具\n```sh\n$ yum install -y epel-release \n$ yum install -y net-tools wget vim  ntpdate\n```\n\n## 1. 安装 kubeadm 必须的软件，在所有节点上运行\n### 1.1 安装Docker\n```sh\n$ yum install -y docker\n$ systemctl enable docker \u0026\u0026 systemctl start docker\n$ #设置系统服务，如果不设置后面 kubeadm init 的时候会有 warning\n$ systemctl enable docker.service\n```\n如果想要用二进制方法安装最新版本的Docker，可以参考我之前的文章[在Redhat 7.3中采用离线方式安装Docker](https://www.cnblogs.com/cocowool/p/install_docker_ce_in_redhat_73.html)\n\n### 1.2 安装kubeadm、kubectl、kubelet\n```sh\n$ yum install -y kubelet kubeadm kubectl kubernetes-cni\n$ systemctl enable kubelet \u0026\u0026 systemctl start kubelet\n```\n这一步之后kubelet还不能正常运行，还处于下面的状态。\n\u003e The kubelet is now restarting every few seconds, as it waits in a crashloop for kubeadm to tell it what to do.\n\n## 2. 安装Master节点\n因为国内没办法访问Google的镜像源，变通的方法是从其他镜像源下载后，修改tag。执行下面这个Shell脚本即可。\n```sh\n#!/bin/bash\nimages=(kube-proxy-amd64:v1.11.0 kube-scheduler-amd64:v1.11.0 kube-controller-manager-amd64:v1.11.0 kube-apiserver-amd64:v1.11.0\netcd-amd64:3.2.18 coredns:1.1.3 pause-amd64:3.1 kubernetes-dashboard-amd64:v1.8.3 k8s-dns-sidecar-amd64:1.14.9 k8s-dns-kube-dns-amd64:1.14.9\nk8s-dns-dnsmasq-nanny-amd64:1.14.9 )\nfor imageName in ${images[@]} ; do\n  docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/$imageName\n  docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/$imageName k8s.gcr.io/$imageName\n  #docker rmi registry.cn-hangzhou.aliyuncs.com/k8sth/$imageName\ndone\ndocker tag da86e6ba6ca1 k8s.gcr.io/pause:3.1\n```\n接下来执行Master节点的初始化，因为我的虚拟机是双网卡，需要指定apiserver的监听地址。\n```sh\n[root@devops-101 ~]# kubeadm init --kubernetes-version=v1.11.0 --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=192.168.0.101\n[init] using Kubernetes version: v1.11.0\n[preflight] running pre-flight checks\nI0724 08:36:35.636931    3409 kernel_validator.go:81] Validating kernel version\nI0724 08:36:35.637052    3409 kernel_validator.go:96] Validating kernel config\n\t[WARNING Hostname]: hostname \"devops-101\" could not be reached\n\t[WARNING Hostname]: hostname \"devops-101\" lookup devops-101 on 172.20.10.1:53: no such host\n\t[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'\n[preflight/images] Pulling images required for setting up a Kubernetes cluster\n[preflight/images] This might take a minute or two, depending on the speed of your internet connection\n[preflight/images] You can also perform this action in beforehand using 'kubeadm config images pull'\n[kubelet] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[preflight] Activating the kubelet service\n[certificates] Generated ca certificate and key.\n[certificates] Generated apiserver certificate and key.\n[certificates] apiserver serving cert is signed for DNS names [devops-101 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.0.101]\n[certificates] Generated apiserver-kubelet-client certificate and key.\n[certificates] Generated sa key and public key.\n[certificates] Generated front-proxy-ca certificate and key.\n[certificates] Generated front-proxy-client certificate and key.\n[certificates] Generated etcd/ca certificate and key.\n[certificates] Generated etcd/server certificate and key.\n[certificates] etcd/server serving cert is signed for DNS names [devops-101 localhost] and IPs [127.0.0.1 ::1]\n[certificates] Generated etcd/peer certificate and key.\n[certificates] etcd/peer serving cert is signed for DNS names [devops-101 localhost] and IPs [192.168.0.101 127.0.0.1 ::1]\n[certificates] Generated etcd/healthcheck-client certificate and key.\n[certificates] Generated apiserver-etcd-client certificate and key.\n[certificates] valid certificates and keys now exist in \"/etc/kubernetes/pki\"\n[kubeconfig] Wrote KubeConfig file to disk: \"/etc/kubernetes/admin.conf\"\n[kubeconfig] Wrote KubeConfig file to disk: \"/etc/kubernetes/kubelet.conf\"\n[kubeconfig] Wrote KubeConfig file to disk: \"/etc/kubernetes/controller-manager.conf\"\n[kubeconfig] Wrote KubeConfig file to disk: \"/etc/kubernetes/scheduler.conf\"\n[controlplane] wrote Static Pod manifest for component kube-apiserver to \"/etc/kubernetes/manifests/kube-apiserver.yaml\"\n[controlplane] wrote Static Pod manifest for component kube-controller-manager to \"/etc/kubernetes/manifests/kube-controller-manager.yaml\"\n[controlplane] wrote Static Pod manifest for component kube-scheduler to \"/etc/kubernetes/manifests/kube-scheduler.yaml\"\n[etcd] Wrote Static Pod manifest for a local etcd instance to \"/etc/kubernetes/manifests/etcd.yaml\"\n[init] waiting for the kubelet to boot up the control plane as Static Pods from directory \"/etc/kubernetes/manifests\" \n[init] this might take a minute or longer if the control plane images have to be pulled\n[apiclient] All control plane components are healthy after 46.002877 seconds\n[uploadconfig] storing the configuration used in ConfigMap \"kubeadm-config\" in the \"kube-system\" Namespace\n[kubelet] Creating a ConfigMap \"kubelet-config-1.11\" in namespace kube-system with the configuration for the kubelets in the cluster\n[markmaster] Marking the node devops-101 as master by adding the label \"node-role.kubernetes.io/master=''\"\n[markmaster] Marking the node devops-101 as master by adding the taints [node-role.kubernetes.io/master:NoSchedule]\n[patchnode] Uploading the CRI Socket information \"/var/run/dockershim.sock\" to the Node API object \"devops-101\" as an annotation\n[bootstraptoken] using token: wkj0bo.pzibll6rd9gyi5z8\n[bootstraptoken] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials\n[bootstraptoken] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token\n[bootstraptoken] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster\n[bootstraptoken] creating the \"cluster-info\" ConfigMap in the \"kube-public\" namespace\n[addons] Applied essential addon: CoreDNS\n[addons] Applied essential addon: kube-proxy\n\nYour Kubernetes master has initialized successfully!\n\nTo start using your cluster, you need to run the following as a regular user:\n\n  mkdir -p $HOME/.kube\n  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config\n  sudo chown $(id -u):$(id -g) $HOME/.kube/config\n\nYou should now deploy a pod network to the cluster.\nRun \"kubectl apply -f [podnetwork].yaml\" with one of the options listed at:\n  https://kubernetes.io/docs/concepts/cluster-administration/addons/\n\nYou can now join any number of machines by running the following on each node\nas root:\n\n  kubeadm join 192.168.0.101:6443 --token wkj0bo.pzibll6rd9gyi5z8 --discovery-token-ca-cert-hash sha256:51985223a369a1f8c226f3ccdcf97f4ad5ff201a7c8c708e1636eea0739c0f05\n```\n看到以上信息表示Master节点已经初始化成功了。如果需要用普通用户管理集群，可以按照提示进行操作，如果是使用root用户管理，执行下面的命令。\n\n```sh\n[root@devops-101 ~]# export KUBECONFIG=/etc/kubernetes/admin.conf \n[root@devops-101 ~]# kubectl get nodes\nNAME         STATUS     ROLES     AGE       VERSION\ndevops-101   NotReady   master    7m        v1.11.1\n[root@devops-101 ~]# kubectl get pods --all-namespaces\nNAMESPACE     NAME                                 READY     STATUS    RESTARTS   AGE\nkube-system   coredns-78fcdf6894-8sd6g             0/1       Pending   0          7m\nkube-system   coredns-78fcdf6894-lgvd9             0/1       Pending   0          7m\nkube-system   etcd-devops-101                      1/1       Running   0          6m\nkube-system   kube-apiserver-devops-101            1/1       Running   0          6m\nkube-system   kube-controller-manager-devops-101   1/1       Running   0          6m\nkube-system   kube-proxy-bhmj8                     1/1       Running   0          7m\nkube-system   kube-scheduler-devops-101            1/1       Running   0          6m\n```\n可以看到节点还没有Ready，dns的两个pod也没不正常，还需要安装网络配置。\n\n## 3. Master节点的网络配置\n这里我选用了 Flannel 的方案。\n\u003e kubeadm only supports Container Network Interface (CNI) based networks (and does not support kubenet).\n\n修改系统设置，创建 flannel 网络。\n```sh\n[root@devops-101 ~]# sysctl net.bridge.bridge-nf-call-iptables=1\nnet.bridge.bridge-nf-call-iptables = 1\n[root@devops-101 ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.10.0/Documentation/kube-flannel.yml\nclusterrole.rbac.authorization.k8s.io/flannel created\nclusterrolebinding.rbac.authorization.k8s.io/flannel created\nserviceaccount/flannel created\nconfigmap/kube-flannel-cfg created\ndaemonset.extensions/kube-flannel-ds created\n```\nflannel 默认会使用主机的第一张网卡，如果你有多张网卡，需要通过配置单独指定。修改 kube-flannel.yml 中的以下部分\n```yaml\ncontainers:\n      - name: kube-flannel\n        image: quay.io/coreos/flannel:v0.10.0-amd64\n        command:\n        - /opt/bin/flanneld\n        args:\n        - --ip-masq\n        - --kube-subnet-mgr\n        - --iface=enp0s3            #指定内网网卡\n```\n执行成功后，Master并不能马上变成Ready状态，稍等几分钟，就可以看到所有状态都正常了。\n\n```sh\n[root@devops-101 ~]# kubectl get pods --all-namespaces\nNAMESPACE     NAME                                 READY     STATUS    RESTARTS   AGE\nkube-system   coredns-78fcdf6894-8sd6g             1/1       Running   0          14m\nkube-system   coredns-78fcdf6894-lgvd9             1/1       Running   0          14m\nkube-system   etcd-devops-101                      1/1       Running   0          13m\nkube-system   kube-apiserver-devops-101            1/1       Running   0          13m\nkube-system   kube-controller-manager-devops-101   1/1       Running   0          13m\nkube-system   kube-flannel-ds-6zljr                1/1       Running   0          48s\nkube-system   kube-proxy-bhmj8                     1/1       Running   0          14m\nkube-system   kube-scheduler-devops-101            1/1       Running   0          13m\n[root@devops-101 ~]# kubectl get nodes\nNAME         STATUS    ROLES     AGE       VERSION\ndevops-101   Ready     master    14m       v1.11.1\n```\n## 4. 加入节点\nNode节点的加入集群前，首先需要按照本文的第0节和第1节做好准备工作，然后下载镜像。\n```sh\n$ docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/kube-proxy-amd64:v1.11.0\n$ docker pull registry.cn-hangzhou.aliyuncs.com/k8sth/pause-amd64:3.1\n$ docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/pause-amd64:3.1 k8s.gcr.io/pause-amd64:3.1\n$ docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/kube-proxy-amd64:v1.11.0 k8s.gcr.io/kube-proxy-amd64:v1.11.0\n$ docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/pause-amd64:3.1 k8s.gcr.io/pause:3.1\n```\n最后再根据Master节点的提示加入集群。\n```sh\n$ kubeadm join 192.168.0.101:6443 --token wkj0bo.pzibll6rd9gyi5z8 --discovery-token-ca-cert-hash sha256:51985223a369a1f8c226f3ccdcf97f4ad5ff201a7c8c708e1636eea0739c0f05\n```\n节点的启动也需要一点时间，稍后再到Master上查看状态。\n```sh\n[root@devops-101 ~]# kubectl get nodes\nNAME         STATUS    ROLES     AGE       VERSION\ndevops-101   Ready     master    1h        v1.11.1\ndevops-102   Ready     \u003cnone\u003e    11m       v1.11.1\n```\n\n我把安装中需要用到的一些命令整理成了几个脚本，放在我的[Github](https://github.com/cocowool/k8s-go)上，大家可以下载使用。\n\n![](https://images2018.cnblogs.com/blog/39469/201807/39469-20180710163655709-89635310.png)\n\n## X. 坑\n\n### pause:3.1\n安装的过程中，发现kubeadmin会找 pause:3.1 的镜像，所以需要重新 tag 。\n```sh\n$ docker tag registry.cn-hangzhou.aliyuncs.com/k8sth/pause-amd64:3.1 k8s.gcr.io/pause:3.1\n```\n\n### 两台服务器时间不同步。\n报错信息\n```sh\n[discovery] Failed to request cluster info, will try again: [Get https://192.168.0.101:6443/api/v1/namespaces/kube-public/configmaps/cluster-info: x509: certificate has expired or is not yet valid]\n```\n解决方法，设定一个时间服务器同步两台服务器的时间。\n```sh\n$ ntpdate ntp1.aliyun.com\n```\n\n## 参考资料\n1. [centos7.3 kubernetes/k8s 1.10 离线安装](https://www.jianshu.com/p/9c7e1c957752)\n2. [Kubeadm安装Kubernetes环境](https://www.cnblogs.com/ericnie/p/7749588.html)\n3. [Steps to install kubernetes](https://www.assistanz.com/steps-to-install-kubernetes-cluster-manually-using-centos-7/)\n4. [kubeadm reference guide](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/)\n5. [kubeadm安装Kubernetes V1.10集群详细文档](https://www.kubernetes.org.cn/3808.html)\n6. [kubeadm reference](https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/#config-file)\n7. [kubeadm搭建kubernetes1.7.5集群](https://blog.csdn.net/zhongyuemengxiang/article/details/79121932)\n8. [安装部署 Kubernetes 集群](https://www.cnblogs.com/Leo_wl/p/8511902.html)\n9. [linux 命令 ---- 同步当前服务器时间](https://www.cnblogs.com/chenzeyong/p/5951959.html)\n10. [CentOS 7.4 安装 K8S v1.11.0 集群所遇到的问题](https://www.cnblogs.com/myzony/p/9298783.html#1.准备工作)\n11. [使用kubeadm部署kubernetes](https://blog.csdn.net/andriy_dangli/article/details/79269348)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcocowool%2Fk8s-go","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcocowool%2Fk8s-go","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcocowool%2Fk8s-go/lists"}