{"id":20458132,"url":"https://github.com/thomas-illiet/k8s-infrastructure","last_synced_at":"2025-05-08T22:32:15.239Z","repository":{"id":116142792,"uuid":"192731601","full_name":"thomas-illiet/k8s-infrastructure","owner":"thomas-illiet","description":"My Kubernetes Configuration","archived":true,"fork":false,"pushed_at":"2019-08-07T22:02:37.000Z","size":4548,"stargazers_count":5,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-21T01:46:01.962Z","etag":null,"topics":["cert-manager","ipvs","kubernetes","kubernetes-setup","nginx"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/thomas-illiet.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2019-06-19T12:51:55.000Z","updated_at":"2024-08-27T14:57:54.000Z","dependencies_parsed_at":null,"dependency_job_id":"e93ee69e-7e03-4454-987e-e968723e8480","html_url":"https://github.com/thomas-illiet/k8s-infrastructure","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fk8s-infrastructure","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fk8s-infrastructure/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fk8s-infrastructure/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/thomas-illiet%2Fk8s-infrastructure/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/thomas-illiet","download_url":"https://codeload.github.com/thomas-illiet/k8s-infrastructure/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253158165,"owners_count":21863247,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cert-manager","ipvs","kubernetes","kubernetes-setup","nginx"],"created_at":"2024-11-15T12:10:51.072Z","updated_at":"2025-05-08T22:32:15.215Z","avatar_url":"https://github.com/thomas-illiet.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# K8S Infrastructure\n\n![banner](assets/k8s-infrastructure-banner.png)\n\n## Installation and configuration\n\n### Environment\n\n```\nCentOS Linux Release 7.6.1810 64bit\nDocker version 18.06.1-ce Git commit: e68fc7a\nKubernetes version: v1.14.3\n```\n\n```\nHostname          IP            Role\nSRV-K8S-MASTER01  172.16.5.211  k8s master node\nSRV-K8S-WORKER01  172.16.5.213  k8s slave node\nSRV-K8S-WORKER02  172.16.5.214  k8s slave node\n```\n\n### Prerequisites\n\nSetup network on CentOS 7\n\nUpdate source\n\n```\nyum clean all\nyum makecache\nyum update\n```\n\nInstall utils\n\n```\nyum install epel-release -y\nyum install htop nano wget nfs-utils -y\n```\n\nSet Up Time Synchronization\n\n```\nyum install ntp ntpdate\nsystemctl enable ntpd\ntimedatectl set-timezone Europe/Paris\n```\n\nClose firewall on all nodes\n\n```\nsystemctl stop firewalld\nsystemctl disable firewalld\n```\n\nDisable SELINUX on all nodes\n\n```\nsetenforce 0\n\nvi /etc/selinux/config\nSELINUX=disabled\n```\n\nDisable swap on all nodes\n\n```\nswapoff -a\n```\n\nComment swap autoload config in the `/etc/fstab` file and varify with `free -m`.\n\nAdd following content on each /etc/hosts file\n\n```\n172.16.5.211 SRV-K8S-MASTER01\n172.16.5.213 SRV-K8S-WORKER01\n172.16.5.214 SRV-K8S-WORKER02\n```\n\nCreate `/etc/sysctl.d/k8s.conf` file\n\n```\nnet.bridge.bridge-nf-call-ip6tables = 1\nnet.bridge.bridge-nf-call-iptables = 1\nnet.ipv4.ip_forward = 1\nvm.swappiness = 0\n```\n\nTake change effect\n\n```\nmodprobe br_netfilter\nsysctl -p /etc/sysctl.d/k8s.conf\n```\n\nInstall recommended ipvs management tools\n\n```\nyum -y install ipset ipvsadm\n```\n\nOpen kernel ipvs modules for kube-proxy\n\n```\ncat \u003e /etc/sysconfig/modules/ipvs.modules \u003c\u003cEOF\n#!/bin/bash\nmodprobe -- ip_vs\nmodprobe -- ip_vs_rr\nmodprobe -- ip_vs_wrr\nmodprobe -- ip_vs_sh\nmodprobe -- nf_conntrack_ipv4\nEOF\nchmod 755 /etc/sysconfig/modules/ipvs.modules \u0026\u0026 bash /etc/sysconfig/modules/ipvs.modules \u0026\u0026 lsmod | grep -e ip_vs -e nf_conntrack_ipv4\n```\n\n### Setup Docker\n\n```\nyum install -y yum-utils device-mapper-persistent-data lvm2\nyum-config-manager \\\n    --add-repo \\\n    https://download.docker.com/linux/centos/docker-ce.repo\n```\n\nInstall and lanuch Docker on each nodes\n\n```\nyum install docker-ce docker-ce-cli containerd.io\n\nsystemctl start docker\nsystemctl stop docker\nsystemctl enable docker\n```\n\nSet docker configuration to use overlay2 Docker storage driver\n\n```\ncat \u003c\u003cEOF \u003e /etc/docker/daemon.json\n{\n  \"exec-opts\": [\"native.cgroupdriver=systemd\"],\n  \"log-driver\": \"json-file\",\n  \"log-opts\": {\n    \"max-size\": \"100m\"\n  },\n  \"storage-driver\": \"overlay2\",\n  \"storage-opts\": [\n    \"overlay2.override_kernel_check=true\"\n  ]\n}\nEOF\n```\n\nStart docker\n\n```\nsystemctl start docker\n```\n\n### Setup Kubernetes\n\nInstall kubelet, kubectl and kubeadm on each node\n\n```\ncat \u003c\u003cEOF \u003e /etc/yum.repos.d/kubernetes.repo\n[kubernetes]\nname=Kubernetes\nbaseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64\nenabled=1\ngpgcheck=1\nrepo_gpgcheck=1\ngpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg\n        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg\nEOF\n```\n\nInstallation\n\n```\nyum makecache fast\nyum install -y kubelet kubeadm kubectl\n```\n\nInit Kubernetes cluster on master node\n\n```\n# auto start on boot\nsystemctl enable kubelet \u0026\u0026 systemctl start kubelet\nkubeadm init --config /opt/k8s/kubeadm-init.conf\n````\n\nDetailed cluster initialization process\n\n```\n[root@SRV-K8S-MASTER01 k8s]# kubeadm init --config kubeadm-init.conf\n[init] Using Kubernetes version: v1.14.3\n[preflight] Pulling images required for setting up a Kubernetes cluster\n[preflight] This might take a minute or two, depending on the speed of your internet connection\n[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'\n[kubelet-start] Writing kubelet environment file with flags to file \"/var/lib/kubelet/kubeadm-flags.env\"\n[kubelet-start] Writing kubelet configuration to file \"/var/lib/kubelet/config.yaml\"\n[kubelet-start] Activating the kubelet service\n[certs] Using certificateDir folder \"/etc/kubernetes/pki\"\n[certs] Generating \"front-proxy-ca\" certificate and key\n[certs] Generating \"front-proxy-client\" certificate and key\n[certs] Generating \"etcd/ca\" certificate and key\n[certs] Generating \"etcd/healthcheck-client\" certificate and key\n[certs] Generating \"apiserver-etcd-client\" certificate and key\n[certs] Generating \"etcd/server\" certificate and key\n[certs] etcd/server serving cert is signed for DNS names [srv-k8s-master01.netboot.lan localhost] and IPs [172.16.5.211 127.0.0.1 ::1]\n[certs] Generating \"etcd/peer\" certificate and key\n[certs] etcd/peer serving cert is signed for DNS names [srv-k8s-master01.netboot.lan localhost] and IPs [172.16.5.211 127.0.0.1 ::1]\n[certs] Generating \"ca\" certificate and key\n[certs] Generating \"apiserver-kubelet-client\" certificate and key\n[certs] Generating \"apiserver\" certificate and key\n[certs] apiserver serving cert is signed for DNS names [srv-k8s-master01.netboot.lan kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10                                                  .96.0.1 172.16.5.211]\n[certs] Generating \"sa\" key and public key\n[kubeconfig] Using kubeconfig folder \"/etc/kubernetes\"\n[kubeconfig] Writing \"admin.conf\" kubeconfig file\n[kubeconfig] Writing \"kubelet.conf\" kubeconfig file\n[kubeconfig] Writing \"controller-manager.conf\" kubeconfig file\n[kubeconfig] Writing \"scheduler.conf\" kubeconfig file\n[control-plane] Using manifest folder \"/etc/kubernetes/manifests\"\n[control-plane] Creating static Pod manifest for \"kube-apiserver\"\n[control-plane] Creating static Pod manifest for \"kube-controller-manager\"\n[control-plane] Creating static Pod manifest for \"kube-scheduler\"\n[etcd] Creating static Pod manifest for local etcd in \"/etc/kubernetes/manifests\"\n[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory \"/etc/kubernetes/manifests\". This can take up to 4m0s\n[apiclient] All control plane components are healthy after 15.502251 seconds\n[upload-config] storing the configuration used in ConfigMap \"kubeadm-config\" in the \"kube-system\" Namespace\n[kubelet] Creating a ConfigMap \"kubelet-config-1.14\" in namespace kube-system with the configuration for the kubelets in the cluster\n[upload-certs] Skipping phase. Please see --experimental-upload-certs\n[mark-control-plane] Marking the node srv-k8s-master01.netboot.lan as control-plane by adding the label \"node-role.kubernetes.io/master=''\"\n[mark-control-plane] Marking the node srv-k8s-master01.netboot.lan as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]\n[bootstrap-token] Using token: dx42vx.5u8emct7bvz6l2kv\n[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles\n[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials\n[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token\n[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster\n[bootstrap-token] creating the \"cluster-info\" ConfigMap in the \"kube-public\" namespace\n[addons] Applied essential addon: CoreDNS\n[addons] Applied essential addon: kube-proxy\n\nYour Kubernetes control-plane has initialized successfully!\n\nTo start using your cluster, you need to run the following as a regular user:\n\n  mkdir -p $HOME/.kube\n  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config\n  sudo chown $(id -u):$(id -g) $HOME/.kube/config\n\nYou should now deploy a pod network to the cluster.\nRun \"kubectl apply -f [podnetwork].yaml\" with one of the options listed at:\n  https://kubernetes.io/docs/concepts/cluster-administration/addons/\n\nThen you can join any number of worker nodes by running the following on each as root:\n\nkubeadm join 172.16.5.211:6443 --token dx42vx.5u8emct7bvz6l2kv --discovery-token-ca-cert-hash sha256:de880889c87fcc683be095bac569bafaaeb80bbdff89307f0afb9398d5199983\n```\n\nConfig kubectl on the master node\n\n```\nmkdir -p $HOME/.kube\nsudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config\nsudo chown $(id -u):$(id -g) $HOME/.kube/config\n````\n\n### Install Pod networks\n\nDownload pod networks config file kube-flannel.yml and create pod networks by follwing command\n\n```\nkubectl apply -f kubectl apply -f /opt/k8s/kube-system/weave-net*.yaml \n```\n\nIf execute success, checked CoreDNS Pod running status\n\n```\nkubectl get pod --all-namespaces -owide --watch\n```\n\nCheck Kubernetes master status to check ready\n\n```\nkubectl get nodes\nNAME              STATUS   ROLES    AGE   VERSION\nsrv-k8s-master01  Ready    master   21m   v1.14.3\n```\n\n### Slave node configuration\n\nAdd slave nodes, run following command on each slave node\n\n```\nkubeadm join 172.16.5.211:6443 --token dx42vx.5u8emct7bvz6l2kv --discovery-token-ca-cert-hash sha256:de880889c87fcc683be095bac569bafaaeb80bbdff89307f0afb9398d5199983\n```\n\nIf you forget token, run following command on master node\n\n```\nkubeadm token list\n```\n\n### Status validation\n\nRun following commands on the master node\n\n```\nkubectl get nodes\nNAME               STATUS   ROLES    AGE    VERSION\nsrv-k8s-master01   Ready    master   13m    v1.14.3\nsrv-k8s-worker01   Ready    \u003cnone\u003e   107s   v1.14.3\nsrv-k8s-worker02   Ready    \u003cnone\u003e   91s    v1.14.3\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomas-illiet%2Fk8s-infrastructure","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fthomas-illiet%2Fk8s-infrastructure","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fthomas-illiet%2Fk8s-infrastructure/lists"}