https://github.com/jonas-be/jk8s-cluster
Fully automated setup of my kubernetes cluster
https://github.com/jonas-be/jk8s-cluster
k8s kubernetes kubernetes-setup
Last synced: 11 months ago
JSON representation
Fully automated setup of my kubernetes cluster
- Host: GitHub
- URL: https://github.com/jonas-be/jk8s-cluster
- Owner: jonas-be
- Created: 2023-12-26T23:15:54.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-07-07T19:11:09.000Z (almost 2 years ago)
- Last Synced: 2024-11-07T18:15:08.033Z (over 1 year ago)
- Topics: k8s, kubernetes, kubernetes-setup
- Language: Shell
- Homepage:
- Size: 36.1 KB
- Stars: 4
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# jk8s-cluster
This repo contains scripts to setup a kubernetes cluster with 3 nodes untainted.
It is fully automated and installs the following:
- Kubernetes (via kubeadm)
- Cilium (CNI)
- Traefik (Ingress)
- Certmanager (TLS certificates)
- Socat binary and a script (for port forwarding)
- Longhorn (Storage)
## Requirements
- 3 Servers
- Debain 12
- Root access
- Domain
- Be able to set DNS records
- Local machine
- kubectl
- helm
## Quickstart
**First prepare your nodes:**
1. Debain 12
2. Add your ssh key to the ``authorized_keys`` file on the root
3. Make sure all hosts are in your known hosts file
**Create control plane endpoint:**
Create a sub domain for your control plane endpoint.
Create an A record for the sub domain pointing to the first node.
After the setup you can add A records for all nodes.
**Then configure the bootstrap script by setting the env vars:**
```bash
# Username and Password of the new user
export NEW_USER='jonasbe'
export PW='abc'
# Control plane endpoint
export CP_ENDPOINT='cp.k8s.jonasbe.de:6443'
# Cluster name
export CLUSTER_NAME='jk8s'
# Email for Let's Encrypt
export EMAIL='jonasbe.dev@gmail.com'
# Servers to setup
# it musst be the domain to connect and the hostname of a server
# the first server is the node which runs kubeadm init
export SERVERS='nc1.jonasbe.de nc2.jonasbe.de nc3.jonasbe.de'
```
**After configuring the setup, execute the script:**
```bash
./bootstrap.sh
```
## What happens?
### Initialize the nodes
1. Create a new user
2. Move ssh keys to new user and remove from root
3. Update the system
4. Install containerd
5. Install kubeadm, kubelet and kubectl
### Initialize the cluster
Runs on the first node you configured
1. Create a kubeadm config file
2. Kubeadm init with the created config file
3. Copy the kubeconfig file to the new user
### Join the other nodes
1. Runs ```upload-certs``` via kubeadm
2. Generate a join command
3. Combines the certs from step 1 and the join command from step 2 and uses it to join the node
### Install CNI
1. Install cilium cli on master node
2. Install cilium CNI command
### Untatint all nodes
All nodes get untainted, so that pods can be scheduled on them.
### Port forwarding
To forward ports to a node port, it uses socat.
By default it forwards port 80 and 443 to the nodeports configure in the Traefik deployment.
1. Install socat and screen
2. Start a screen session's *(you have start them after every reboot)*
**How to start the forwarding manually:**
```bash
./start-forward.sh
```
### Deploy Traefik Ingress
1. Install Traefik via helm
2. Install certmanager
3. Wait 10s to let certmanager get ready
4. Apply letsencrypt clusterissuer, for tls certificates
### Deploy Longhorn
1. Prepare the nodes for longhorn
2. Install longhorn via helm
## Deployments
### Test Deployment with Ingress and SSL
Edit the ``traefik/whoami/whoami.yaml`` file and change the domain to your domain.
Make sure the domain is pointing to all of your servers.
Then apply the manifest ``kubectl apply -f traefik/whoami/whoami.yaml``.