Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/arthurvardevanyan/homelab
HomeLab Server & Desktop Configuration
https://github.com/arthurvardevanyan/homelab
homelab kubernetes okd terraform
Last synced: 21 days ago
JSON representation
HomeLab Server & Desktop Configuration
- Host: GitHub
- URL: https://github.com/arthurvardevanyan/homelab
- Owner: ArthurVardevanyan
- License: unlicense
- Created: 2021-11-05T00:41:06.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-09-18T10:31:06.000Z (4 months ago)
- Last Synced: 2024-09-18T17:44:08.377Z (4 months ago)
- Topics: homelab, kubernetes, okd, terraform
- Language: YAML
- Homepage: https://www.arthurvardevanyan.com/homelab.html
- Size: 47.8 MB
- Stars: 16
- Watchers: 3
- Forks: 11
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# HomeLab
HomeLab Server/Cluster, Virtual Sandbox Cluster, & Desktop Configuration
- Desktop: Fedora 39
## Table of Contents
- [HomeLab](#homelab)
- [Table of Contents](#table-of-contents)
- [Desktop](#desktop)
- [Gnome](#gnome)
- [Cura](#cura)
- [Virtual Sandbox](#virtual-sandbox)
- [KVM Sandbox Terraform](#kvm-sandbox-terraform)
- [Server](#server)
- [Kubernetes](#kubernetes)
- [KVM Config Dump](#kvm-config-dump)
- [OKD Longhorn Secondary Disk Setup](#okd-longhorn-secondary-disk-setup)
- [OKD Upgrade](#okd-upgrade)
- [OKD Host Disk Expansion](#okd-host-disk-expansion)
- [OKD Host Bad Block Recovery](#okd-host-bad-block-recovery)
- [OKD WIF](#okd-wif)
- [Kubernetes Commands](#kubernetes-commands)
- [Delete Pod Using Graceful Termination Eviction Request](#delete-pod-using-graceful-termination-eviction-request)
- [SSH Keyscan](#ssh-keyscan)
- [Vault Kubernetes Integration](#vault-kubernetes-integration)
- [k3s Install](#k3s-install)
- [Database](#database)
- [MariaDB](#mariadb)
- [Postgres](#postgres)
- [Quay](#quay)
- [Tekton](#tekton)
## Desktop
```bash
ansible-playbook -i ansible/inventory --ask-become-pass ansible/desktop.yaml --ask-pass \
-e 'ansible_python_interpreter=/usr/bin/python3'
git merge --no-ff
scp -r /mnt/storage/vm/*.img [email protected]:/backup/WindowsBackup/vm
sudo sensors-detect
```
### Gnome
Manually Install Extensions from extensions.gnome.org
- gnome-shell-extension-netspeed
- gnome-shell-extension-places-menu
- gnome-shell-extension-transparentnotification
### Cura
Config files need to be applied manually.
```bash
machineConfigs/desktop/home/arthur/cura
```
## Virtual Sandbox
```bash
# Terminal 1
# Generate Preseed Config Password and Startup Temporary Web Server
bash kvm_k3s.bash preseed_server
# Terminal 2
# Enter Password Defined with Hash in Pre Seed Config
mkdir -p notes time bash kvm_k3s.bash install_cluster > notes/install.log
# KubeConfig
export KUBECONFIG=${HOME}/vm/sk3s/sk3s.yaml
# Dashboard Secret
bash main.bash get_dashboard_secret
```
## KVM Sandbox Terraform
TF Provider
-
-
OpenShift Terraform Example
-
-
Permission Denied Issue
-
## Server
```mermaid
graph TD
linkStyle default interpolate basis
subgraph Networking
wan1[WAN
192.168.100.1]---|1000/50 Mb|router{UDM Pro SE
10.0.0.1}
wan2[LTE
192.168.1.1]---|100/25 Mb|router
router---|10GbE|switch1[USW-Pro-Max-16
10.0.0.106]
switch1----|2.5GbE|switch10[USW-Flex-XG
10.0.0.105]
switch1---|2.5GbE|ap2{U7 Pro Max
10.0.0.104}
router-.-|1GbE Fail Over|switch10[USW-Flex-XG
10.0.0.105]
end
subgraph Homelab
router----|1GbE|microshift[MicroShift / PiHole
10.0.0.99]
router-.-|1GbE Fail Over|truenas(TrueNas
10.0.0.3)
end
subgraph HomeLab
switch1-.-|1GbE Fail Over|kvm-1(kvm-1
10.0.0.107)
switch1-.-|1GbE Fail Over|kvm-2(kvm-2
10.0.0.108)
switch1-.-|1GbE Fail Over|kvm-3(kvm-3
10.0.0.109)
switch1---|1GbE|truenas(TrueNas
10.0.0.3)
switch10----|10 GbE|kvm-1(kvm-1
10.0.0.107)
switch10----|10 GbE|kvm-2(kvm-2
10.0.0.108)
switch10----|10 GbE|kvm-3(kvm-3
10.0.0.109)
subgraph OKD KVM-1
kvm-1-.-server-1(server-1
10.0.0.101)
kvm-1-.-infra-1(infra-1
10.0.0.121)
kvm-1-.-worker-1(worker-1
10.0.0.111)
kvm-1-.-worker-4(worker-4
10.0.0.114)
end
subgraph OKD KVM-2
kvm-2-.-server-2(server-2
10.0.0.102)
kvm-2-.-infra-2(infra-2
10.0.0.121)
kvm-2-.-worker-2(worker-2
10.0.0.112)
kvm-2-.-worker-5(worker-5
10.0.0.115)
end
subgraph OKD KVM-3
kvm-3-.-server-3(server-3
10.0.0.103)
kvm-3-.-infra-3(infra-3
10.0.0.123)
kvm-3-.-worker-3(worker-3
10.0.0.113)
kvm-3-.-worker-6(worker-6
10.0.0.116)
end
end
```
### Kubernetes
| Kubernetes Channel | OKD Version | OKD Channel | OKD OS | Host Operating System | Storage Layer |
| ------------------ | ----------- | ------------- | --------- | --------------------- | ------------- |
| v1.30.\* | 4.17-\* | stable-scos-4 | SCOS 4.18 | RHEL 9.5 | CEPH |
**Machines:**
[CPU Benchmark](https://www.cpubenchmark.net/compare/Intel-i5-6600-vs-AMD-RX-427BB-vs-Intel-i3-2130-vs-AMD-GX-415GA-SOC-vs-AMD-Ryzen-7-5700G/2594vs2496vs755vs2081vs4323)
| Machine | Model | CPU | CPU | Mem | Storage | ZFS Storage |
| ---------- | -------------- | -------- | --- | ---- | ------------------------------ | ------------ |
| pfSense | Hp t730 | RX-427BB | 4 | 4G | 16G SSD | N/A |
| MicroShift | Raspberry Pi 5 | BCM2712 | 4 | 8G | 1TB NVME | N/A |
| Bare Metal | Hp t620 | GX-415GA | 4 | 6G | 16G SSD & 16G USB | N/A |
| kvm-1 | N/A | R7-5700G | 16 | 128G | 2x4TB NVME, 1x1TB SSD,.5TB SSD | N/A |
| kvm-2 | N/A | R7-5700G | 16 | 128G | 2x4TB NVME, 1x1TB SSD,.5TB SSD | N/A |
| kvm-3 | N/A | R7-5700G | 16 | 128G | 2x4TB NVME, 1x1TB SSD,.5TB SSD | N/A |
| TrueNas | Hp ProDesk | i5-6600 | 4 | 32G | 120G SSD Boot Mirror | 5x2TB RaidZ2 |
| Spare | Hp p7-1226s | i3-2130 | 4 | 8G | 240G SSD | N/A |
| Machine | PPT | CPU Curve | GFX Curve | CPU Frequency | vMem | Memory Freq |
| ------- | --- | --------- | --------- | ------------- | ---- | ----------- |
| kvm-1 | 40W | -20 | -30 | -750 | 1.35 | 3200 |
| kvm-2 | 40W | -20 | -30 | -750 | 1.35 | 3200 |
| kvm-3 | 40W | -20 | -30 | -750 | 1.35 | 3200 |
**ZFS Storage:**
| Machine | Use | Dataset | Size | Dataset | Size | Dataset | Size | Disks (SSD) |
| ------- | ------ | --------- | ----- | ----------- | ----- | ------------- | ----- | ------------ |
| TrueNas | Backup | Nextcloud | 750GB | Ceph Backup | 175GB | WindowsBackup | 750GB | 5x2TB RaidZ2 |
**Kubernetes Nodes:**
| NAME | ROLES | Machine | vCPU | Mem | Storage |
| -------- | -------------- | ------- | ---- | ----- | --------------- |
| server-1 | cp,etcd,master | kvm-1 | 8 | 35.0G | N/A |
| server-2 | cp,etcd,master | kvm-2 | 8 | 35.0G | N/A |
| server-3 | cp,etcd,master | kvm-1 | 8 | 35.0G | N/A |
| infra-1 | infra,worker | kvm-1 | 6 | 29.0G | 1x4TB CEPH NVME |
| infra-2 | infra,worker | kvm-2 | 6 | 29.0G | 1x4TB CEPH NVME |
| infra-3 | infra,worker | kvm-3 | 6 | 29.0G | 1x4TB CEPH NVME |
| worker-1 | worker | kvm-1 | 12 | 57.0G | N/A |
| worker-2 | worker | kvm-2 | 12 | 57.0G | N/A |
| worker-3 | worker | kvm-3 | 12 | 57.0G | N/A |
#### KVM Config Dump
```bash
scp ./* [email protected]:/home/arthur/Downloads
sudo virsh dumpxml infra-1 > infra-1.xml
sudo virsh dumpxml server-1 > server-1.xml
sudo virsh dumpxml worker-1 > worker-1.xml
sudo virsh dumpxml worker-4 > worker-4.xml
sudo virsh dumpxml infra-2 > infra-2.xml
sudo virsh dumpxml server-2 > server-2.xml
sudo virsh dumpxml worker-2 > worker-2.xml
sudo virsh dumpxml worker-5 > worker-5.xml
sudo virsh dumpxml infra-3 > infra-3.xml
sudo virsh dumpxml server-3 > server-3.xml
sudo virsh dumpxml worker-3 > worker-3.xml
sudo virsh dumpxml worker-6 > worker-6.xml
```
#### OKD Longhorn Secondary Disk Setup
```bash
# https://askubuntu.com/questions/144894/add-physical-disk-to-kvm-virtual-machine
sudo mkfs.ext4 -L longhorn /dev/nvme0n1
sudo mkfs.ext4 -L longhorn1 /dev/nvme1n1
# Sandbox
sudo mkfs.ext4 -L longhorn /dev/vdb
sudo mkfs.ext4 -L longhorn1 /dev/vdc
# Pre Machine Config (Sandbox)
sudo su
echo "/dev/vdb /var/mnt/longhorn auto nofail" > /etc/fstab
sudo reboot
export NODE=""
oc annotate node ${NODE} --overwrite node.longhorn.io/default-disks-config='[{"path":"/var/mnt/longhorn","allowScheduling":true}]'
oc label node ${NODE} node.longhorn.io/create-default-disk=config
# Infra
kubectl taint node ${NODE} node-role.kubernetes.io/infra:NoSchedule
kubectl label node ${NODE} node-role.kubernetes.io/infra=""
```
#### OKD Upgrade
```bash
bash main.bash stateful_workload_stop
kubectl delete pdb -n longhorn-system --all
bash main.bash stateful_workload_start
```
#### OKD Host Disk Expansion
```bash
# KVM
sudo qemu-img resize X.raw +XG
# Node
# https://access.redhat.com/discussions/6230831#comment-2163981
sudo su
growpart /dev/vda 4
lsblk
sudo su -
unshare --mount
mount -o remount,rw /sysroot
xfs_growfs /sysroot
df -h | grep vda
```
#### OKD Host Bad Block Recovery
```bash
dd if=/mnt/source/source.raw of=/mnt/destination/destination.raw bs=4k conv=noerror,sync
```
#### OKD WIF
-
-
File Configuration Locations
```bash
ls ./terraform/gcp/HomeLab/homelab
ls ./terraform/gcp/HomeLab/homelab/wif
ls ./okd/okd-configuration/wif.yaml
```
CCOCTL Binary:
```bash
PROJECT_ID="$(vault kv get -field=project_id secret/gcp/org/av/projects)"
ccoctl gcp create-workload-identity-pool --name=okd-homelab-wif --project=homelab-${PROJECT_ID} --dry-run
ccoctl gcp create-workload-identity-provider --name=okd-homelab-wif --region=us --project=homelab-${PROJECT_ID} \
--public-key-file=serviceaccount-signer.public --workload-identity-pool=okd-homelab-wif --dry-run
```
#### Kubernetes Commands
```bash
oc login --web --server https://api.okd.arthurvardevanyan.com:6443
# Kubernetes Dashboard
# https://upcloud.com/community/tutorials/deploy-kubernetes-dashboard
kubectl get secret -n kubernetes-dashboard admin-user-token -o jsonpath="{.data.token}" | base64 --decode
# Watch ALl Pods
watch kubectl get pods -A -o wide --sort-by=.metadata.creationTimestamp
# Delete Pods that Have a Restart
kubectl get pods -A | awk '$5>0' | awk '{print "kubectl delete pod -n " $1 " " $2}' | bash -
# Drain Node
kubectl drain k3s-server --ignore-daemonsets --delete-emptydir-data
# Vault
kubectl exec -it vault-0 -n vault -- vault operator unseal --tls-skip-verify
# Nextcloud
kubectl exec -it nextcloud-0 -n nextcloud -- runuser -u www-data -- php -f /var/www/html/occ
kubectl label node ${NODE} topology.kubernetes.io/zone=${ZONE} --overwrite
```
##### Delete Pod Using Graceful Termination Eviction Request
```bash
NAMESPACE=homelab
POD=el-webhook-6b56cc5f84-clfc6
curl --header "Authorization: Bearer $(oc whoami -t)" -H 'Content-type: application/json' \
"$(oc whoami --show-server)/api/v1/namespaces/{$NAMESPACE}/pods/{$POD}/eviction" \
-d '{"apiVersion": "policy/v1","kind": "Eviction","metadata": {"name": "'"${POD}"'","namespace": "'"${NAMESPACE}"'"}}'
```
-
-
#### SSH Keyscan
```bash
export IP_LIST="3 4 5 17 107 108 109 101 102 103 111 112 113 114 115 116 121 122 123"
rm -f /tmp/ssh_keyscan.txt
for IP in $( echo "$IP_LIST" ); do
ssh-keyscan 10.0.0."${IP}" >> /tmp/ssh_keyscan.txt
done
echo "\n\n\nSSH Keyscan\n\n"
cat /tmp/ssh_keyscan.txt
```
#### Vault Kubernetes Integration
```bash
# https://blog.ramon-gordillo.dev/2021/03/gitops-with-argocd-and-hashicorp-vault-on-kubernetes/
# https://cloud.redhat.com/blog/how-to-use-hashicorp-vault-and-argo-cd-for-gitops-on-openshift
# https://itnext.io/argocd-secret-management-with-argocd-vault-plugin-539f104aff05
vault auth enable kubernetes
token_reviewer_jwt=$(kubectl get secrets -n argocd -o jsonpath="{.items[?(@.metadata.annotations.kubernetes.io/service-account.name=='argocd-repo-server')].data.token}" |base64 -d)
#kubernetes_host=$(oc whoami --show-server)
kubernetes_host="https://kubernetes.default.svc:443"
# Pod With Service Account Token Mounted
kubectl cp -n homelab toolbox-0:/var/run/secrets/kubernetes.io/serviceaccount/..data/ca.crt /tmp/ca.crt
vault write auth/kubernetes/config \
token_reviewer_jwt="${token_reviewer_jwt}" \
kubernetes_host=${kubernetes_host} \
kubernetes_ca_cert=@/tmp/ca.crt \
disable_local_ca_jwt=true
vault write auth/kubernetes/role/argocd \
bound_service_account_names=argocd-repo-server \
bound_service_account_namespaces=argocd \
policies=argocd \
ttl=1h
vault policy write argocd - <