Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/cmwylie19/hello-pepr-soak
https://github.com/cmwylie19/hello-pepr-soak
Last synced: about 1 month ago
JSON representation
- Host: GitHub
- URL: https://github.com/cmwylie19/hello-pepr-soak
- Owner: cmwylie19
- Created: 2024-04-16T13:02:30.000Z (8 months ago)
- Default Branch: main
- Last Pushed: 2024-05-08T15:19:34.000Z (8 months ago)
- Last Synced: 2024-05-08T16:57:45.945Z (8 months ago)
- Language: TypeScript
- Size: 512 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Audit: audit-policy.yaml
Awesome Lists containing this project
README
# Soak Test
- [Background](#background)
- [Cluster Setup](#cluster-setup)
- [k3d](#k3d)
- [Kind](#kind)
- [Get Started](#get-started)
- [Random Debugging](#random-debugging)
## Background
Pepr watches for `Pods` with labels `api` and `bug` and for `Secrets` with label `deletedeletedelete` in `pepr-demo` namespace.
A successful soak should result in:
1. No pods in the `pepr-demo` namespace
2. No secrets in the `pepr-demo` namespace
The Watcher deployment is running at `LOG_LEVEL` debug while the admission deployment is on info to keep the irrelevant noise down.
## Cluster Setup
#### k3d
Create a k3d cluster with audit logging enabled
```yaml
cat < audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
EOF
k3d cluster create auditer \
--k3s-arg '--kube-apiserver-arg=audit-policy-file=/etc/kubernetes/policies/audit-policy.yaml@server:*' \
--k3s-arg '--kube-apiserver-arg=audit-log-path=/var/log/kubernetes/audit.log@server:*' \
--k3s-arg '--kube-apiserver-arg=audit-log-format=json@server:*' \
--volume $(pwd)/audit-policy.yaml:/etc/kubernetes/policies/audit-policy.yaml
```
View audit logs
```bash
docker exec -it k3d-auditer-server-0 cat /var/log/kubernetes/audit.log
```
#### Kind
Create a kind cluster with auditing.
```yaml
cat < audit-policy.yaml
apiVersion: audit.k8s.io/v1
kind: Policy
rules:
- level: Metadata
EOF
cat < kind-config.yaml
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
- role: control-plane
kubeadmConfigPatches:
- |
kind: ClusterConfiguration
apiServer:
# enable auditing flags on the API server
extraArgs:
audit-log-path: /var/log/kubernetes/kube-apiserver-audit.log
audit-policy-file: /etc/kubernetes/policies/audit-policy.yaml
# mount new files / directories on the control plane
extraVolumes:
- name: audit-policies
hostPath: /etc/kubernetes/policies
mountPath: /etc/kubernetes/policies
readOnly: true
pathType: "DirectoryOrCreate"
- name: "audit-logs"
hostPath: "/var/log/kubernetes"
mountPath: "/var/log/kubernetes"
readOnly: false
pathType: DirectoryOrCreate
# mount the local file on the control plane
extraMounts:
- hostPath: ./audit-policy.yaml
containerPath: /etc/kubernetes/policies/audit-policy.yaml
readOnly: true
EOF
kind create cluster --config kind-config.yaml
```
Make sure you got audit logs
```bash
docker exec kind-control-plane cat /var/log/kubernetes/kube-apiserver-audit.log
```
Troubleshoot
```bash
docker exec kind-control-plane ls /etc/kubernetes/policies
```
expected
```bash
audit-policy.yaml
```
API Server contain the mounts and arugments?
```bash
docker exec kind-control-plane cat /etc/kubernetes/manifests/kube-apiserver.yaml | grep audit
```
expected
```yaml
- --audit-log-path=/var/log/kubernetes/kube-apiserver-audit.log
- --audit-policy-file=/etc/kubernetes/policies/audit-policy.yaml
name: audit-logs
name: audit-policies
name: audit-logs
name: audit-policies
```
## Get Started
Download [istioctl](https://istio.io/latest/docs/setup/getting-started/#download)
```bash
istioctl install --set profile=demo -y
```
Deploy the module and watch logs in one terminal
```yaml
kubectl apply -f dist
```
Logs
```bash
k logs -n pepr-system -l pepr.dev/controller=watcher -f | jq 'select(.url != "/healthz")'
```
In another terminal create 10 `CronJob`(s) that produces 10 pods with sidecars every 60 seconds
```yaml
kubectl apply -f -<