Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/linuxacademy/webinar-automating-kubernetes-security

Will Boyd 10.30.2020 ACG webinar
https://github.com/linuxacademy/webinar-automating-kubernetes-security

Last synced: about 1 year ago
JSON representation

Will Boyd 10.30.2020 ACG webinar

Awesome Lists containing this project

README 

          
# Automating Kubernetes Security



This is a walkthrough guide for the live demo performed during the `Automating Kubernetes Security` webinar.



## Cluster Setup

Set up a Kubernetes cluster with `kubeadm`, or use an existing one.



Demo environment info:

- OS: `Ubuntu 18.04 Bionic Beaver LTS`

- Container Runtime: `Docker 19.03.12`

- Kubernetes version: `1.19.2`

- Networking Plugin: `Calico`



## Set Up a Namespace and Non-Admin ServiceAccount to Use for Testing

Create a Namespace.



```

kubectl create ns development

```



Create the ServiceAccount.



```

kubectl create sa nonadmin -n development

```



Create a RoleBinding allowing the account to edit objects in the `development` Namespace.



```

vi rb-nonadmin-edit.yml

```



```

apiVersion: rbac.authorization.k8s.io/v1

kind: RoleBinding

metadata:

  name: rb-nonadmin-edit

  namespace: development

roleRef:

  kind: ClusterRole

  name: edit

  apiGroup: rbac.authorization.k8s.io

subjects:

- kind: ServiceAccount

  name: nonadmin

  namespace: development

```



```

kubectl create -f rb-nonadmin-edit.yml --save-config

```



Verify that you can run commands as the `nonadmin` user. You should see the message `No resources found in development namespace.` since no Pods have been created in the Namespace.



```

kubectl --as=system:serviceaccount:development:nonadmin get pods -n development

```



## Create a PodSecurityPolicy

Create a new PodSecurityPolicy that will prevent pods from using privileged mode.



```

vi psp-nopriv.yml

```



```

apiVersion: policy/v1beta1

kind: PodSecurityPolicy

metadata:

  name: psp-nopriv

spec:

  privileged: false

  runAsUser:

    rule: RunAsAny

  fsGroup:

    rule: RunAsAny

  seLinux:

    rule: RunAsAny

  supplementalGroups:

    rule: RunAsAny

  volumes:

  - configMap

  - downwardAPI

  - emptyDir

  - persistentVolumeClaim

  - secret

  - projected

```



Create the PodSecurityPolicy.



```

kubectl create -f psp-nopriv.yml --save-config

```



## Set Up RBAC to Authorize the Use of the PodSecurityPolicy for a New ServiceAccount

Create a ClusterRole that allows the use of our PodSecurityPolicy.



```

vi cr-use-psp.yml

```



```

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

  name: cr-use-psp

rules:

- apiGroups: ['policy']

  resources: ['podsecuritypolicies']

  verbs:     ['use']

  resourceNames:

  - psp-nopriv

```



Create the ClusterRole.



```

kubectl create -f cr-use-psp.yml --save-config

```



Create a ServiceAccount.



```

kubectl create sa nonprivileged -n development

```



Create a RoleBinding that allows the `nonprivileged` ServiceAccount to use the PodSecurityPolicy in the `default` Namespace.



```

vi rb-nonprivileged.yml

```



```

apiVersion: rbac.authorization.k8s.io/v1

kind: RoleBinding

metadata:

  name: rb-nonprivileged

  namespace: development

roleRef:

  kind: ClusterRole

  name: cr-use-psp

  apiGroup: rbac.authorization.k8s.io

subjects:

- kind: ServiceAccount

  name: nonprivileged

  namespace: development

```



Create the RoleBinding.



```

kubectl create -f rb-nonprivileged.yml --save-config

```



## Turn on the PodSecurityPolicy Admission Controller



```

sudo vi /etc/kubernetes/manifests/kube-apiserver.yaml

```



Locate the line for the `--enable-admission-plugins` and add `PodSecurityPolicy` to the list.



```

- --enable-admission-plugins=NodeRestriction,PodSecurityPolicy

```



Run a command to make sure the API Server is still responding after the change to the manifest file (it may take a few moments to come back up).



```

kubectl get nodes

```



## Create a Pod

Create a basic Pod, using the `nonadmin` ServiceAccount created earlier.



```

vi pod-basic.yml

```



```

apiVersion: v1

kind: Pod

metadata:

  name: pod-basic

  namespace: development

spec:

  containers:

  - name: nginx

    image: nginx

```



```

kubectl create -f pod-basic.yml --as=system:serviceaccount:development:nonadmin --save-config

```



This operation will fail, since the Pod does not have access to any PodSecurityPolicies that would allow it to be created.



Let's try again, this time ensuring the Pod has access to our PodSecurityPolicy by giving it the `nonprivileged` ServiceAccount.



```

vi pod-with-psp-access.yml

```



```

apiVersion: v1

kind: Pod

metadata:

  name: pod-with-psp-access

  namespace: development

spec:

  serviceAccountName: nonprivileged

  containers:

  - name: nginx

    image: nginx

```



```

kubectl create -f pod-with-psp-access.yml --as=system:serviceaccount:development:nonadmin --save-config

```



This time, it should succeed since the Pod's ServiceAccount has access to the PodSecurityPolicy, and the Pod does not violate any policies.



Examine the Pod with `kubectl describe`.



```

kubectl describe pod pod-with-psp-access -n development

```



Note that there an annotation which lists the PodSecurityPolicy that allowed the Pod.



Let's try to create a Pod that does violate the policy by requesting privileged access.



```

vi pod-privileged.yml

```



```

apiVersion: v1

kind: Pod

metadata:

  name: pod-privileged

  namespace: development

spec:

  serviceAccountName: nonprivileged

  containers:

  - name: nginx

    image: nginx

    securityContext:

      privileged: true

```



```

kubectl create -f pod-privileged.yml --as=system:serviceaccount:development:nonadmin --save-config

```



This should fail, since the privileged mode containers are not allowed by the PodSecurityPolicy.



## Fix Mirror Pod Creation for Static Pods

List your system Pods.



```

kubectl get pods -n kube-system

```



You may notice that the kube-apiserver mirror Pod is missing. While the API Server itself is in fact running since it is managed by kubelet and bypasses PodSecurityPolicies, the PodSecurityPolicies are preventing the mirror Pod from being created. This makes these system Pods invisible in the Kubernetes API. Let's fix it!



Create a new PodSecurityPolicy that will allow static mirror Pods.



```

vi psp-static.yml

```



```

apiVersion: policy/v1beta1

kind: PodSecurityPolicy

metadata:

  name: psp-static

spec:

  allowPrivilegeEscalation: true

  fsGroup:

    rule: RunAsAny

  hostNetwork: true

  runAsUser:

    rule: RunAsAny

  seLinux:

    rule: RunAsAny

  supplementalGroups:

    rule: RunAsAny

  volumes:

  - configMap

  - downwardAPI

  - emptyDir

  - persistentVolumeClaim

  - secret

  - projected

  - hostPath

```



```

kubectl create -f psp-static.yml --save-config

```



Create a ClusterRole with access to the policy.



```

vi cr-use-psp-static.yml

```



```

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRole

metadata:

  name: cr-use-psp-static

rules:

- apiGroups: ['policy']

  resources: ['podsecuritypolicies']

  verbs:     ['use']

  resourceNames:

  - psp-static

```



```

kubectl create -f cr-use-psp-static.yml --save-config

```



Create a ClusterRoleBinding to allow Kubernetes Nodes to use the policy.



```

vi crb-use-psp-static.yml

```



```

apiVersion: rbac.authorization.k8s.io/v1

kind: ClusterRoleBinding

metadata:

  name: crb-use-psp-static

roleRef:

  kind: ClusterRole

  name: cr-use-psp-static

  apiGroup: rbac.authorization.k8s.io

subjects:

- kind: Group

  name: system:nodes

```



```

kubectl create -f crb-use-psp-static.yml --save-config

```



List system Pods again.



```

kubectl get pods -n kube-system

```



You should see a `kube-apiserver` Pod appear. If it doesn't, you may need to wait a minute or two and try again.