{"id":15577860,"url":"https://github.com/shanepeckham/aks_security","last_synced_at":"2025-04-24T02:10:19.940Z","repository":{"id":69130414,"uuid":"138564339","full_name":"shanepeckham/AKS_Security","owner":"shanepeckham","description":null,"archived":false,"fork":false,"pushed_at":"2018-07-20T10:31:40.000Z","size":2663,"stargazers_count":46,"open_issues_count":0,"forks_count":24,"subscribers_count":11,"default_branch":"master","last_synced_at":"2025-04-18T09:34:31.806Z","etag":null,"topics":["azure","azure-active-directory","kubernetes","rbac-management","security"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/shanepeckham.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2018-06-25T08:18:05.000Z","updated_at":"2024-08-12T19:39:36.000Z","dependencies_parsed_at":null,"dependency_job_id":"fe3e2e64-9959-4a4a-b248-811366db3fa7","html_url":"https://github.com/shanepeckham/AKS_Security","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shanepeckham%2FAKS_Security","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shanepeckham%2FAKS_Security/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shanepeckham%2FAKS_Security/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/shanepeckham%2FAKS_Security/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/shanepeckham","download_url":"https://codeload.github.com/shanepeckham/AKS_Security/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250546086,"owners_count":21448260,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["azure","azure-active-directory","kubernetes","rbac-management","security"],"created_at":"2024-10-02T19:02:00.599Z","updated_at":"2025-04-24T02:10:19.933Z","avatar_url":"https://github.com/shanepeckham.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"![alt text](http://www.thingx.cloud/wp-content/uploads/2018/04/kubernates_azure_thumb-1.png)\n\n# Deploying secure solutions on Azure Kubernetes Service\n\nMost Kubernetes security breaches are due to humar error, deploying with defaults, and not locking down components.\n\n## Key risks of an insecure Kubernetes cluster\n\n* Access to sensitive data\n* Ability to take over a Kubernetes cluster with elevated privileges\n* Gain root access to Kubernetes worker nodes\n* Run workloads or access components outside the Kubernetes cluster\n* Deploying unvetted malicious images on to the cluster\n\n## First Principles:\n\n* Apply least privileged access\n* Segregation of responsibility\n* Integrate security into DevOps\n* Trust your sources\n* Minimise attack surfaxce\n* Apply security in a layered approach\n\n## Kubernetes best practices\n\n* Authentication RBAC\n* Authorisation\n* Network Segmentation - tightly control all communication\n* Pod Security Policy\n* Encrypt Secrets\n* Auditing\n* Admission Controllers\n* Layered security approach\n* Label everything for granular control\n* Apply networking segmentation at Level 4 (e.g. Kuberouter) and Level 7 (Istio, Linkerd)\n* A user should not be able to override Kubernetes security by crafting a YAML file if layered security controls have been successfully implemented\n* Create administrative boundaries between resources\n* Store secrets centrally, preferably in a secure valault such as Azure Key Vault\n\nThe following Kubernetes blog also contains a wealth of information: [11 Ways (Not) to Get Hacked](https://kubernetes.io/blog/2018/07/18/11-ways-not-to-get-hacked/)\n\n## Container Level \n\n* Use a trusted registry so that only authorised images are depployed to the cluster. Introduce a process to approve images for uploading to registry\n* Regularly apply security updates to cluster and container images (AKS will auto patch. Azure automatically applies security patches to the nodes in an AKS cluster on a nightly schedule\n* Avoid access to HOST PIC namespace - only if absolutely necessary\n* Avoid access toi Host PID namespace - only if absolutely necessary\n* A pod policy cannot necessarily protect against a container image that has privileged root access\n* Apply [AppArmor](https://kubernetes.io/docs/tutorials/clusters/apparmor/) security profile \n* Apply [seccomp](https://docs.docker.com/engine/security/seccomp/)\n* Apply SELinux policy\n\n**Sandboxed containers - options include**\n* [Kata containers](https://katacontainers.io/)\n* [Gvisor containers](https://github.com/google/gvisor)\n\n### Scan container - solutions include:\n\n* [Aqua](www.aquasec.com)\n* [Twistlock](https://www.twistlock.com/)\n* [Docker Bench for security](https://github.com/docker/docker-bench-security)\n* [CoreOS Clair](https://github.com/coreos/clair)\n* [OpenScap](https://www.open-scap.org/tools/)\n* [Neuvector](https://neuvector.com/container-compliance-auditing-solutions/)\n* Scan image with Aqua MicroScanner - https://github.com/aquasecurity/microscanner - can be run be developer on dev workstation prior to uploading to container registry\n\nAdd the following to the Dockerfile (get your token docker run --rm -it aquasec/microscanner --register [email address])\n\n```\nADD https://get.aquasec.com/microscanner /\nRUN chmod +x /microscanner\nRUN /microscanner \u003cTOKEN\u003e [--continue-on-failure]\n```\n\n* [Secure Docker](https://www.cisecurity.org/benchmark/docker/)\n\n## Pod Level\n\n### PodSecurityPolicies are only available if admission controllers have been implemented - dynamic admission controllers and initialisers (alpha) are available in 1.10 - \n\n**PodSecurityPolicy can:**\n\n* Avoid privilged containers from being run\n* Avoid containers that use the root namespaces from being run\n* Limit permissions on bvolume types that can be used\n* Enforce read only access to root file system\n* Ensure SELinux and AppArmor context\n* Apply Secomp/SELinux/App Armor profile\n* Can disable hostPath volumes\n* Restrict access to Host PID\n* Avoid priviled pods\nAdd security context, see:\nhttps://kubernetes.io/docs/tasks/configure-pod-container/security-context/\nhttps://kubernetes.io/docs/concepts/policy/pod-security-policy/\nhttps://sysdig.com/blog/kubernetes-security-psp-network-policy/\n* Exposed credentials\n* Mount host with write access\n* Expose unnecessary ports\n\n#### Use AllwaysPullImages\n\n* Force registry authentication and can prevent other pods using the image\n* Only those with correct credentials can pull pod\n* Can result in a crashloopbackoff if the credentials are not provided or incorrect\n\n#### Use DenyEscalatingExec\n\n* If container has priviliged access, user this DenyEscalatingControl as mitigation as this will deny user trying to issue kubectl exec against the image and gain access to the node/cluster\n\n### Namespace level\n\n* Define allowed communication between namespaces using network policies\n\n**By applying a ResourceQuota, DoS attacks that target on malicious resource consumptio can be mitigated against. Apply a ResourceQuote admission controller to restrict resources such as:**\n* CPU\n* Memory\n* Pods\n* Services\n* ReplicationControllers\n* ResourceQuota\n* Secrets\n* PersistentVolumeClaims\n\n**Apply RBAC - this operates at the namespace level**\n\n### Node level\n\n#### Use admission controller webhooks to prevent intra-pod leakage, exposed secrets/ config maps etc:\n\n* Limit the Node and Pod that a kubelet can modify\n* Enforce that kubelets must use credentials in system nodes\n* Limit SSH access to nodes - this is possible with AKS https://docs.microsoft.com/en-us/azure/aks/aks-ssh. Use kubectl exec instead if absolutely necessary - see DenyEscalating policy\n\n### Cluster level\n\n**RBAC for Kubelet flags**\n* ```--authorization-mode=RBAC,Node```\n* ```--admission-control=...,NodeRestriction```\n* Rotate certs  ```--rotate-certificates```\n\n#### Admission Controllers (Webhooks)\n\n* Operates at the API Server level\n* Intercepts request before it is persisted to etcd\n* Occurrs after authentication\n* Only cluster admin can configure an admission controller\n* Failure to configure the admission controller results in other functionality not being available\n \n\n**Two types of admission control**\n\n* Mutuating - can modify the request\n* Validation - can only validate, not modify\n\n**Any request that is rejected will fail and pass an error message to the user. Admission controllers are:**\n\n* Developed out of tree and configured at runtime\n* Facilitates dynamic action responses\n* Should be within the same cluster\n\n**Available in Kubernetes 1.10 - \n\n**The following are the recommended admission controllers:**\n* NamespaceLifeCycle\n* LimitRanger\n* ServiceAccount\n* DefaultStorageClass\n* DefaultTolerationSeconds\n* MutuatingAdmissionWebhoon\n* Validating AdmissionWebhook\n* ResourceQuota\n\n**Applying the ImagePolicyWebhopok allows an external service to be invoked (Aqua, Twistlock) for scanning at the cluster level will protect against:**\n\n* Images running vulnerabilities\n* Images running malware\n* Images that embed secrets\n* Images that run as UID 0 (root privileges)\n\n#### Apply network segmentation, tools include:\n\n* [Weave Net](https://kubernetes.io/docs/tasks/administer-cluster/network-policy-provider/weave-network-policy/)\n* [Kube-router](https://www.kube-router.io/) - [ahmetb's has some examples here](https://github.com/ahmetb/kubernetes-network-policy-recipes) for examples \n* [Cillium](https://cilium.io/)\n* [Trireme](https://github.com/aporeto-inc/trireme-kubernetes)\n\n#### Apply service mesh and application routing with mutual TLS\n\n* [Istio Service Mesh](https://istio.io/)\n* [Linkerd Service Mesh](https://linkerd.io/)\n* [Heptio Contour](https://heptio.com/products/#heptio-contour)\n* [Twistlock cloud native firewall](https://www.twistlock.com/platform/cloud-native-firewall/)\n\n#### Manage configuration\n\n* [Heptio Sonobuoy](https://heptio.com/products/#heptio-sonobuoy)\n\n#### Kubernetes conformance tests\n\n* [Heptio Sonobuoy Scanner](https://scanner.heptio.com/)\n* [kubesec.io](https://kubesec.io/)\n* [Falco](https://sysdig.com/opensource/falco/)\n\n### Azure level\n\n#### RBAC\n* Integrate AKS RABC with Azure Active Directory - https://docs.microsoft.com/en-us/azure/aks/aad-integration\n\n\n* Encrypt Storage [encrypt data at rest](https://docs.microsoft.com/en-us/azure/storage/common/storage-service-encryption)\n* Apply regular updates- Azure automatically applies security patches to the nodes in your cluster on a nightly schedule\n* Apply NSGs for cross cluster communication\n\n### CI/CD pipeline\n\n* Add scanning to pipeline build\n\n### Auditing and Logging\n\n**Audit everything at the cluster level,  tools include:**\n* [AKS containerlogging](https://docs.microsoft.com/en-us/azure/monitoring/monitoring-container-health)\n* [Fluentd](https://www.fluentd.org/)\n* [Grafana](https://grafana.com/)\n* [Kibana](https://www.elastic.co/products/kibana)\n* [Prometheus](https://prometheus.io/)\n\n# Additional resource for security\n* Kube-Bench open source tool- CIS benchmark testing - https://github.com/aquasecurity/kube-bench . This will raise issues and remediations\n* Kube-Hunter - penetration testing tool to be run by the security team. Identify key security risks at the cluster level. In private beta and will be a free tool\n* [Aqua Microscanner](https://github.com/aquasecurity/microscanner) to assess security of image at build time. Can be run on developer workstation prior to upload to regstry\n* Using Kured, an open-source reboot daemon for Kubernetes. Kured runs as a DaemonSet and monitors each node for the presence of a file indicating that a reboot is required. It then orchestrates those reboots across the cluster, following the same cordon and drain process described earlier. - https://github.com/weaveworks/kured\n\n\n\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshanepeckham%2Faks_security","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fshanepeckham%2Faks_security","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fshanepeckham%2Faks_security/lists"}