{"id":20858827,"url":"https://github.com/controlplaneio/threat-modelling-envoy-gateway-talk","last_synced_at":"2025-04-10T21:21:11.766Z","repository":{"id":204723394,"uuid":"710704204","full_name":"controlplaneio/threat-modelling-envoy-gateway-talk","owner":"controlplaneio","description":"Demos and investigation work supporting the Envoy Gateway threat model","archived":false,"fork":false,"pushed_at":"2023-10-31T13:25:24.000Z","size":198,"stargazers_count":2,"open_issues_count":5,"forks_count":2,"subscribers_count":14,"default_branch":"main","last_synced_at":"2025-03-24T18:49:40.115Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/controlplaneio.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-10-27T09:02:40.000Z","updated_at":"2023-11-26T02:07:49.000Z","dependencies_parsed_at":null,"dependency_job_id":"ac339197-e808-494c-ac2c-dfc16cfa73d8","html_url":"https://github.com/controlplaneio/threat-modelling-envoy-gateway-talk","commit_stats":null,"previous_names":["controlplaneio/threat-modelling-envoy-gateway-talk"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fthreat-modelling-envoy-gateway-talk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fthreat-modelling-envoy-gateway-talk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fthreat-modelling-envoy-gateway-talk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/controlplaneio%2Fthreat-modelling-envoy-gateway-talk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/controlplaneio","download_url":"https://codeload.github.com/controlplaneio/threat-modelling-envoy-gateway-talk/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248298987,"owners_count":21080445,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-18T04:47:41.129Z","updated_at":"2025-04-10T21:21:11.751Z","avatar_url":"https://github.com/controlplaneio.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Threat Modelling Envoy Gateway Talk\n\n## Testing Environment\n\n```mermaid\ngraph TD\n    sharedGWC[Shared GatewayClass]--\u003e|linked to| shared_EG_controller\n    shared_EG_controller--\u003e|provisions| shared_EP\n    tenantAGWC[Tenant A GatewayClass]--\u003e|linked to| tenant_A_EG_controller\n    tenant_A_EG_controller--\u003e|provisions| tenant_A_EP\n    shared_EP--\u003e|Tenant B HTTPRoute \u0026\\n Shared Gateway Listener with \\nallowedRoute from Tenant B ns| Backend_B\n    shared_EP--\u003e|Tenant C HTTPRoute \u0026\\n Shared Gateway Listener with \\nallowedRoute from Tenant C ns| Backend_C\n    tenant_A_EP--\u003e|HTTPRoute| Backend_A\n    tenant_A_EP--\u003e|Tenant A HTTPRoute \u0026 \\nTenant D ReferenceGrant| Backend_D\n    subgraph tenant-a-ns\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\n        tenant_A_EG_controller[Tenant A EG Controller]\n        tenant_A_EP[Tenant A Envoy Proxy]\n        Backend_A[Tenant A Backend]\n    end\n    subgraph tenant-b-ns\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\n        Backend_B[Tenant B Backend]\n    end\n    subgraph tenant-c-ns\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\n        Backend_C[Tenant C Backend]\n    end\n    subgraph tenant-d-ns\n        Backend_D[Tenant D Backend]\n    end\n    subgraph shared-ns\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\u0026nbsp\n        shared_EG_controller[Shared EG Controller]\n        shared_EP[Shared Envoy Proxy]\n    end\n```\n\n## Running the Example in a Kind Cluster\n\nSpin up a kind cluster:\n\n```bash\nmake cluster-up\n```\n\nThe Envoy Gateway project recommends a [multi-tenancy model](https://gateway.envoyproxy.io/v0.5.0/user/deployment-mode.html) whereby each tenant deploys their own Envoy Gateway controller in a namespace which they own. We will also explore the implications and risks associated with multiple tenants using a shared controller. As such, we will create two Envoy Gateway controllers - one dedicated controller for Tenant A, and one shared controller for Tenants B and C. To create these controllers, run the following (using Envoy Gateway v0.5.0):\n\n```bash\nmake tenant-a-controller-install\nmake shared-controller-install\n```\n\nEach Envoy Gateway controller can accept a single GatewayClass resource, so let's create a Tenant A and a shared GatewayClass:\n\n```bash\nmake gwc-tenant-a\nmake gwc-shared\n```\n\nCreate the shared Gateway in the `shared` namespace:\n\n```bash\nmake shared-gw-create\n```\n\nNote that the shared Gateway defines a listener which allows routes from the `tenant-b` and `tenant-c` namespaces to attach to it, as per our example architecture:\n\n```yaml\napiVersion: gateway.networking.k8s.io/v1beta1\nkind: Gateway\nmetadata:\n  name: eg\n  namespace: shared\nspec:\n  gatewayClassName: eg-shared\n  listeners:\n    - name: http\n      protocol: HTTP\n      port: 8080\n      allowedRoutes:  \n        namespaces:\n          from: Selector\n          selector:\n            matchExpressions:\n            - key: kubernetes.io/metadata.name\n              operator: In\n              values:\n              - tenant-b\n              - tenant-c\n```\n\n:warning: We can see how a potential risk could occur by looking at this listener. Imagine that instead of the `kubernetes.io/metadata.name` label (which is automatically set to the name of the namespace), we used a custom label. In this case, a malicious internal actor with the ability to label namespaces would be able to change the set of namespaces supported by the Gateway. With some social engineering, users of a legitimate service which uses the shared gateway could possibly be tricked into sending traffic to a malicious backend. We can look at an example of this once we've set up our backend services for all the tenants.\n\nCreate backend services and deployments for Tenants A, B, C and D, noting that we will also create a dedicated Gateway for Tenant A:\n\n```bash\nmake run-backend-services\n```\n\nNote the creation of `HTTPRoute`s in each of Tenant A, B and C's namespaces. For Tenant A, the `parentRef` is its own Gateway:\n\n```yaml\napiVersion: gateway.networking.k8s.io/v1beta1\nkind: HTTPRoute\nmetadata:\n  name: backend\n  namespace: tenant-a\nspec:\n  parentRefs:\n    - name: eg\n  hostnames:\n    - \"www.tenant-a.example.com\"\n  rules:\n    - backendRefs:\n        - group: \"\"\n          kind: Service\n          name: backend\n          port: 3000\n          weight: 1\n      matches:\n        - path:\n            type: PathPrefix\n            value: /\n```\n\nFor Tenants B and C, the `parentRef` refers to the shared Gateway:\n\n```yaml\napiVersion: gateway.networking.k8s.io/v1beta1\nkind: HTTPRoute\nmetadata:\n  name: backend\n  namespace: tenant-c\nspec:\n  parentRefs:\n    - name: eg\n      namespace: shared\n  hostnames:\n    - \"www.tenant-c.example.com\"\n  rules:\n    - backendRefs:\n        - group: \"\"\n          kind: Service\n          name: backend\n          port: 3000\n          weight: 1\n      matches:\n        - path:\n            type: PathPrefix\n            value: /\n```\n\nFor now, we have not set up a route to Tenant D's backend.\n\n### Access Tenant A's Service\n\nSet up port forwarding to the Tenant A Gateway:\n\n```bash\nmake port-forward-tenant-a\n```\n\nCurl the service:\n\n```bash\nmake curl-tenant-a\n```\n\n### Access Tenant B and C Services\n\nSet up port forwarding to the Shared Gateway:\n\n```bash\nmake port-forward-shared-tenants\n```\n\nCurl the Tenant B service:\n\n```bash\nmake curl-tenant-b\n```\n\nCurl the Tenant C service:\n\n```bash\nmake curl-tenant-c\n```\n\n#### Malicious HTTPRoute Example\n\n:warning: We explored the risk earlier that in the context of the shared gateway topology, a tenant could create a malicious backend and HTTPRoute, and attempt to use social engineering techniques to direct a user of a co-tenant's legitimate service to the malicious backend. Let's consider the case where a threat actor has the ability to create HTTPRoutes in the `tenant-c` namespace:\n\n```bash\nmake malicious-httproute\n```\n\nNow let's imagine that this threat actor can trick a user of Tenant B's service to visit `www.tenant-b.example.com/totally-legit`:\n\n```bash\nmake curl-tenant-b-malicious\n```\n\nNote that the response is served by the backend pod in `\"namespace\": \"tenant-c\"`\n\n### Exploring ReferenceGrants with Tenant D\n\nSo far, we have not set up a route to Tenant D's backend Service. Note that until now, all `HTTPRoute`s have been created in the same namespaces as the backend which they route to.\n\nImagine someone with the permission to create `HTTPRoute`s in the `tenant-a` namespace (as in [cross-ns-route.yaml](./manifests/cross-ns-route.yaml)) tries to create a route to Tenant D's backend service:\n\n```bash\nmake cross-ns-route-create\n```\n\nIf we try to access Tenant D's Service via Tenant A's Gateway, we will see an error:\n\n```bash\nmake curl-tenant-d\n```\n\nIn accordance with Gateway API [security principles](https://gateway-api.sigs.k8s.io/concepts/security-model/), for such a cross-namespace connection to succeed, a handshake has to occur between namespaces. In this case, as the `HTTPRoute` was created in the `tenant-a` namespace, a [ReferenceGrant](https://gateway-api.sigs.k8s.io/api-types/referencegrant/) would need to exist in the `tenant-d` namespace. Let's demonstrate this:\n\n```bash\nmake ref-grant-create\n```\n\nTry accessing the Tenant D Service again, and observe that it now works:\n\n```bash\nmake curl-tenant-d\n```\n\n:warning: In this case, if a threat actor with the ability to create a ReferenceGrant in the `tenant-d` namespace wanted to set up an HTTPRoute to a malicious backend, they would need to compromise a service account or set of user credentials which would allow them to create an HTTPRoute in the `tenant-a` namespace.\n\n## Compromised Proxy Scenario\n\n:warning: We have explored what an attacker would need to do in order to create a malicious HTTPRoute. An even more attractive target would be for the attacker to take over an Envoy Proxy instance, either by exploiting an existing vulnerability to achieve RCE (remote code execution), or by configuring Envoy Gateway to deploy a proxy using a vulnerable or maliciously crafted image. Let's look at the more naive case, whereby a new tenant (Tenant E) simply deploys a maliciously crafted Envoy Proxy.\n\nFirst, we will build an image and load it into our kind cluster. For this demo, we are simply going to install [tcpdump](https://www.tcpdump.org/) on top of an Envoy Proxy base image, and copy in the `tcpdump.sh` script, which we can run from within the container to demonstrate the threat:\n\n```bash\nmake build-malicious-envoy\n```\n\nEnvoy Gateway allows you to customise the Envoy Proxy image as per the documentation [here](https://gateway.envoyproxy.io/v0.5.0/user/customize-envoyproxy/#customize-envoyproxy-image). This can be used to ensure that your image complies with any mandated security standards, but in this case, we are going to use our toy maliciously crafted image.\n\nLet's deploy the Tenant E infrastructure, including the new GatewayClass and EnvoyProxy:\n\n```bash\nmake tenant-e-infra-install\n```\n\nRun the `tcpdump.sh` script from within the Envoy Proxy container:\n\n```bash\nmake exec-into-proxy-e\n```\n\nOpen a new terminal window, set up port forwarding and curl the Tenant E service:\n\n```bash\nmake port-forward-tenant-e\nmake curl-tenant-e\n```\n\nSwitch back to the original terminal window and observe the request that we just sent being captured. An attacker with access to (or malicious code running inside) a proxy may want to send captured traffic to an attacker controlled service, or tamper with traffic in transit.\n\n## Compromised Controller Scenario\n\n:warning: To affect the Confidentiality, Integrity or Availability of user data, an attacker will ultimately need to compromise the data plane (i.e. the Envoy Proxy). This could be carried out by a malicious insider, as shown in the previous section, or by malicious external actors who could, for example:\n\n- :warning: carry out a supply chain attack resulting in a malicious image being run\n- :warning: exploit known vulnerabilities in an unpatched Envoy Proxy\n- :warning: exploit weaknesses in the Kubernetes cluster to move laterally from a compromised pod to an Envoy Pod\n\n:warning: To further explore the consequences of [multi-tenancy](https://gateway.envoyproxy.io/latest/user/deployment-mode/#multi-tenancy), let's consider a general scenario where we have multiple Gateways in different namespaces within the cluster (each referencing a different GatewayClass). Within this simple scenario, one of the Gateway pods is compromised (e.g. through one of the above routes). Now, could our attacker modify an EnvoyProxy in a different namespace to that of the compromised Envoy Gateway?\n\nNote that in our example, we have not used Envoy Gateway's [namespaced mode](https://github.com/envoyproxy/gateway/pull/1656) which will be available in v0.6.0, so each Envoy Gateway has Kubernetes permissions defined by a ClusterRole, and watches for resources in all namespaces as detailed [here](https://gateway.envoyproxy.io/latest/user/deployment-mode/). Let's imagine that Tenant A has been compromised, and the attacker wants to modify the Shared Envoy Proxy. Firstly we can look at the Envoy Gateway ClusterRole used by Tenant A's Gateway:\n\n```bash\nkubectl describe clusterrole eg-tenant-a-gateway-helm-envoy-gateway-role | grep envoyproxies\n```\n\nNote that we cannot patch existing envoy proxies, so by using Tenant A's Envoy Gateway Service Account, our attacker will not be able to modify the EnvoyProxy referred to by the Shared GatewayClass. However, note that we can patch GatewayClasses:\n\n```bash\nkubectl describe clusterrole eg-tenant-a-gateway-helm-envoy-gateway-role | grep gatewayclasses\n```\n\nLet's use the `./scripts/perform-action-as-gateway.sh` script to see if we can patch the Shared GatewayClass to use a malicious EnvoyProxy in the `tenant-a` namespace using the Envoy Gateway's Service Account. Firstly we will create the EnvoyProxy (note that this resource cannot be created using the EG Service Account):\n\n```bash\nmake create-malicious-proxy\n```\n\nNow we will patch the Shared GatewayClass using the Tenant A Gateway Service Account:\n\n```bash\nmake patch-gatewayclass\n```\n\nLet's force the Envoy Gateway and Proxy pods to restart:\n\n```bash\nmake restart-shared-pods\n```\n\nObserve that the Envoy image has not changed!\n\n```bash\nmake grep-shared-envoy-image\n```\n\nThis is by design in Gateway API! As per the [API specification](https://gateway-api.sigs.k8s.io/v1alpha2/references/spec/#gateway.networking.k8s.io/v1alpha2.GatewayClass): \"a Gateway is based on the state of the GatewayClass at the time it was created and changes to the GatewayClass or associated parameters are not propagated down to existing Gateways. This recommendation is intended to limit the blast radius of changes to GatewayClass or associated parameters.\"\n\n:warning: However, we must still be careful with ClusterRoles. For example, we can check that the Tenant A Envoy Gateway can observe secrets across the cluster:\n\n```bash\n./scripts/perform-action-as-gateway.sh kubectl describe secrets -A\n```\n\n## Talk Demos\n\nThe minimal base infra required to run the demos for the talk can be set up using:\n\n```bash\nmake demo-base-infra\n```\n\nThe `demo-magic` scripts [demo-1.sh](./demo-1.sh), [demo-2.sh](./demo-2.sh), and [demo-3.sh](./demo-3.sh) can then be run in order.\n\n## Teardown\n\nIf you are still port warding connections, you can stop by running:\n\n```bash\nmake stop-port-forwarding\n```\n\nTo delete the kind cluster, run:\n\n```bash\nmake cluster-down\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontrolplaneio%2Fthreat-modelling-envoy-gateway-talk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcontrolplaneio%2Fthreat-modelling-envoy-gateway-talk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcontrolplaneio%2Fthreat-modelling-envoy-gateway-talk/lists"}