{"id":18425446,"url":"https://github.com/xvzf/lightpath","last_synced_at":"2026-04-25T21:34:21.338Z","repository":{"id":181596978,"uuid":"486890362","full_name":"xvzf/lightpath","owner":"xvzf","description":"[Master Thesis] Lightpath - protocol aware proxies as kube-proxy replacement","archived":false,"fork":false,"pushed_at":"2023-07-16T11:11:12.000Z","size":569,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-02-16T08:15:42.764Z","etag":null,"topics":["distributed-systems","envoy","golang","kubernetes","loadbalancing"],"latest_commit_sha":null,"homepage":"https://stl.htwsaar.de/tr/STL-TR-2023-02.pdf","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/xvzf.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-04-29T08:11:44.000Z","updated_at":"2023-10-11T17:01:56.000Z","dependencies_parsed_at":"2023-07-16T12:37:01.632Z","dependency_job_id":null,"html_url":"https://github.com/xvzf/lightpath","commit_stats":null,"previous_names":["xvzf/lightpath"],"tags_count":3,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvzf%2Flightpath","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvzf%2Flightpath/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvzf%2Flightpath/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/xvzf%2Flightpath/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/xvzf","download_url":"https://codeload.github.com/xvzf/lightpath/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248758435,"owners_count":21156957,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["distributed-systems","envoy","golang","kubernetes","loadbalancing"],"created_at":"2024-11-06T05:04:00.038Z","updated_at":"2026-04-25T21:34:21.281Z","avatar_url":"https://github.com/xvzf.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Lightpath - a kube-proxy replacement based on Envoy (PoC)\n\n\u003e :warning: This software is by no means production ready! It is neither feature complete nor has full test coverage!\n\nLightpath has been the underlying reference-implementation for my Master Thesis *Evaluating the usage of protocol-aware proxies as replacement for kube-proxy*. The thesis has been published [here](https://stl.htwsaar.de/tr/STL-TR-2023-02.pdf)\n\n\n# TL;DR Research Results\nThe layer 7 proxy Envoy has proven to be effective for adding visibility and increasing the reliability of microservices.\nWhile the layer 7 processing comes with the tradeoff of consuming more compute resources, Lightpath outperformed the kube-proxy implementation in terms of consistency, overload handling, and reliability.\nHowever, for most requests, the layer 4 operations of kube-proxy are more efficient on the latency, resulting in a 61% latency decrease for 50% of subjected requests compared to Lightpath.\nConsidering the tail-latency distributions and the total average, Envoy improves the situation with a 25.7% lower average latency, and there are up to 80% improvements for the 99th latency percentile.\nThe connection latency can be decreased by leveraging the eBPF technology instead of iptables for redirecting requests to ClusterIPs or ExternalIPs to Envoy.\n\nThe advanced traffic control options (circuitbreaking, retries, ...) improve application resilience and allow to decrease application code complexity further, e.g. by offloading retry operations to Envoy, which has full knowledge of the system topology.\n\nUsing the system topology and reacting to passive health checks allows quicker reactivity to changes on endpoints or the network state.\nAn interesting idea would be to transfer information on passively detected failures to Kubernetes, which otherwise would wait for a periodically executed active health check to fail.\n\nAs Kubernetes is also considered in a more distributed deployment model with edge computing, the reactivity of individual nodes which might not have consistent network connectivity speeds or latency would likely improve the availability and consistency.\nIn this context, the option for optimizing load balancing for latency by using e.g. a peak exponentially-weighed moving average based algorithm could improve reactivity further.\n\n## Components\n\nLightpath consists of multiple data plane and control plane components:\n1. **Control Plane**:\n  - `controlplane` (node-scoped): runs on every node and acts as control server for Envoy. Could eventuall scoped per failure-domain.\n  - `webook` (cluster-scoped): The webhook components intercepts the service creation and updates the handling proxy to Lightpath. This allows a hybrid deployment next to the kube-proxy\n2. **Data Plane**:\n  - `proxy`(node-scoped): Envoy acts as proxy instance and is configured to connect to the node-local control plane\n  - `redirect` configures the iptables redirect targets for ClusterIPs handled by Lightpath\n\n## Limitations\nAs a PoC, the TCP and HTTP protocol are targeted. STCP and UDP based protocols are not supported (yet) and need to be handled by kube-proxy. Therefore, a hybrid setup is required for full service connectivity.\n\n## Deployment\n\nLightpath can be deployed on any Kubernetes Cluster with an iptables based CNI (e.g. Calico, Flannel, ...).\n\n### Test cluster\nThe `make k8s-up` command bootstraps a new [Kubernetes in Docker](https://kind.sigs.k8s.io/docs/user/quick-start/), which can be used for further testing\n\n### Deploying\n\u003e Lightpath images are (publicly) available for linux/amd64 and linux/arm64.\n\nThe deployment manifests are based on [Kustomize](http://kustomize.io) and are generated with `kustomize build deploy/default/`. Lightpath requires cert-manager to be installed on the tareted cluster; the installation will fail otherwise. The `make k8s-up` command already setups all required dependencies.\n\nAs a shortcut, `make deploy` applies the kustomize manifests to Kubernetes results.\n\nA test deployment running a simple echo server can be applied to a cluster with `kubectl apply -f hack/ci/test-deploy.yaml`.\nA new pod with an interactive shell can be bootstrapped using e.g. `kubectl run tmp-net-debug-shell -it --rm --image nicolaka/netshoot -- /bin/bash`, which can then be used to access the deployed service:\n```\nbash-5.1# curl whoami.whoami.svc\nHostname: whoami-5978d4b87d-dwx2t\nIP: 127.0.0.1\nIP: ::1\nIP: 10.244.2.3\nIP: fe80::e8f1:57ff:fefa:2e66\nRemoteAddr: 172.18.0.3:42850\nGET / HTTP/1.1\nHost: whoami.whoami.svc\nUser-Agent: curl/7.83.1\nAccept: */*\nX-Envoy-Attempt-Count: 1\nX-Envoy-Expected-Rq-Timeout-Ms: 5000\nX-Forwarded-Proto: http\nX-Request-Id: c686bc0b-1e33-445c-b945-9ae883313303\n```\n\nThe X-Envoy headers indicate that the request has been successfully handled by lightpath. Furthermore, the Envoy admin interface can ben accessed by e.g. port-forwarding the configured admin port (15000) to the local machine:\n```bash\n$ kubectl port-forward $(kubectl get pod -oname -l app.kubernetes.io/name=proxy | head -n 1) 15000:15000 \u0026\n\n$ # E.g. retrieve the configuration dump:\n$ curl http://localhost:15000/config_dump\n# ...\n```\n\n## Repository Structure\n\nOverall, the repository acts as mono-repository for all lightpath components, configuration and performed benchmarks performed throughout the master thesis.\n\nThe actual application source code is distributed in the `cmd`, `pkg` and `internal` folders, with the `cmd/{controlplane,redirect}` holding the two Golang based services.\nThe webhook is implemented in the Rego language as part of the [open policy agent](https://www.openpolicyagent.org) in the Kustomize based Kubernetes manifests (`deploy/{proxy,redirect,webhook,controlplane}`). The `deploy/default` Kustomization acts as meta package for all components.\n\nThe main Golang packages are located in `pkg/state/` and `pkg/translations/`. The translation package defines the actual mapping between Kubernetes resources and the Envoy representation.\n\nPerformed benchmark/load tests and scalability analysis is included in the `hack/benchmark` folder, defining the infrastructure in `hack/benchmark/bench-infra` distributed across a monitoring cluster and workload cluster. The actual experiments performed are located alongside the [Fortio](https://fortio.org) results in `hack/benchmark/experiments/{scenarios,results}`.\nAll experiments base on the `hack/benchmark/experiments/generator` environment, which has been used to generate the scenarios.\n\nThe `bpf` directory includes early steps towards eBPF based forwarding of traffic to the node-local proxy and has not been covered in the thesis and is in a non functional state, but shipped for completeness.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxvzf%2Flightpath","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fxvzf%2Flightpath","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fxvzf%2Flightpath/lists"}