https://github.com/retocode/envoy-request-buffer
A PoC of a Proxy-WASM based implementation to hold requests until upstream targets are scaled + healthy
https://github.com/retocode/envoy-request-buffer
envoy proxy-wasm proxy-wasm-go-sdk scaletozero wasm wasm-go
Last synced: 11 months ago
JSON representation
A PoC of a Proxy-WASM based implementation to hold requests until upstream targets are scaled + healthy
- Host: GitHub
- URL: https://github.com/retocode/envoy-request-buffer
- Owner: ReToCode
- License: apache-2.0
- Created: 2024-02-09T06:59:43.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-02-20T14:18:16.000Z (over 2 years ago)
- Last Synced: 2025-03-25T01:43:29.811Z (over 1 year ago)
- Topics: envoy, proxy-wasm, proxy-wasm-go-sdk, scaletozero, wasm, wasm-go
- Language: Go
- Homepage:
- Size: 106 KB
- Stars: 4
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# PoC: Envoy based request buffering with WASM
This repo contains a simple PoC for request buffering in Envoy using Proxy-WASM.
Note: The control-plane does not really check if the backend is scaled, nor is the control-plane called to actually scale the upstream target.
## Architecture
### In Envoy
See [envoy](./ENVOY.md#architecture)
### In Kubernetes
In the Kubernetes example, this PoC is using [istio with Wasm Plugin](https://istio.io/latest/docs/reference/config/proxy_extensions/wasm-plugin/).
The networking is configured using [gateway-api](https://gateway-api.sigs.k8s.io/) resources, while the WASM plugin in Envoy takes care of buffering requests if necessary.
**References**
* [Proxy-WASM spec](https://github.com/proxy-wasm/spec/blob/master/docs/WebAssembly-in-Envoy.md)
* [Proxy-WASM golang SDK docs](https://github.com/tetratelabs/proxy-wasm-go-sdk/blob/main/doc/OVERVIEW.md)
## Variants
* [In Kubernetes](./KUBERNETES.md)
* [Local envoy in docker-compose](./ENVOY.md)
## Where to find what
```bash
tree -d
.
├── kubernetes # kubernetes variant
│ ├── control-plane # control-plane that reads from K8s API and scales via K8s API
│ └── yaml # manifests used on K8s
├── local-envoy # variant with local envoy in docker-compose
│ ├── static-control-plane # a static control-plane where you set "scaled-to-zero" manually
├── upstream # dummy upstream backend with http and gRPC
└── wasm-request-buffer # wasm code
├── filter # code running on every http-request
├── service # shared code running on a ticker to talk to control-plane
└── shared # shared stuff
```
## Kubernetes Demo
For more info about the demo see the scripts in [demo](./kubernetes/demo)
## Things to look at next
- [ ] Testing performance and overhead
- [ ] Testing behaviour, CPU+memory consumption with a lot of hold requests
- [ ] Test data architecture (how many request context can be held, how does envoy scale the HTTP contexts)?
- [x] Make it work with Envoy Gateway-API integration
- [x] Make it work on K8s, get the scaled-to-zero status from K8s
- [x] Poke the control-plane to actually scale on requests when scaled to zero
## Learnings
* For now the proxy-wasm-go-sdk uses `tinygo`, which has some downsides
* Not all standard library packages are supported
* No serialization packages work
* Limited working dependencies available
* No gRPC support for now --> https://github.com/tetratelabs/proxy-wasm-go-sdk/issues/14
* The API of the SDK is a bit peculiar
* Error reporting is quite limited (e.g. when a httpContextID is no longer available, a generic `bad argument` error is returned)
* Another option might be to consider using Rust for the module, [as Kuadrant dit](https://github.com/Kuadrant/wasm-shim)
* Istio does not yet support `kind: WASMModule` for singletons (WASMServices) --> https://github.com/istio/istio/issues/49451