{"id":13641585,"url":"https://github.com/flashbots/prio-load-balancer","last_synced_at":"2025-07-31T18:06:15.059Z","repository":{"id":103821611,"uuid":"566761318","full_name":"flashbots/prio-load-balancer","owner":"flashbots","description":"Priority JSON-RPC load balancer (with retries, good logging, and other goodies like SGX/SEV attestation support)","archived":false,"fork":false,"pushed_at":"2024-04-05T10:07:01.000Z","size":182,"stargazers_count":33,"open_issues_count":1,"forks_count":5,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-04-12T21:08:33.597Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flashbots.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-16T11:14:35.000Z","updated_at":"2025-03-24T23:21:20.000Z","dependencies_parsed_at":"2024-04-05T10:43:37.608Z","dependency_job_id":null,"html_url":"https://github.com/flashbots/prio-load-balancer","commit_stats":null,"previous_names":[],"tags_count":24,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flashbots%2Fprio-load-balancer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flashbots%2Fprio-load-balancer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flashbots%2Fprio-load-balancer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flashbots%2Fprio-load-balancer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flashbots","download_url":"https://codeload.github.com/flashbots/prio-load-balancer/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248631676,"owners_count":21136562,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-02T01:01:22.011Z","updated_at":"2025-04-12T21:08:51.344Z","avatar_url":"https://github.com/flashbots.png","language":"Go","funding_links":[],"categories":["Network"],"sub_categories":["Library OSes and SDKs"],"readme":"# Transparent JSON-RPC proxy and load balancer\n\n[![Goreport status](https://goreportcard.com/badge/github.com/flashbots/prio-load-balancer)](https://goreportcard.com/report/github.com/flashbots/prio-load-balancer)\n[![Test status](https://github.com/flashbots/prio-load-balancer/workflows/Checks/badge.svg)](https://github.com/flashbots/prio-load-balancer/actions?query=workflow%3A%22Checks%22)\n[![Docker hub](https://badgen.net/docker/size/flashbots/prio-load-balancer?icon=docker\u0026label=image)](https://hub.docker.com/r/flashbots/prio-load-balancer/tags)\n\n**With priority queues, retries, good logging and metrics, and even SGX/SEV attestation support.**\n\nQueues:\n\n1. low-prio\n2. high-prio\n3. fast-track\n\nQueueing:\n\n- All high-prio requests will be proxied before any of the low-prio queue\n- [N](https://github.com/flashbots/prio-load-balancer/blob/main/server/consts.go#L20) fast-tracked requests get processed for every 1 high-prio request\n\nFurther notes:\n\n- A _node_ represents one JSON-RPC endpoint (i.e. geth instance)\n- Each node spins up N workers, which proxy requests concurrently to the execution endpoint\n- You can add/remove nodes through a JSON API without restarting the server\n- Each node starts the default number of workers, but you can also specify a custom number of workers by adding `?_workers=` to the node URL\n- It's possible to tweak [a few knobs](/server/consts.go)\n\u003c!-- - The load balancer exposes a HTTP API for managing nodes, and uses Redis as a source of truth for configured nodes (i.e. the cli node config only sets the initial state in redis, but a restart won't override the node setup created through the HTTP API. --\u003e\n\n---\n\n### Application structure and request flow\n\n![App structure and request flow](https://user-images.githubusercontent.com/116939/202170917-bcd98c98-f40e-4025-8084-06adec27ff96.png)\n\n### Example logs\n\nAt the end of a request:\n\n```json\n{\n    \"level\": \"info\",\n    \"ts\": 1685122704.4079978,\n    \"caller\": \"server/webserver.go:154\",\n    \"msg\": \"Request completed\",\n    \"service\": \"validation-queue\",\n    \"durationMs\": 144,\n    \"requestIsHighPrio\": true,\n    \"requestIsFastTrack\": false,\n    \"payloadSize\": 209394,\n    \"statusCode\": 200,\n    \"nodeURI\": \"http://validation-1.internal:8545\",\n    \"requestTries\": 1,\n    \"queueItems\": 11,\n    \"queueItemsFastTrack\": 0,\n    \"queueItemsHighPrio\": 7,\n    \"queueItemsLowPrio\": 4\n}\n```\n\nFull request cycle:\n\n```json\n// request getting added to the queue\n{\"level\":\"info\",\"ts\":1685126514.8569882,\"caller\":\"server/webserver.go:112\",\"msg\":\"Request added to queue. prioQueue size:\",\"service\":\"validation-queue\",\"requestIsHighPrio\":true,\"requestIsFastTrack\":true,\"fastTrack\":1,\"highPrio\":0,\"lowPrio\":0}\n\n// completed request\n{\"level\":\"info\",\"ts\":1685126514.9174724,\"caller\":\"server/webserver.go:154\",\"msg\":\"Request completed\",\"service\":\"validation-queue\",\"durationMs\":78,\"requestIsHighPrio\":true,\"requestIsFastTrack\":true,\"payloadSize\":121291,\"statusCode\":200,\"nodeURI\":\"http://validation-2.internal:8545\",\"requestTries\":1,\"queueItems\":0,\"queueItemsFastTrack\":0,\"queueItemsHighPrio\":0,\"queueItemsLowPrio\":0}\n\n// http server logs\n{\"level\":\"info\",\"ts\":1685126514.9175296,\"caller\":\"server/http_logger.go:54\",\"msg\":\"http: POST /sim 200\",\"service\":\"validation-queue\",\"status\":200,\"method\":\"POST\",\"path\":\"/sim\",\"duration\":0.078877451}\n```\n\n---\n\n## Getting started\n\nDocker images are available at https://hub.docker.com/r/flashbots/prio-load-balancer\n\n#### Run the program\n\n```bash\n# Run with a mock execution backend and debug output\ngo run . -mock-node           # text logging\ngo run . -mock-node -log-prod # json logging\n\n# low-prio queue request\ncurl -d '{\"jsonrpc\":\"2.0\",\"method\":\"eth_callBundle\",\"params\":[],\"id\":1}' localhost:8080\n\n# high-prio queue request\ncurl -H 'X-High-Priority: true' -d '{\"jsonrpc\":\"2.0\",\"method\":\"eth_callBundle\",\"params\":[],\"id\":1}' localhost:8080\n\n# fast-track queue request\ncurl -H 'X-Fast-Track: true' -d '{\"jsonrpc\":\"2.0\",\"method\":\"eth_callBundle\",\"params\":[],\"id\":1}' localhost:8080\n\n# adding a custom request ID\ncurl -H 'X-Request-ID: yourLogID' -d '{\"jsonrpc\":\"2.0\",\"method\":\"eth_callBundle\",\"params\":[],\"id\":1}' localhost:8080\n\n# Get execution nodes\ncurl localhost:8080/nodes\n\n# Add a execution node\ncurl -d '{\"uri\":\"http://foo\"}' localhost:8080/nodes\n\n# Add a execution node with custom number of workers\ncurl -d '{\"uri\":\"http://foo?_workers=8\"}' localhost:8080/nodes\n\n# Remove a execution node\ncurl -X DELETE -d '{\"uri\":\"http://foo\"}' localhost:8080/nodes\ncurl -X DELETE -d '{\"uri\":\"http://localhost:8095\"}' localhost:8080/nodes\n```\n\nNote: there's a bunch of constants that can be configured with env vars in [server/consts.go](server/consts.go).\n\n#### Node selection\n\n* Redis is used as source of truth for which execution nodes to use.\n* If you restart with a different set of configured nodes (i.e. in env vars), the previous nodes will still be in Redis and still be used by the load balancer.\n* See the commands in the readme above on how to get the nodes it uses, and how to add/remove nodes.\n\n#### Test, lint, build\n\n```bash\n# lint \u0026 staticcheck (staticcheck.io)\nmake lint\n\n# run tests\nmake test\n\n# test coverage\nmake cover\nmake cover-html\n\n# build\nmake build\n```\n\n#### Node TEE attestation via TLS\n```\n# build prio-load-balancer with SGX and SEV support\nmake build-tee\n```\n\n\u003e **IMPORTANT:** SGX and SEV attestation support requires additional dependencies. See [Dockerfile.tee](Dockerfile.tee) for details.\n\n#### SEV Node aTLS attestation\n\n```\n# base64 encode the VM measurements\n\nMEASUREMENTS=$(cat \u003c\u003c EOF | gzip | basenc --base64url -w0\n{\n  \"1\": {\n    \"expected\": \"3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969\",\n    \"warnOnly\": true\n  },\n  \"2\": {\n    \"expected\": \"3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969\",\n    \"warnOnly\": true\n  },\n  \"3\": {\n    \"expected\": \"3d458cfe55cc03ea1f443f1562beec8df51c75e14a9fcf9a7234a13f198e7969\",\n    \"warnOnly\": true\n  },\n  \"4\": {\n    \"expected\": \"82736cdd6b4f3c718bf969b545eaaa6eb3f1e6d229ad9712e6a4ddf431418ab7\",\n    \"warnOnly\": false\n  },\n  \"5\": {\n    \"expected\": \"54c04bcd7cf8adadafee915bf325f92d958050c14e086c1e180258113d376c1a\",\n    \"warnOnly\": true\n  },\n  \"6\": {\n    \"expected\": \"9319868ef4dad6a79117f14b9ac1870ccf5f9d178b39a3fd84e6230fa93a7993\",\n    \"warnOnly\": true\n  },\n  \"7\": {\n    \"expected\": \"32fe42b385b47cb22c906b8a7e4f134e9f2270818f90e94072d1101ef72f1c00\",\n    \"warnOnly\": true\n  },\n  \"8\": {\n    \"expected\": \"0000000000000000000000000000000000000000000000000000000000000000\",\n    \"warnOnly\": false\n  },\n  \"9\": {\n    \"expected\": \"0000000000000000000000000000000000000000000000000000000000000000\",\n    \"warnOnly\": false\n  },\n  \"11\": {\n    \"expected\": \"0000000000000000000000000000000000000000000000000000000000000000\",\n    \"warnOnly\": false\n  },\n  \"12\": {\n    \"expected\": \"f1a142c53586e7e2223ec74e5f4d1a4942956b1fd9ac78fafcdf85117aa345da\",\n    \"warnOnly\": false\n  },\n  \"13\": {\n    \"expected\": \"0000000000000000000000000000000000000000000000000000000000000000\",\n    \"warnOnly\": false\n  },\n  \"14\": {\n    \"expected\": \"e3991b7ddd47be7e92726a832d6874c5349b52b789fa0db8b558c69fea29574e\",\n    \"warnOnly\": true\n  },\n  \"15\": {\n    \"expected\": \"0000000000000000000000000000000000000000000000000000000000000000\",\n    \"warnOnly\": false\n  }\n}\nEOF\n)\n```\n\n```\n# Add the SEV execution node\ncurl -d \"{\\\"uri\\\":\\\"https://SEV_${MEASUREMENTS}@foo\\\"}\" localhost:8080/nodes\n```\n\nExecution nodes running within SEV and providing attestation consumables via constellations aTLS implementation are supported. The aTLS certificate of the execution node is automatically attested with the VM measurements which are submitted as part of the user part of the **node URI** (`SEV_\u003cgzipped, base64url encoded measurements\u003e`). You can read more about the attestation measurements in the [constellation docs](https://docs.edgeless.systems/constellation/architecture/attestation#runtime-measurements)\n\n#### SGX Node RA-TLS attestation\n```\n# Add an SGX execution node\ncurl -d '{\"uri\":\"https://SGX_\u003cMRENCLAVE\u003e@foo\"}' localhost:8080/nodes\n```\n\nExecution nodes running within SGX and providing attestation consumables via RA-TLS are supported. The RA-TLS certificate of the execution node is automatically attested with the `MRENCLAVE` which is submitted as part of the user part of **node URI** (`SGX_\u003cMRENCLAVE\u003e`).\n\n---\n\n## Queue Benchmarks\n\n```\ngoarch: amd64\npkg: github.com/flashbots/prio-load-balancer/server\ncpu: Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz\n\n1 worker, 10k tasks:\nBenchmarkPrioQueue-12    \t    \t    2338\t    492219 ns/op\t  298109 B/op\t      34 allocs/op\n\n5 workers, 10k tasks:\nBenchmarkPrioQueueMultiReader-12    \t    2690\t    596315 ns/op\t  292507 B/op\t      50 allocs/op\n\n5 workers, 100k tasks:\nBenchmarkPrioQueueMultiReader-12    \t     261\t   4637403 ns/op\t 4245243 B/op\t      66 allocs/op\n```\n\n---\n\n## Todo\n\nPossibly\n\n* Configurable redis prefix, to allow multiple sim-lbs per redis instance\n* Execution-node health checks (currently not implemented)\n\n---\n\n## Maintainers\n\n- [@metachris](https://twitter.com/metachris)\n\n---\n\n## License\n\nThe code in this project is free software under the [MIT License](LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflashbots%2Fprio-load-balancer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflashbots%2Fprio-load-balancer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflashbots%2Fprio-load-balancer/lists"}