{"id":30598142,"url":"https://github.com/grafbase/graphql-federation-benchmarks","last_synced_at":"2025-08-29T22:11:05.922Z","repository":{"id":312085872,"uuid":"951358310","full_name":"grafbase/graphql-federation-benchmarks","owner":"grafbase","description":"GraphQL Federation benchmarks","archived":false,"fork":false,"pushed_at":"2025-08-28T13:05:15.000Z","size":2748,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":5,"default_branch":"main","last_synced_at":"2025-08-28T19:33:18.364Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/grafbase.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-03-19T14:54:55.000Z","updated_at":"2025-08-28T13:05:18.000Z","dependencies_parsed_at":"2025-08-28T19:38:42.028Z","dependency_job_id":"56c50015-7ab6-4f76-bbb9-8aaa087d55a4","html_url":"https://github.com/grafbase/graphql-federation-benchmarks","commit_stats":null,"previous_names":["grafbase/graphql-federation-benchmarks"],"tags_count":null,"template":false,"template_full_name":null,"purl":"pkg:github/grafbase/graphql-federation-benchmarks","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grafbase%2Fgraphql-federation-benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grafbase%2Fgraphql-federation-benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grafbase%2Fgraphql-federation-benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grafbase%2Fgraphql-federation-benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/grafbase","download_url":"https://codeload.github.com/grafbase/graphql-federation-benchmarks/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/grafbase%2Fgraphql-federation-benchmarks/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":272772597,"owners_count":24990513,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-29T02:00:10.610Z","response_time":87,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-08-29T22:11:04.022Z","updated_at":"2025-08-29T22:11:05.913Z","avatar_url":"https://github.com/grafbase.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# GraphQL Federation Gateway Benchmarks\n\nThis repo contains a list of complex benchmark cases to measure the performance of GraphQL Federation gateways.\n\n## Benchmarks\n\n### big-response\n\nA single subgraph that returns a 4MiB response.\nThe goal is to measure how much overhead the gateway adds parsing and validating and incoming response.\n\nK6 runs for 30s with a single VU, executing requests sequentially.\n\n### many-plans\n\nThe supergraph consists of 7 subgraphs, served by a single server, are exposing very similar schemas. The query is constructed to have many different possible plans. The goal is to measure how efficient the query planer is at finding a good solution and how much time it takes.\nA good solution is one that minimizes the number of subgraph request.\n\nK6 runs for 30s with a single VU, executing requests sequentially.\n\nFor Apollo Router, we had to fork the project to disable the plan caching as the whole point of this benchmark is to measure the planning performance. The code used is there: https://github.com/Finistere/router/tree/qp-disable-cache\n\n## Protocol\n\nThe gateway and the subgraphs are run through docker with `--network host` to avoid any overhead. The load testing and latency measure is done by K6.\nThe subgraphs keep track of the number of incoming GraphQL requests, excluding health checks. K6 retrieves any statistics computed by the subgraph at the end and propagates it in its summary. During the tests, we track the resource usage (CPU \u0026 MEM) of the gateway through `docker stats`. We only keep the measurements when K6 was running.\n\nA report is provided at the end.\n\n## Latest report\n\n```\n# System Information\n\nDate: 2025-08-28\nCPU: AMD Ryzen 9 7950X3D 16-Core Processor\nMemory: 93.4 GiB\nCPU Boost: Disabled\nGit Commit: d729680366dc2050ed77c9d47381989fa7771542\nLinux Version: 6.16.1\nDocker Version: 28.3.3\n\n# Benchmarks\n\n## big-response\n\n### Requests\n\n| Gateway          | Requests | Failures | Subgraph requests (total) |\n| :--------------- | -------: | -------: | ------------------------: |\n| Apollo Router    |      369 |        0 |                   1 (369) |\n| Cosmo Router     |      465 |        0 |                   1 (465) |\n| Grafbase Gateway |      790 |        0 |                   1 (790) |\n| Hive Gateway     |      286 |        0 |                   1 (286) |\n\n### Latencies (ms)\n\n| Gateway          |     Min |     Med |     P90 |     P95 |     P99 |     Max |\n| :--------------- | ------: | ------: | ------: | ------: | ------: | ------: |\n| Apollo Router    |    73.4 |    80.3 |    85.8 |    87.9 |    99.3 |   117.4 |\n| Cosmo Router     |    52.4 |    63.5 |    70.7 |    72.7 |    77.4 |    89.5 |\n| Grafbase Gateway |    34.8 |    37.1 |    40.4 |    41.4 |    43.6 |    56.3 |\n| Hive Gateway     |    98.9 |   103.2 |   107.8 |   110.3 |   132.7 |   199.5 |\n\n### Resources\n\n| Gateway          |  CPU avg |  CPU max |   MEM avg |   MEM max |\n| :--------------- | -------: | -------: | --------: | --------: |\n| Apollo Router    |      71% |      73% |   484 MiB |   563 MiB |\n| Cosmo Router     |     148% |     163% |    84 MiB |   118 MiB |\n| Grafbase Gateway |      38% |      40% |   149 MiB |   169 MiB |\n| Hive Gateway     |      98% |     102% |   434 MiB |   521 MiB |\n\n## many-plans\n\n### Requests\n\n| Gateway                     | Requests | Failures | Subgraph requests (total) |\n| :-------------------------- | -------: | -------: | ------------------------: |\n| Apollo Router (no cache)    |        9 |        0 |                203 (1827) |\n| Cosmo Router (no cache)     |       74 |        0 |               156 (11615) |\n| Grafbase Gateway (no cache) |     1430 |        0 |               83 (118690) |\n\n### Latencies (ms)\n\n| Gateway                     |     Min |     Med |     P90 |     P95 |     P99 |     Max |\n| :-------------------------- | ------: | ------: | ------: | ------: | ------: | ------: |\n| Apollo Router (no cache)    |  3353.0 |  3358.2 |  3382.5 |  3404.7 |  3422.5 |  3426.9 |\n| Cosmo Router (no cache)     |   400.0 |   410.1 |   417.9 |   419.6 |   422.4 |   424.5 |\n| Grafbase Gateway (no cache) |    18.7 |    21.0 |    22.1 |    22.4 |    23.1 |    57.6 |\n\n### Resources\n\n| Gateway                     |  CPU avg |  CPU max |   MEM avg |   MEM max |\n| :-------------------------- | -------: | -------: | --------: | --------: |\n| Apollo Router (no cache)    |     100% |     101% |  1320 MiB |  2253 MiB |\n| Cosmo Router (no cache)     |     155% |     170% |    74 MiB |    80 MiB |\n| Grafbase Gateway (no cache) |     107% |     108% |   174 MiB |   210 MiB |\n```\n\n## Running the benchmarks\n\n### Requirements\n\n- Docker and Docker Compose\n- Rust toolchain (rustup)\n- K6 load testing tool\n\n### CPU boost\n\nCPU boost can skew the results as frequencis will be higher when few CPU cores are used. You can disable it on Linux with:\n\n```sh\necho \"0\" | sudo tee /sys/devices/system/cpu/cpufreq/boost\n```\n\n### Commands\n\n```bash\n# Run all benchmarks with all gateways\ncargo run --release -- run\n\n# Run specific benchmark with specific gateway\ncargo run --release -- run --benchmark many-possible-query-plans --gateway grafbase\n\n# List available benchmarks and gateways\ncargo run --release -- list\n```\n\n### Troubleshoot\n\nIf you encounter errors, you might need to clean your running containers:\n\n```sh\n docker stop $(docker ps -a -q) -t 2 \u0026\u0026 docker rm $(docker ps -a -q)\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrafbase%2Fgraphql-federation-benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgrafbase%2Fgraphql-federation-benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgrafbase%2Fgraphql-federation-benchmarks/lists"}