{"id":16812858,"url":"https://github.com/serathius/http-performance","last_synced_at":"2025-04-11T01:44:26.219Z","repository":{"id":79029179,"uuid":"102387778","full_name":"serathius/http-performance","owner":"serathius","description":"Minimal setup for testing http servers performance. ","archived":false,"fork":false,"pushed_at":"2018-09-30T18:18:47.000Z","size":4268,"stargazers_count":5,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-03-24T22:51:18.934Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/serathius.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-09-04T17:53:36.000Z","updated_at":"2019-03-25T06:23:03.000Z","dependencies_parsed_at":"2023-03-31T15:37:25.848Z","dependency_job_id":null,"html_url":"https://github.com/serathius/http-performance","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serathius%2Fhttp-performance","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serathius%2Fhttp-performance/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serathius%2Fhttp-performance/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/serathius%2Fhttp-performance/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/serathius","download_url":"https://codeload.github.com/serathius/http-performance/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248327754,"owners_count":21085257,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T10:23:43.003Z","updated_at":"2025-04-11T01:44:26.212Z","avatar_url":"https://github.com/serathius.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Benchmarking HTTP servers overhead\n\nCompared to other benchmarks this focuses less on big numbers like maximal requests/second and more about\nstability of latency. For microservice architecture it's more important what's your tail latency that can \nslow down your whole system because of multiple slow responses. Building for pretictible number of requests \nand dropping excess traffic can also improve stability of whole system, because having one faster microservice \ndoesn't mean that rest of system can handle traffic and could lead to taking system down.\n\n## Scenarios\n* Respond with content `Hello, World!`\n* Burn for 75% CPU in around 20 steps (e.g. with 100 requests per second in every request burn 100% CPU for 7.5 ms)\n* Sleep for 75% CPU in around 20 steps\n* Respond with 1MB content\n\n## Approach:\n* Focusing on P99 under stable requests per second\n* Restricting http server to 1 core\n* Required to response to 99% of requests\n* Access logs disabled\n\n## TODO\n* More reproducable tests\n* Test multicore scalability\n\n### Hello World response P99 latency in ms\n| Language | Framework | Server    | Status        | 100 r/s    | 1000 r/s  | 10000 r/s |\n|----------|-----------|-----------|--------------:|-----------:|----------:|----------:|\n| python3  | falcon    | meinheld  |               | 1.99       | 2.04      | 881.15    |     \n| golang   |           | fast/http |               | 2.23       | 2.07      | 20.35     |\n| haskell  |           | warp      |               | 2.12       | 2.08      | 23.31     |\n| python3  | japronto  |           | early preview | 2.09       | 2.14      | 12.38     |\n| rust     | hyper     |           |               | 2.05       | 2.18      | 16.35     |\n| python3  | vibora    |           | alpha         | 2.21       | 2.22      | 19.87     |\n| golang   |           | nett/http |               | 2.33       | 2.35      |           |\n| java     |           |           |               | 2.17       | 2.39      |           |\n| node     |           |           |               | 2.17       | 2.55      | 35.74     |\n| python3  | falcon    | bjoern    |               | 2.33       | 2.93      | 28.29     |\n| python3  | sanic     |           |               | 2.52       | 3.07      | 20.61     |\n| python3  | flask     | bjoern    |               | 3.11       | 5.11      |           |\n| python3  | aiohttp   |           |               | 2.88       | 5.85      |           |\n| python3  | django    | meinheld  |               | 2.79       | 6.19      |           |\n| pypy2    | falcon    | meinheld  |               | 2.17       | 7.13      |           |\n| python3  | tornado   |           |               | 4.09       | 8.60      |           |\n| python3  | flask     | gevent    |               | 4.19       | 10.61     |           |\n| python3  | flask     | eventlet  |               | 4.41       | 12.33     |           |\n| python3  | flask     | gthread   |               | 4.85       | 9.52      |           |\n| python3  | flask     | netius    |               | 4.19       | 12.69     |           |\n| pypy2    | flask     | sync      |               | 98.11      | 112.96    |           |\n| pypy3    | flask     | sync      |               | 95.81      | 121.66    |           |\n| python3  | flask     | sync      |               | 100.48     | 127.49    |           |\n| python2  | flask     | sync      |               | 100.93     | 129.47    |           |\n\n### Burn 75% CPU response P99 latency in ms\n| Language | Framework | Server    | Status        | 100 r/s    |\n|----------|-----------|-----------|--------------:|-----------:|\n| rust     | hyper     |           |               | 31.81      |\n| pypy2    | falcon    | meinheld  |               | 33.34      | \n| python3  | falcon    | meinheld  |               | 34.94      |\n| python3  | flask     | meinheld  |               | 36.22      |\n| python3  | django    | meinheld  |               | 37.79      |\n| golang   |           | fast/http |               | 38.14      |\n| python3  | aiohttp   |           |               | 39.26      |\n| python3  | falcon    | gthread   |               | 39.84      |\n| python3  | falcon    | tornado   |               | 40.83      |\n| python3  | tornado   |           |               | 42.53      |\n| golang   |           | net/http  |               | 43.52      |\n| python3  | japronto  |           | early preview | 45.09      |\n| python3  | vibora    |           | alpha         | 45.63      |\n| python3  | sanic     |           |               | 46.78      |\n| haskell  |           | warp      |               | 47.81      |\n| python3  | falcon    | eventlet  |               | 48.19      |\n| python3  | falcon    | netius    |               | 49.22      |\n| python3  | falcon    | gevent    |               | 49.28      |\n| python3  | falcon    | waitress  |               | 50.08      |\n| python3  | flask     | gevent    |               | 50.97      |\n| python3  | falcon    | bjoern    |               | 60.06      |\n| python3  | falcon    | sync      |               | 139.26     |\n\n### Sleep response P99 latency in ms\n| Language | Framework | Server    | Status        | 100 r/s    |\n|----------|-----------|-----------|--------------:|-----------:|\n| python3  | vibora    |           | alpha         | 9.45       |\n| python3  | japronto  |           | early preview | 9.58       |\n| python3  | sanic     |           |               | 9.87       |\n| golang   |           | fast/http |               | 9.88       |\n| golang   |           | net/http  |               | 9.95       |\n| haskell  |           | warp      |               | 10.55      |\n| python3  | aiohttp   |           |               | 12.18      |\n| python3  | falcon    | waitress  |               | 12.4       |\n| python3  | falcon    | gevent    |               | 12.71      |\n| python3  | tornado   |           |               | 14.15      |\n| python3  | flask     | gevent    |               | 14.92      |\n| rust     | hyper     |           |               | 36.32      |\n| pypy2    | falcon    | meinheld  |               | 38.69      | \n| python3  | falcon    | meinheld  |               | 38.91      |\n| python3  | flask     | meinheld  |               | 42.59      |\n| python3  | django    | meinheld  |               | 44.67      |\n| python3  | falcon    | gthread   |               | 52.06      |\n| python3  | falcon    | tornado   |               | 61.12      |\n| python3  | falcon    | bjoern    |               | 63.55      |\n| python3  | falcon    | eventlet  |               | 63.97      |\n| python3  | falcon    | netius    |               | 65.66      |\n| python3  | falcon    | sync      |               | 151.42     |\n\n### Return 1MB response P99 latency in ms\n| Language | Framework | Server    | Status        | 100 r/s    |\n|----------|-----------|-----------|--------------:|-----------:|\n| python3  | vibora    |           | alpha         | 3.32       |\n| python3  | japronto  |           | early preview | 3.54       |\n| python3  | falcon    | bjoern    |               | 3.61       |\n| python3  | falcon    | meinheld  |               | 3.65       |\n| python3  | django    | meinheld  |               | 3.88       |\n| python3  | sanic     |           |               | 3.88       |\n| python3  | flask     | meinheld  |               | 4.05       |\n| python3  | falcon    | eventlet  |               | 4.74       |\n| golang   |           | net/http  |               | 4.84       |\n| python3  | falcon    | netius    |               | 5.42       |\n| python3  | flask     | gevent    |               | 5.59       |\n| python3  | falcon    | tornado   |               | 6.47       |\n| python3  | falcon    | gevent    |               | 6.51       |\n| python3  | aiohttp   |           |               | 6.56       |\n| python3  | falcon    | gthread   |               | 7.34       |\n| golang   |           | fast/http |               | 11.28      |\n| python3  | tornado   |           |               | 18.61      |\n| pypy2    | falcon    | meinheld  |               | 22.78      | \n| python3  | falcon    | waitress  |               | 29.39      |\n| python3  | falcon    | sync      |               | 112.64     |\n\n\n### [Doc with all results](https://docs.google.com/spreadsheets/d/18TQOIZhg8CLunikgJyi3EGFDjYGLvQ5_RGj-DI2afT4/edit?usp=sharing)\n\n## Running\n\n### Start server \n```bash\nmake python3_sanic\n```\n\n### Benchmark 100r/s hello world scenario\n```bash\nmake benchmark_1_hello\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserathius%2Fhttp-performance","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fserathius%2Fhttp-performance","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fserathius%2Fhttp-performance/lists"}