{"id":13300716,"url":"https://github.com/Sanshain/web_benchmarks","last_synced_at":"2025-03-10T11:33:01.889Z","repository":{"id":44790079,"uuid":"444487117","full_name":"Sanshain/web_benchmarks","owner":"Sanshain","description":"Performance comparing of backend frameworks","archived":false,"fork":false,"pushed_at":"2022-01-31T16:43:27.000Z","size":14554,"stargazers_count":0,"open_issues_count":4,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2023-03-10T00:13:20.176Z","etag":null,"topics":["apache-benchmark","benchmarks","django","expressjs","fastapi","fastify","gunicorn","loadtest","vibe-d"],"latest_commit_sha":null,"homepage":"","language":"HTML","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Sanshain.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-01-04T16:22:57.000Z","updated_at":"2022-01-30T14:52:59.000Z","dependencies_parsed_at":"2022-07-25T23:17:32.286Z","dependency_job_id":null,"html_url":"https://github.com/Sanshain/web_benchmarks","commit_stats":null,"previous_names":[],"tags_count":null,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sanshain%2Fweb_benchmarks","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sanshain%2Fweb_benchmarks/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sanshain%2Fweb_benchmarks/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Sanshain%2Fweb_benchmarks/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Sanshain","download_url":"https://codeload.github.com/Sanshain/web_benchmarks/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242843484,"owners_count":20194374,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-benchmark","benchmarks","django","expressjs","fastapi","fastify","gunicorn","loadtest","vibe-d"],"created_at":"2024-07-29T17:43:03.881Z","updated_at":"2025-03-10T11:33:01.883Z","avatar_url":"https://github.com/Sanshain.png","language":"HTML","readme":"# Benchmarks: \n\nPerformance comparison for **django**, **fastapi**, **express**, **fastify** vs **vibe-d**:\n\n## Quick start:\n\n#### First step:\n\nTo measure performance using [loadtest](https://www.npmjs.com/package/loadtest). So install it: \n\n```\nnpm i -g loadtest\n```\n\n#### Second step:\n\n```\ndocker-compose up\n```\n\n### Configuration:\n\nThe following ports are configured by default:\n\n- Django: *8000*\n- Fastapi: *8008*\n- Express: *9000*\n- Vibe-d: *7000*\n- Fastify: *9001*\n\n****\n\n\n## Results for linux: \n\nTests was running on docker-machine with 4 virtual cores and 1256 MB memory available\n\n### loadtest:\n\n```shell\nloadtest -n 20000 -c 32 http://127.0.0.1:8000\nloadtest -n 20000 -c 32 -k http://127.0.0.1:8000\n```\n\nInstead of `8000` would be either port of testing framework\n\n#### Tests w/o keepalive:\n\nMetric |Requests per sec |CPU usage (motionless) |CPU usage (max) |Memory usage |pids |errors |\n:--------------------- |:---------------------:|:---------------------:|:---------------------:|:---------------------:|:---------------------:|:--------------------- |\n Django | 1514 requests | 0.05% | 390% | 220Mb | 9 | 0 |\n Fastapi (sync) | 1216 requests | 2% | 380% | 180Mb | 9-106 | 1 |\n Fastapi (async) | 1622 requests | 2% | 353% | 175Mb | 9 | 1 |\n Django (meinheld) | 1629 requests | 0.15% | 335% | 335Mb | 9 | 0 |\n Express (w c) | 1586 requests | 0% | 260% | 105Mb | 35 | 0 ** |\n Fastify * | 1750 requests | 0% | 105% | 75 Mb | 23 | 0 |\n Fastify (w c) | | | | | | |\n vibe-d (dmd) **** | 2342 requests | 0% | 105% | 33Mb | 6 | 0 |\n\n\n#### Tests with keepalive:\n\nMetric |Requests per sec |CPU usage (motionless) |CPU usage (max) |Memory usage |pids |errors |\n:--------------------- |:---------------------:|:---------------------:|:---------------------:|:---------------------:|:---------------------:|:--------------------- |\n Django | 1514 requests | 0.05% | 390% | 220Mb | 9 | 0 |\nFastapi (sync) | 2114 requests | 2% | 405% | 175Mb | 61 | 0 |\n Fastapi(async) | 3569 requests | 2% | 380% | 180Mb | 9 | 0 |\n Django (meinheld) | 2884 requests | 0.15% | 400% | 335Mb | 9 | 0 |\nExpress (w c)*** | 4500 requests | 0% | 195% | 60Mb | 9 | 0 |\nFastify (w/o c) | 3500 requests | 0% | 105% | 75 Mb | 35 | 0 |\n Fastify (w c) | 4500 requests | 0% | 212% | 477 Mb | 47 | 0 |\n vibe-d (dmd) **** | 4150 requests | 0% | 105% | 33Mb | 6 | 0 |\n\n\n**Footnotes:**\n* w/o clusterisation (manual clusterisation did not have any effect (expect the hugest memory allocation) because of (I suppose) clusterisation integrated in framework)\n* w/o clusterisation was at less one error\n* Results with pm2 is not included, because its showed worse results on *loadtest* than on manual clusterisation tuning (1500 and 3800 r/sec suitably and 55 processes)\n* vibe-d was launched on only one core (w/o clusterisation)\n\n### wrk:\n\n```shell\n# start wrk service:\ndocker exec -it wrk_test /bin/sh\n\n# get target ip inside internal network (get it via `docker network ls` command):\ndocker network inspect pentest_default\n\n# start test:\nwrk -t4 -c100 -d3s \u003curl\u003e # for example http://172.21.0.3:3000\n```\n\nMetric |Requests per sec |CPU usage (max) |Memory usage |\n:---------------------:|:---------------------:|:---------------------:|:---------------------:|\n Django | 2200 requests | 390% | 220Mb |\nFastapi (sync) | 3500 requests | 380% | 176 Mb |\n Fastapi(async) | 7000 requests | 380% | 100Mb |\n Django (m) | 4100 requests | 390% | 235Mb |\n Express (w c)* | 35000 requests | 425% | 120Mb |\nFastify (w/o c) | 10500 requests | 425% | 75 Mb |\n Fastify (w c) | 30000 requests | 425% | 420 Mb |\n vibe-d (one core) | 16000 requests | 120% | 6 Mb |\n vibe-d (4 core) ** | 31500 requests | 312% | 8Mb |\n vibe-d (4 core) *** | 36500 requests | 338% | 34Mb |\n\n* Results with pm2 is not included, because its showed worse results on loadtest than on manual clusterisation tuning (9000 r/sec)\n* via `setupWorkerThreads \u0026\u0026 runWorkerTaskDist`\n* via `fork` from `core.sys.posix.unistd`\n\n\n### Windows \n\n#### Loadtest:\n\nFX-8350 \u0026\u0026 8Gb memory is available (x64)\n\n#### Prepareing: \n\n- Apply `npm i` inside *Express* and *Fastify* directories\n- Apply `pipenv shell` inside *Django* and *Fastapi* directories\n- I need *ldc2* compliler for vibe-d compilation\n\nInit scripts:\n- **Express** *:3000*: `node index.js`\n- **Fastify** *:3000*: `npm run begin`\n- **Django** *:8000*: `waitress-serve --listen=*:8000 --threads=8 project.wsgi:application` from *project* catalog\n- **Fastapi** *:8000*: `uvicorn-run.bat`\n- **VibeD** *:8088*: `run.bat`\n- **asp net** : *depends on your iis settings*\n\n#### Results w/o keepalive:\n\nMetric | Django (waitress) | Fastapi (just uvicorn) | Express | ASP NET | Fastify | IIS | vibeD |\n:------------- |:-----------------:|:----------------------:| :-------------:|:-------------:| :-------------:|:-------------:|:-------------:|\nRequest per sec | 1077 requests | 568 (642) requests | 2300 requests | 2205 requests | 2300 requests | 2416 requests | 2426 requests |\nlongest request | 58 ms | 113ms | 43 ms | 42 ms | 52 ms | 42 ms | 41 ms |\n\n\n#### Results with keepalive:\n\nMetric | Django (waitress) | Fastapi (just uvicorn) | Express | ASP NET ** | Fastify * | IIS | vibeD |\n:------------- |:-----------------:|:----------------------:| :-------------:|:-------------:| :-------------:|:-------------:|:-------------:|\nRequest per sec | -//- | -//- | 4132 requests | 4572 requests | 5350 requests | 5000 requests | 5300 requests |\nlongest request | | | 45 ms | 53 ms | 45 ms | 41 ms | 41 ms |\n\n\n* With clusterisations and w/o is equals results (Node.js clusterisation is handling via [clusters](https://www.npmjs.com/package/cluster))\n* ASP NET consumes about 150 MB of RAM in dev mode. I didn't check it after deployment, but it's unlikely anymore\n\n---- \n\n## Another comparisions:\n\n- from [the-benchmarker](https://github.com/the-benchmarker/web-frameworks)\n- from [techempower](https://www.techempower.com/benchmarks/)\n- look up also apache banchmarks tests [here](https://github.com/Sanshain/web_benchmarks/tree/master/.apache%20banchmarks)\n\n## Notes:\n\n- **fastify_cluster** container was based on [this one](https://github.com/joseluisq/fastify-cluster-example). Thanks for [joseluisq](https://github.com/joseluisq)\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSanshain%2Fweb_benchmarks","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FSanshain%2Fweb_benchmarks","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FSanshain%2Fweb_benchmarks/lists"}