{"id":25670112,"url":"https://github.com/oya163/api-benchmark","last_synced_at":"2026-06-11T01:31:29.014Z","repository":{"id":87687545,"uuid":"328278470","full_name":"oya163/api-benchmark","owner":"oya163","description":"Multiple python API framework benchmarking based on image conversion task","archived":false,"fork":false,"pushed_at":"2021-01-20T03:36:44.000Z","size":120,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-02-24T11:37:09.537Z","etag":null,"topics":["asyncio","benchmarking","fastapi","flask"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/oya163.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-01-10T01:12:37.000Z","updated_at":"2024-12-05T06:55:27.000Z","dependencies_parsed_at":null,"dependency_job_id":"175ca5fc-8386-4e86-88ef-54a94ec43bba","html_url":"https://github.com/oya163/api-benchmark","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/oya163/api-benchmark","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oya163%2Fapi-benchmark","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oya163%2Fapi-benchmark/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oya163%2Fapi-benchmark/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oya163%2Fapi-benchmark/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/oya163","download_url":"https://codeload.github.com/oya163/api-benchmark/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/oya163%2Fapi-benchmark/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":34178819,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-26T15:22:16.424Z","status":"online","status_checked_at":"2026-06-10T02:00:07.152Z","response_time":89,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","benchmarking","fastapi","flask"],"created_at":"2025-02-24T11:29:35.823Z","updated_at":"2026-06-11T01:31:29.009Z","avatar_url":"https://github.com/oya163.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# API Benchmark\n\nThis project is about benchmarking three different Python API frameworks based on simple image processing task. The basic concept is the APIs consumes either image file in the form of base64 encoding or multipart-data or just a image url, and returns the corresponding pure black and white image to the client.\n\n## Frameworks:\n- [x] [FlaskAPI](https://flask-restful.readthedocs.io/en/latest/)\n- [x] [FastAPI](https://fastapi.tiangolo.com/)\n- [x] [aioHTTP](https://docs.aiohttp.org/en/stable/)\n\n\n## How to run\n\n### Flask API\n\n#### Server side\n    uvicorn app:app --reload       # --reload to automatically reflect code changes\n\n    uvicorn app:app --reload --log-level critical  # to suppress the console log\n\n\n#### Client side\n    python flask_client.py\n    pytest\n\n### Jinja\nOnce you run the server, please goto [http://127.0.0.1:5000/](http://127.0.0.1:5000/)\nin order to check the results visually. This API server retrieves random dog images\nusing [Dog API](https://dog.ceo/dog-api/) and converts it into pure black and white.\n\n### Fast API\n\n#### Server side\n    uvicorn app:app --reload       # --reload to automatically reflect code changes\n\n    uvicorn app:app --reload --log-level critical  # to suppress the console log\n\n\n#### Client side\n    python fast_client.py\n    pytest\n\n\n### aioHTTP\n\n#### Server side\n    python -m app.py -H localhost -P 8000 package.module:init_func\n\n\n#### Client side\n    python client_aiohttp.py\n    pytest\n\n\n## Benchmarking\n1. [locust](https://locust.io/)\n   \n\n        locust -f locustfile.py --headless --host http://127.0.0.1:5000 -u 2000 -r 50 -t 30s --csv reports/imageurl ImageUrlUser\n        locust -f locustfile.py --headless --host http://127.0.0.1:5000 -u 2000 -r 50 -t 30s --csv reports/base64 Base64User\n        locust -f locustfile.py --headless --host http://127.0.0.1:5000 -u 2000 -r 50 -t 30s --csv reports/multipart MultipartUser\n\nor, just run `./benchmark.sh` for 5-trial run\n\n2. [wrk](https://github.com/wg/wrk)\n\n*Note: This currently works for `imageurl` endpoint only*\n\n- Flask API\n\n\n        wrk http://127.0.0.1:5000/magic/imageurl -t12 -c400 -d30s -s lua_scripts/post.lua\n\n- Fast API\n\n\n        wrk http://127.0.0.1:8000/magic/imageurl -t12 -c400 -d30s -s lua_scripts/post.lua\n\n- aioHTTP\n\n\n        wrk http://0.0.0.0:8080/magic/imageurl -t12 -c400 -d30s -s lua_scripts/post_aiohttp.lua\n\n## Experiments\n\nInitially, aLL API endpoints were tested for performance using **wrk** HTTP benchmarking tool. \n**wrk** tool is based on C and requires in-depth knowledge of Lua to pass the base64 encoded images as multipart/data. I faced great difficulty in passing base64 encoding image or multipart/form-data as a payload through **wrk**. Hence, I switched to **locust**, which is a python based HTTP benchmarking tool. It made the task very much easier since it utilizes the `requests` library. However, it should be noted that **wrk** is extremely fast compared to locust.\n\nParameters for load testing on *locust**:\n\n- Number of users = 2000\n\n- User spawning rate = 50/second\n\n- Duration of test = 30s\n\n\n## Results\n\n\n![Latency](/assets/latency.png)\n\n\n![Request/Second](/assets/request_per_second.png)\n\n\n![Content Size](/assets/content_size.png)\n\n\n![Request Count](/assets/request_count.png)\n \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foya163%2Fapi-benchmark","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Foya163%2Fapi-benchmark","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Foya163%2Fapi-benchmark/lists"}