{"id":22781817,"url":"https://github.com/pratikgajjar/rustio","last_synced_at":"2025-03-30T14:40:40.412Z","repository":{"id":265796923,"uuid":"609668597","full_name":"pratikgajjar/rustio","owner":"pratikgajjar","description":"Unleashing Rust's Network Performance: Achieving an 11x Increase in throughput with Axum, Hyper, and Tokio","archived":false,"fork":false,"pushed_at":"2023-03-25T19:00:18.000Z","size":100,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-03-24T05:11:56.370Z","etag":null,"topics":["axum","hyper-rs","rust-lang","tokio","tokio-rs"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/pratikgajjar.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-04T21:45:30.000Z","updated_at":"2024-04-16T20:52:39.000Z","dependencies_parsed_at":"2024-12-01T00:50:58.689Z","dependency_job_id":null,"html_url":"https://github.com/pratikgajjar/rustio","commit_stats":null,"previous_names":["pratikgajjar/rustio"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratikgajjar%2Frustio","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratikgajjar%2Frustio/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratikgajjar%2Frustio/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/pratikgajjar%2Frustio/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/pratikgajjar","download_url":"https://codeload.github.com/pratikgajjar/rustio/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246333931,"owners_count":20760638,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["axum","hyper-rs","rust-lang","tokio","tokio-rs"],"created_at":"2024-12-11T21:08:49.651Z","updated_at":"2025-03-30T14:40:40.391Z","avatar_url":"https://github.com/pratikgajjar.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# rustio\n\n## Unleashing Rust's Network Performance: Achieving an 11x Increase in throughput with Axum, Hyper, and Tokio\n\n#### Scenario  \nNginx and Rust server are running on separate ec2 instance c6a.large in same network.\n\nIn rust server we have 2 APIs\n \n1. Returns static response =\u003e Throughput **47000** requests/second\n2. Make HTTP request to Nginx server -\u003e Parse Json -\u003e Return parsed data. =\u003e Throughput **2462** requests/second. [Issue]\n\nFor the similar benchmark in GoLang we got ~**20000** requests/second, which means there are no issues with infra/docker/client used to test rust server.\n\nGoLang App Specs: \n\nhttp server - [Fiber](https://github.com/gofiber/fiber) with prefork enabled\nJSON lib - [json-iterator](https://github.com/json-iterator/go)\n\nNginx request throughput ~**30000** requests/second. \n```\ncurl -v 172.31.50.91/\n{\"status\": 200, \"msg\": \"Good Luck!\"}\n```\n\nThe goal is to identify the cause of the performance regression in the Rust code and find ways to improve it. Some possible factors that could be causing the performance issue are:\n\nBenchmark result\n```\n[ec2-user@ip-172-31-50-91 ~]$ hey -z 10s  http://172.31.50.22:80/io\n\nSummary:\n  Total:        10.0168 secs\n  Slowest:      0.0692 secs\n  Fastest:      0.0006 secs\n  Average:      0.0203 secs\n  Requests/sec: 2462.4534\n\n  Total data:   813978 bytes\n  Size/request: 33 bytes\n\nResponse time histogram:\n  0.001 [1]     |\n  0.007 [12766] |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■\n  0.014 [1185]  |■■■■\n  0.021 [227]   |■\n  0.028 [494]   |■■\n  0.035 [1849]  |■■■■■■\n  0.042 [3840]  |■■■■■■■■■■■■\n  0.049 [3127]  |■■■■■■■■■■\n  0.055 [992]   |■■■\n  0.062 [174]   |■\n  0.069 [11]    |\n```\n\nMy attempts to improve perf.  \n- Both golang and rust are running on docker container on same instance one at a time. \n- System ulimit / somaxcon has been updated to not cause any bottleneck, since static response able to perform 47K rps, it shouldn't cause limitation \n- Moved external url to lazy_static but it didn't improve performance \n```rust\nlazy_static! {\n    static ref EXTERNAL_URL: String = env::var(\"EXTERNAL_URL\").unwrap();\n}\n```\n- Tried changing tokio flavour config, workerthreads = 2, 10, 16 - it didn't improve perf. \n```rust\n#[tokio::main(flavor = \"multi_thread\", worker_threads = 10)]\n```\n- Looked into how to make sure hyper network call is being done in tokio async compatible way -\u003e Earlier It had **247** requests/second. Was able to improve IO call by 10x via moving to stream based response processing. Reaching 2400 but still there is scope to improve. \n\nIO Call API - [GitHub Link](https://github.com/pratikgajjar/rustio/blob/bb893a864e3225f9448c76fa0ccaab23f9ec930c/src/main.rs#L35)\n```rust\npub async fn io_call( State(state): State\u003cAppState\u003e) -\u003e Json\u003cIOCall\u003e {\n    let external_url = state.external_url.parse().unwrap();\n    let client = Client::new();\n    let resp = client.get(external_url).await.unwrap();\n    let body = hyper::body::aggregate(resp).await.unwrap();\n\n    Json(serde_json::from_reader(body.reader()).unwrap())\n}\n```\n\n### Solution \n\nThanks to [@kmdreko](https://stackoverflow.com/users/2189130/kmdreko)\n\nMoving hyper client initialization to AppState resolved the problem. \n\n[Git diff](https://github.com/pratikgajjar/rustio/commit/1885fd1e56e3eae156433a8e589e61422757f4fe)\n\n```rust\n pub async fn io_call(State(state): State\u003cAppState\u003e) -\u003e Json\u003cIOCall\u003e {\n    let external_url = state.external_url.parse().unwrap();\n    let resp = state.client.get(external_url).await.unwrap();\n    let body = hyper::body::aggregate(resp).await.unwrap();\n\n    Json(serde_json::from_reader(body.reader()).unwrap())\n}\n```\n\n```log\n[ec2-user@ip-172-31-50-91 ~]$ hey -z 10s http://172.31.50.22:80/io\n\n\nSummary:\n  Total:        10.0026 secs\n  Slowest:      0.0235 secs\n  Fastest:      0.0002 secs\n  Average:      0.0019 secs\n  Requests/sec: 26876.1036\n\n  Total data:   8871456 bytes\n  Size/request: 33 bytes\n\nResponse time histogram:\n  0.000 [1]     |\n  0.003 [212705]        |■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■\n  0.005 [39980] |■■■■■■■■\n  0.007 [10976] |■■\n  0.010 [4259]  |■\n  0.012 [794]   |\n  0.014 [94]    |\n  0.016 [17]    |\n  0.019 [0]     |\n  0.021 [4]     |\n  0.023 [2]     |\n\n\nLatency distribution:\n  10% in 0.0006 secs\n  25% in 0.0009 secs\n  50% in 0.0013 secs\n  75% in 0.0022 secs\n  90% in 0.0038 secs\n  95% in 0.0052 secs\n  99% in 0.0083 secs\n\nDetails (average, fastest, slowest):\n  DNS+dialup:   0.0000 secs, 0.0002 secs, 0.0235 secs\n  DNS-lookup:   0.0000 secs, 0.0000 secs, 0.0000 secs\n  req write:    0.0000 secs, 0.0000 secs, 0.0086 secs\n  resp wait:    0.0018 secs, 0.0002 secs, 0.0234 secs\n  resp read:    0.0001 secs, 0.0000 secs, 0.0109 secs\n\nStatus code distribution:\n  [200] 268832 responses\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpratikgajjar%2Frustio","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpratikgajjar%2Frustio","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpratikgajjar%2Frustio/lists"}