https://github.com/ishtms/rust-framework-bench
Benchmarking utility to test the performance of backend servers, written in Rust.
https://github.com/ishtms/rust-framework-bench
Last synced: about 1 month ago
JSON representation
Benchmarking utility to test the performance of backend servers, written in Rust.
- Host: GitHub
- URL: https://github.com/ishtms/rust-framework-bench
- Owner: ishtms
- Created: 2022-07-16T17:46:13.000Z (almost 3 years ago)
- Default Branch: master
- Last Pushed: 2023-07-17T17:27:26.000Z (almost 2 years ago)
- Last Synced: 2025-05-03T03:02:49.943Z (about 1 month ago)
- Language: Rust
- Homepage:
- Size: 461 KB
- Stars: 19
- Watchers: 2
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: readme.dev.md
Awesome Lists containing this project
README
# Rust framework benchmarks
Benchmarking utility to test the performance of all the rust web frameworks. Built with [rust](https://rust-lang.org) 🚀.
# Demo
### **(Last updated: Wed Jul 27 2022 02:59:45)**
## Frameworks included
**[Nickel](https://github.com/nickel-org/nickel.rs)** - An expressjs inspired framework for rust
# Results
| Concurrency: 2048 | Duration: 45 secs | Threads: 2 |
|:-------------------:|:---------------------:|:--------------:|
| **Name** | Req/sec | Avg Latency | Min Latency | Max Latency | Std Dev | 95% | 99% | 99.9% | # Requests | Transfer Rate | # Errors |
|:------------:|:-----------:|:-----------:|:-----------:|:-----------:|:-----------:|:-----------:|:----:|:----:|:----:|:-----------:|:-----------:|
|**Nickel** |20,128|96.23ms|14.54ms|1340.39ms|147.12ms|701.37ms|945.37ms|1061.11ms|905,718|39.36KB/Sec|894756|
## Benchmarking tool
The benchmarks have been performed using [rewrk](https://github.com/ChillFish8/rewrk), locally.
Check the raw output from rewrk [here](https://github.com/Ishtmeet-Singh/rust-framework-benchmarks/tree/master/perf).
## Try it yourself
Everything is automated, including adding a framework, generating `md` file output, and running the tests without having to start all the servers at once!
Pleas make sure you do not have capped soft limit or hard limit for file descriptors, this may cause benchmarks with high concurrency (-c) fail with OS error 54,
to fix that -
## Linux
1. Open `/etc/sysctl.conf`
2. Add `fs.file-max = 65536`
3. Reboot your pc and verify it after rebooting with `sysctl -p`
## Mac
1. Check your current limit - `launchctl limit maxfiles`
2. The first entry is the soft limit, and the second entry is hard limit. In most cases the hard limit is unlimited, it's the soft limit that we need to change
3. Update the hard limit `sudo launchctl limit maxfiles 65536 200000`
4. Reboot.
Alternatively, if you wish to change the soft limit to only run the benchmark one time, you can change it for your current terminal session by
`ulimit -n 65536`.
This doesn't require boot, and will reset back to the default (usually 2560) after restarting the terminal.
To run the tests locally, please follow the steps -
1. Download the repository as a zip, or clone/fork it.
2. `cd rust-framework-benchmarks`
3. `cargo build --release --bins`
4. Run the main script and you're good to go..
`./target/release/main` or `cargo run --release --bin main`
All the output will be stored in `perf/{name}/{concurrency}.txt*`
## Machine used
M1 Max MacBook Pro 2021 - 64GB ram, 10 CPU cores and 32 GPU cores
## Suggestions and changes
All the suggestions, code changes or additions of another web framework is appreciated. I'd like to keep the code as close as a real world scenario, instead of optimising it to the metal.
To add a new library/framework, please make sure to use the `PORT` provided through the benchmark dynamically. Default is `3000` for all. You can change it in `config.json`.
Also, to add a framework, add an entry inside `config.json` for the benchmarks to detect it.
```json
[
{
// Name of your framework. Displayed in the readme and during logs
"name": "Axum",
// Default for all the frameworks
"port": 3000,
// the name of the file that your framework is located
"binary": "axum",
// Github or Crates.io or Website
"url": "https://github.com/tokio-rs/axum"
}
]
```
```rs
fn get_port_number() -> String {
std::env::args().collect::>()[1].clone()
}
Server::build().bind("hello-world", format!("127.0.0.1:{}", port_number))
```