Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/jonhoo/volley
Volley is a benchmarking tool for measuring the performance of server networking stacks.
https://github.com/jonhoo/volley
benchmark comparison network
Last synced: 3 months ago
JSON representation
Volley is a benchmarking tool for measuring the performance of server networking stacks.
- Host: GitHub
- URL: https://github.com/jonhoo/volley
- Owner: jonhoo
- License: mit
- Created: 2015-06-06T19:06:16.000Z (over 9 years ago)
- Default Branch: master
- Last Pushed: 2015-06-27T16:09:12.000Z (over 9 years ago)
- Last Synced: 2024-10-14T20:57:10.072Z (3 months ago)
- Topics: benchmark, comparison, network
- Language: C
- Size: 2.57 MB
- Stars: 123
- Watchers: 11
- Forks: 12
- Open Issues: 9
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Volley
Volley is a benchmarking tool for measuring the performance (latency in
particular) of server networking stacks. It can be used to determine the
performance of the lower levels of the stack (i.e. the kernel), of the
server application's programming language abstractions for sockets, or
of the libraries and code running within the application.
Volley spawns a configurable number of concurrent clients that all
connect to the server, and then performs ping-pong-like single-word
operations with the server to gather statistics. It will continue doing
so for as many iterations as it takes to produce statistically relevant
results.
## Goals
Volley can be used to benchmark many things, but I believe there are two
main categories of servers that are of interest. In particular, I
envision that there will be at least two implementations for every
language represented in Volley: one that is idiomatic, and one that is
optimized. Comparing the idiomatic solutions' performance is interesting
in and of itself, and comparing the optimized solutions might yield
insights into how well-suited a particular language is for doing
high-performance networking.
I say *at least* two, because there might be other interesting designs
to explore. For example, does the conventional wisdom that having a
worker pool improves performance actually still hold? What is the
performance difference between a forking and a threaded server? What is
the performance difference between asynchronous and synchronous I/O?
## Preliminary results
Running the volley benchmarks on an 80-core machine running Linux 3.16
with 40, 80 and 200 clients distributed across 40 cores yields the
results given in the graph below. Error bars denote the 99% confidence
interval. The benchmark machine has four NUMA nodes, each with 10 cores.
To reproduce, run:
```
benchmark/ $ experiment -r $(PWD)/..
benchmark/ $ grep us 'out/*/run-1/stdout.log' | sed 's@/run-[^/]*/stdout.log@@' | tr '/:-' '\t' | awk '{printf $2; for (i=3;i
```
This should output something akin to:
```
numactl -C +0-0 /home/.../volley/target/servers/ -p 2222
numactl -C 3-3 /home/.../volley/target/ -p 2222 -c 1
priming with 1000000 iterations across 1 clients
iteration complete: mean is 9us, stddev is 1.25us
9.04us
```
You may also see the error:
```
./benchmark/bench.sh: line 47: kill: (12345) - No such process
```
This can safely be ignored, and is an artefact of the fact that the
server may terminate when it receives a 0 challenge.
## Server improvements
If you believe an already implemented server could be improved (either
syntactically or semantically), please file an issue detailing the
problem, and if possible, submit a PR giving a proposed solution.