https://github.com/second-state/docker-quark-benchmark
https://github.com/second-state/docker-quark-benchmark
Last synced: 8 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/second-state/docker-quark-benchmark
- Owner: second-state
- Created: 2022-11-07T05:53:18.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2022-12-27T14:52:30.000Z (over 3 years ago)
- Last Synced: 2025-05-07T18:57:29.731Z (about 1 year ago)
- Language: JavaScript
- Size: 2.51 MB
- Stars: 0
- Watchers: 5
- Forks: 0
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
## Usage
- Install [Quark][]
- Use [commit b0dd795298c5804a0e323c6956d3149ce4a6c5e1](https://github.com/QuarkContainer/Quark/commit/92d1cb74ff6dbde184160e0bb400a7153a6f9e00)
- Install [gVisor][].
- Use [commit f054c314ec020f0b26266f267221033ffda82b67](https://github.com/google/gvisor/commit/f054c314ec020f0b26266f267221033ffda82b67)
- Install [Docker buildx plugin][buildx]
- Build and install [second-state/runwasi][runwasi]
- Use [commit ba7ea907f8593ef677d46aee7edc0ff8e3cd4be8](https://github.com/second-state/runwasi/commit/ba7ea907f8593ef677d46aee7edc0ff8e3cd4be8)
- Use [wasmedge branch of moby](https://github.com/CaptainVincent/moby/tree/wasmedge) for dockerd
- Use [commit 270211953ea4c3a88cae6eb566aa3a32d2d4b1ba](https://github.com/CaptainVincent/moby/commit/270211953ea4c3a88cae6eb566aa3a32d2d4b1ba)
- Start `dockerd` with `quark` and `runsc` runtime and `config/daemon.json` config:
```
sudo dockerd -D -H unix:///tmp/dockerd/docker.sock \
--data-root /tmp/dockerd/root \
--pidfile /tmp/dockerd/docker.pid \
--config-file config/daemon.json \
--add-runtime quark=/usr/local/bin/quark \
--add-runtime runsc=/usr/local/bin/runsc
```
- Create and use docker context:
```
docker context create benchmark --docker host=unix:///tmp/dockerd/docker.sock
docker context use benchmark
```
- Run benchmarks:
```
./benchmark.sh
```
- Results
- For time measurement, the unit is sec (lower is better).
- We use `time` command to monitor time usage.
- For memory measurement, the unit is MB (lower is better).
- We use `docker stats` command to monitor memory usage.
- Since [runwasi][] does not support `docker stats` to get memory usage,
we did not list memory usage for `benchmark_wasmedge_quickjs` here.
```
### container_start_time
benchmark_name min max avg std
benchmark_runc_nodejs 0.419 0.454 0.436 0.011
benchmark_quark_nodejs 0.572 0.626 0.606 0.016
benchmark_gvisor_nodejs 0.671 0.759 0.707 0.027
benchmark_wasmedge_quickjs 0.346 0.392 0.360 0.013
### container_execution_time
benchmark_name min max avg std
benchmark_runc_nodejs 20.422 20.856 20.504 0.128
benchmark_quark_nodejs 20.701 20.804 20.753 0.032
benchmark_gvisor_nodejs 20.590 20.800 20.658 0.061
benchmark_wasmedge_quickjs 55.012 58.590 55.974 1.045
### max_memory_usage
benchmark_name min max avg std
benchmark_runc_nodejs 341.400 369.200 352.990 8.389
benchmark_quark_nodejs 1398.784 1446.912 1423.462 12.168
benchmark_gvisor_nodejs 355.400 380.100 367.000 9.080
```
## Environments
- **benchmark_runc_nodejs**
- Use docker default runtime (runc) to start a `node:16` container to run `v8-v7` benchmark.
- **benchmark_quark_nodejs**
- Use [Quark][] runtime to start a `node:16` container to run `v8-v7` benchmark.
- **benchmark_gvisor_nodejs**
- Use [gVisor][] runtime to start a `node:16` container to run `v8-v7` benchmark.
- **benchmark_wasmedge_quickjs**
- Use `wasmedge_quickjs.wasm` from [wasmedge-quickjs][].
- Compile `wasmedge_quickjs.wasm` to `wasmedge_quickjs.wasm.so` using `wasmedgec`.
- Use [second-state/runwasi][runwasi] WasmEdge runtime to start a container,
which uses `wasmedge_quickjs.wasm.so` as a javascript engine to run `v8-v7` benchmark.
## Node.js Benchmarks
Here we use [V8 benchmark suite version 7][v8-v7]. It contains the following benchmarks:
- `crypto.js`
- Crypto related benchmarks.
- `deltablue.js`
- A JavaScript implementation of the DeltaBlue constraint-solving algorithm.
- `earley-boyer.js`
- Classic Scheme benchmarks translated to JavaScript.
- `navier-stokes.js`
- 2D NavierStokes equations solver, heavily manipulates double precision arrays.
- `raytrace.js`
- Ray tracer benchmark.
- `regexp.js`
- Regular expression benchmark generated by extracting regular expression operations from 50 of the most popular web pages.
- `richards.js`
- OS kernel simulation benchmark, originally written in BCPL.
- `splay.js`
- Data manipulation benchmark that deals with splay trees and exercises the automatic memory management subsystem.
- Check [here](https://developers.google.com/octane/benchmark) for more information.
### Run the Node.js Benchmarks
- Run it seperately:
```
$ cd nodejs/v8-v7/
$ node crypto.js
Crypto: 46674
----
Score (version 7): 46674
```
- Run all the benchmarks (score, higher is better):
```
$ cd nodejs/v8-v7/
$ node v8-v7.js
Crypto: 46714
DeltaBlue: 83686
EarleyBoyer: 65225
NavierStokes: 43691
RayTrace: 63639
RegExp: 9799
Richards: 45886
Splay: 37323
----
Score (version 7): 43096
```
[Quark]: https://github.com/QuarkContainer/Quark
[gVisor]: https://github.com/google/gvisor
[buildx]: https://github.com/docker/buildx
[v8-v7]: https://github.com/mozilla/arewefastyet/tree/master/benchmarks/v8-v7
[wasmedge-quickjs]: https://github.com/second-state/wasmedge-quickjs
[runwasi]: https://github.com/second-state/runwasi.git