https://github.com/ivotron/aver
Validation library
https://github.com/ivotron/aver
Last synced: about 1 year ago
JSON representation
Validation library
- Host: GitHub
- URL: https://github.com/ivotron/aver
- Owner: ivotron
- License: bsd-3-clause
- Created: 2015-05-13T00:08:41.000Z (about 11 years ago)
- Default Branch: master
- Last Pushed: 2015-10-28T06:42:24.000Z (over 10 years ago)
- Last Synced: 2025-03-29T11:43:06.120Z (about 1 year ago)
- Language: Go
- Homepage:
- Size: 205 KB
- Stars: 2
- Watchers: 5
- Forks: 0
- Open Issues: 14
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Project Aver
[](https://travis-ci.org/ivotron/aver)
Aver lets you quickly test claims made about the performance of a
system. Aver can be thought as a system that checks high-level
assertions (in the
[TDD](http://en.wikipedia.org/wiki/Test-driven_development) sense)
against a dataset corresponding to system performance metrics.
## Overview
Consider the following gathered metrics:
|Size|Workload|Method|Replication|Throughput|
|:--:|:------:|:----:|:---------:|:--------:|
| 1 | read | a | 1 | 141 |
| 1 | read | b | 1 | 145 |
| 1 | write | a | 1 | 142 |
| 1 | write | b | 1 | 149 |
| . | ... | . | . | . |
| . | ... | . | . | . |
| 32 | write | a | 5 | 210 |
| 32 | write | b | 5 | 136 |
The above (truncated) table contains performance measurements of a
distributed storage system (throughput column in MB/s). The throughput
is in function of the size of the cluster, replication factor (how
many copies of the data), type of request (read/write) and the method
used to replicate data.
Now, suppose we make the claim that method `a` beats method `b` by 2x.
At this point, you have two options: (1) you believe in our expertise
and trust our word, or (2) you re-run the test and check the results
to confirm our claims. Aver shortens the time it takes to validate
results by providing a simple language that can be used to express
this type of assertions, as well as a small program that test the
validations over the captured data. The following statement expresses
our claim:
```
for
size > 4 and replication = *
expect
throughput(method='a') > throughput(method='b') * 2
```
In prose, the above states that, regardless of the replication factor,
method `a` is twice as fast as method `b` when the size of the cluster
goes above a certain threshold (4 in this case). Given this statement
and a pointer to where the dataset resides, aver checks whether this
validation holds.
## Usage
There are two ways of using Aver. Programatically or through the CLI.