https://github.com/workday/warp-core
Scala framework for collecting performance metrics and conducting sound experimental benchmarking.
https://github.com/workday/warp-core
Last synced: 12 months ago
JSON representation
Scala framework for collecting performance metrics and conducting sound experimental benchmarking.
- Host: GitHub
- URL: https://github.com/workday/warp-core
- Owner: Workday
- License: mit
- Created: 2018-04-12T02:09:44.000Z (about 8 years ago)
- Default Branch: master
- Last Pushed: 2025-04-23T21:49:19.000Z (about 1 year ago)
- Last Synced: 2025-06-17T18:52:02.990Z (12 months ago)
- Language: Scala
- Homepage:
- Size: 20.6 MB
- Stars: 13
- Watchers: 8
- Forks: 17
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING
- License: LICENSE
Awesome Lists containing this project
README
# warp-core
[](https://travis-ci.org/Workday/warp-core)
[](https://coveralls.io/github/Workday/warp-core?branch=master)
[](https://maven-badges.herokuapp.com/maven-central/com.workday.warp/warp-core_2.11)
[](https://maven-badges.herokuapp.com/maven-central/com.workday.warp/warp-core_2.12)
[](https://maven-badges.herokuapp.com/maven-central/com.workday.warp/warp-core_2.13)
WARP (Workday Automated Regression Platform) is a flexible, lightweight, (mostly) functional Scala framework for collecting performance metrics and conducting sound experimental benchmarking.
WARP features a domain specific language for describing experimental designs (conducting repeated trials for different experimental groups). Our library allows users to wrap existing tests with layers of measurement collectors that write performance metrics to a relational database. We also allow users to create arbiters to express custom failure criteria and performance requirements. One arbiter included out of the box is the RobustPcaArbiter, which uses machine learning to detect when a test is deviating from expected behavior and signal an alert.
We believe engineers should reason more scientifically about code performance, and we are excited to provide a platform that allows for easily describing an experiment, collecting benchmark results, and conducting statistical analyses to verify hypotheses about expected performance.
More detailed documentation can be found [here.](https://workday.github.io/warp-core)
## Getting Started
### Starting local services
You can start up the required services (note that the included docker-compose file is not intended to be used in production) to run tests with
```
$ docker-compose up -d
```
And test via
```
$ ./gradlew clean test
```
or to just run unit tests, run the `unitTest` task.
All port values and service version numbers are in `.env`.
## Code Coverage Requirements
We use scoverage and coveralls gradle plugins to track code coverage. We enforce that high coverage should be maintained. At time of
writing, coverage must be at least 92% for a build to pass. If you want to test coveralls out on your fork, sign in to coveralls
and get your repo token. Then you can generate the coverage reports and submit them to coveralls using
```
$ export COVERALLS_REPO_TOKEN=abcdefg
$ ./gradlew clean reportScoverage coveralls
```
## Scalafix
We use [scalafix](https://scalacenter.github.io/scalafix/) to automatically refactor code. Since we are interested in using semantic rules,
we also need to use semanticdb scala compiler plugin, which harvests and dumps semantic information about the symbols and types in our program.
Scalafix semantic rules depend on semanticdb compiler output. Scalafix should be run like this:
```
$ ./gradlew clean scalafix
```
## Publishing
We use the `maven-publish` gradle plugin.
https://docs.gradle.org/current/userguide/publishing_maven.html
Please use the included `publish.sh` for uploading artifacts. This script handles some subtle interaction between
creating repo tags and scala multiversion plugin.
Example usage:
```
./publish.sh snapshot minor local
```
Will increment minor version component and publish a snapshot (eg 2.3.0-SNAPSHOT) to local maven repo.
To publish to sonatype, the invocation would be something like:
```
./publish.sh candidate minor sonatype
```
To publish to sonatype, you'll need to configure your sonatype and signing credentials as project properties:
[create sonatype jira account](https://issues.sonatype.org/secure/Signup!default.jspa)
[create pgp keys](https://central.sonatype.org/pages/working-with-pgp-signatures.html)
```
signing.keyId=BEEF
signing.password=abc123
signing.secretKeyRingFile=/full/path/to/secring.gpg
sonatypeUsername=jean-luc.picard
sonatypePassword=makeItSoNumberOne
```
Signing is not required for a local publish.
## Scala Multiversion
We use [gradle-scala-multiversion-plugin](https://github.com/ADTRAN/gradle-scala-multiversion-plugin)
to cross-compile the project with different scala-lang major versions and publish artifacts with scala version suffixes.
The versions are defined in gradle.properties, however you can also override from the command line:
```
$ ./gradlew -PscalaVersions=2.11.8,2.12.6,2.13.5 test
```
This plugin works by repeatedly invoking the gradle task graph with each different scala version specified.
Without any version specified, gradle will use the defaultScalaVersion from gradle.properties. This means local IDE builds
will use just one scala version. If you need to run with all configured scala versions, pass the project property `allScalaVersions`
```
$ ./gradlew -PallScalaVersions test
```
We support Scala cross-compilation for 2.11, 2.12, and 2.13, with the only caveat being the `warp-core-gatling` submodule. This is due to Gatling's choice to only compile with one (usually the latest) Scala version.
## Versioning
We use the `nebula.release` plugin to determine versions.
https://github.com/nebula-plugins/nebula-release-plugin
Versions exist as git tags in the canonical fork. If you are publishing from your fork, you may need to periodically
sync the tags:
```
git fetch upstream --tags
git push origin --tags
```
There are 4 types of releases we support:
- final
- candidate (rc)
- devSnapshot (includes some extra information in the version, including branch name and commit hash)
- snapshot
Artifacts with type `snapshot` or `devSnapshot` are published to sonatype snapshots repo,
while `final` and `candidate` artifacts are published to sonatype releases repo.
Please use the included `publish.sh` script for publishing, as that script handles interaction between creating repo tags
and scala multiversion. We don't want to create multiple tags during a release process and incorrectly publish some artifacts
under the wrong version.
## Dependencies
To enforce repeatable builds while allowing developers to use the flexibility of dynamic dependency ranges, we use the
gradle's native dependency-locking mechanism.
We create/update lock files as follows:
```
./gradlew resolveAndLockAll --write-locks
```
Note that you may need to run this task for other scala versions besides the project default scala version
`./gradlew resolveAndLockAll --write-locks -PallScalaVersions`
Also note that this locks all configurations in one single build execution. If you have updated `versionInfo.gradle`, you probably need to recreate the lock files as well. If any configuration needs to be excluded/filtered, you can do so in the `resolveAndLockAll` step. Please see https://docs.gradle.org/6.2.1/userguide/dependency_locking.html#generating_and_updating_dependency_locks for more detailed usage.
Please avoid using global locks. We have noticed behavior where this can override the version of scala-library used by the
zinc compiler.
## Common Errors
```
~/code/warp-core(master*) » gradle clean unitTest --refresh-dependencies
* What went wrong:
A problem occurred evaluating root project 'warp-core'.
> com.google.common.util.concurrent.ExecutionError: java.lang.NoClassDefFoundError: org/gradle/api/artifacts/VersionConstraint
```
You probably need a newer version of gradle. The above error has been observed with gradle 4.2.1, however version 4.4 works.
The root cause appears to be a runtime incompatibility with one of the plugins.
```
$ docker-compose up -d
...
ERROR: for mysql Cannot start service mysql: driver failed programming external connectivity on endpoint warpcore_mysql_1 (31cb1d98ed696bba0c9ef8cdee368186c52c15c2d0c0f69f1ffae0e7db406a3d): Error starting userland proxy: Bind for 0.0.0.0:3306 failed: port is already allocated
ERROR: Encountered errors while bringing up the project.
```
Change the MySQL port (or that of the offending service) in `.env`
## Authors
Note that this code has been ported from an internal repository and the public commit history does not accurately reflect authorship.
See `CONTRIBUTORS.md` for a full list of everyone who has contributed to this project.