https://github.com/smarr/rebenchdb
ReBenchDB records benchmark results and provides customizable reporting to track and analyze run-time performance of software programs.
https://github.com/smarr/rebenchdb
benchmarking continuous-benchmarking continuous-testing performance-analysis performance-monitoring performance-tracking reproducibility research-tool science
Last synced: 6 months ago
JSON representation
ReBenchDB records benchmark results and provides customizable reporting to track and analyze run-time performance of software programs.
- Host: GitHub
- URL: https://github.com/smarr/rebenchdb
- Owner: smarr
- License: mit
- Created: 2019-12-19T01:01:39.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2025-09-27T21:31:40.000Z (6 months ago)
- Last Synced: 2025-09-27T23:34:02.759Z (6 months ago)
- Topics: benchmarking, continuous-benchmarking, continuous-testing, performance-analysis, performance-monitoring, performance-tracking, reproducibility, research-tool, science
- Language: TypeScript
- Homepage:
- Size: 8.93 MB
- Stars: 17
- Watchers: 3
- Forks: 6
- Open Issues: 48
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# ReBenchDB
[](https://travis-ci.com/smarr/ReBenchDB)
[](https://rebench.readthedocs.io/)
[](https://doi.org/10.5281/zenodo.1311762)
ReBenchDB records benchmark results and provides customizable reporting
to track and analyze run-time performance of software programs.
It is used for example to track the performance of multiple language implementations,
including [SOMns](https://github.com/smarr/SOMns) and other implementations
from the [family of Simple Object Machines](https://som-st.github.io/).
[ReBench](https://github.com/smarr/ReBench) is currently the main benchmark
runner to record data for ReBenchDB.
## Goals and Features
ReBenchDB is designed to
- record and store benchmark measurements
- track information about the environment in which the benchmarks were executed
- enable performance tracking over time
- enable performance comparison of experiments
### Features
- compare performance between specific commits
- show aggregated results on an overview plot
- give summary statistics
- show per-benchmark details
- a plot that allows to judge noise
- a plot with per-iteration data
- a plot of previous results
- various metrics and the command line
- profiling information
- compare performance across different executors
- per-project timeline view
- per-project data inventory and data export
## Non-Goals
ReBenchDB isn't
- a benchmark runner or benchmarking framework:
Check [ReBench](https://github.com/smarr/ReBench) if you need one
- a statistics library:
We currently use R for our statistic needs, but anything outputting HTML would be suitable.
## Docker/Podman Setup
The repository contains a `Dockerfile` and a `docker-compose.yml`, which will
install all dependencies and setup the required PostgreSQL database.
With Docker, this should be usable with:
```bash
docker compose -f ./docker-compose.yml up
```
For Podman users, podman-compose is needed:
```bash
pip3 install podman-compose # if not already available
podman-compose up
```
## Installation and Usage
ReBenchDB is implemented in TypeScript on top of Node.js.
Data is stored in a PostgreSQL database.
Custom reports are executed via a shell script, though, we use R to create
reports.
To ensure that the SVG for plots is generated using a font available in most browsers,
the server needs them when generating plots. We currently use Arial as default font.
On Ubuntu, these fonts can be installed with:
```bash
apt install ttf-mscorefonts-installer
```
TODO: write a detailed description on what is required to set things up.
## Support and Contributions
In case you encounter issues,
please feel free to [open an issue](https://github.com/smarr/ReBenchDB/issues/new)
so that we can help.
For contributions, we use the [GitHub flow](https://guides.github.com/introduction/flow/)
of pull requests, discussion, and revisions. For larger contributions,
it is likely useful to discuss them upfront in an issue first.
## Similar Projects
[Codespeed](https://github.com/tobami/codespeed/) "is a web application to monitor
and analyze the performance of your code." We used it for almost 10 years before
starting ReBenchDB. As such it provided a lot of inspiration and at this point,
is a more mature and proven alternative.