https://github.com/technobre/powerutils.benchmarkdotnet.reporter
Tool to analyze and compare .NET benchmark reports
https://github.com/technobre/powerutils.benchmarkdotnet.reporter
benchmark benchmarkdotnet csharp dotent reports tests
Last synced: 9 days ago
JSON representation
Tool to analyze and compare .NET benchmark reports
- Host: GitHub
- URL: https://github.com/technobre/powerutils.benchmarkdotnet.reporter
- Owner: TechNobre
- License: mit
- Created: 2025-03-12T02:47:48.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2026-06-04T23:29:03.000Z (28 days ago)
- Last Synced: 2026-06-10T00:05:12.999Z (23 days ago)
- Topics: benchmark, benchmarkdotnet, csharp, dotent, reports, tests
- Language: C#
- Homepage: https://www.nuget.org/packages/PowerUtils.BenchmarkDotnet.Reporter
- Size: 971 KB
- Stars: 0
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Codeowners: .github/CODEOWNERS
- Security: SECURITY.md
Awesome Lists containing this project
README
# PowerUtils.BenchmarkDotnet.Reporter
***Tool to analyze and compare .NET benchmark reports***
[](https://dashboard.stryker-mutator.io/reports/github.com/TechNobre/PowerUtils.BenchmarkDotnet.Reporter/main)
[](https://sonarcloud.io/summary/new_code?id=TechNobre_PowerUtils.BenchmarkDotnet.Reporter)
[](https://sonarcloud.io/summary/new_code?id=TechNobre_PowerUtils.BenchmarkDotnet.Reporter)
[](https://sonarcloud.io/summary/new_code?id=TechNobre_PowerUtils.BenchmarkDotnet.Reporter)
[](https://sonarcloud.io/summary/new_code?id=TechNobre_PowerUtils.BenchmarkDotnet.Reporter)
[](https://www.nuget.org/packages/PowerUtils.BenchmarkDotnet.Reporter)
[](https://www.nuget.org/packages/PowerUtils.BenchmarkDotnet.Reporter)
[](https://github.com/TechNobre/PowerUtils.BenchmarkDotnet.Reporter/blob/main/LICENSE)
**PowerUtils.BenchmarkDotnet.Reporter** is a command-line tool used to analyze and compare .NET benchmark reports generated by [BenchmarkDotNet](https://www.nuget.org/packages/benchmarkdotnet). This tool is designed to run locally or in CI/CD pipelines, providing a simple way to visualize and compare benchmark results.
- [Prepare environment](#prepare-environment)
- [Install Tool](#install-tool)
- [Update Tool](#update-tool)
- [List installed tools](#list-installed-tools)
- [Uninstall Tool](#uninstall-tool)
- [Export reports using BenchmarkDotNet](#export-reports-using-benchmarkdotnet)
- [How to use](#how-to-use)
- [Run the tool](#run-the-tool)
- [Commands](#commands)
- [`compare`](#compare)
- [Options:](#options)
- [Threshold Units](#threshold-units)
- [Example of usage:](#example-of-usage)
- [Error Handling Options](#error-handling-options)
- [Exit Codes](#exit-codes)
- [GitHub Actions Setup](#github-actions-setup)
- [Acknowledgments](#acknowledgments)
- [Contribution](#contribution)
## Prepare environment
### Install Tool
This package is available through Nuget Repository: https://www.nuget.org/packages/PowerUtils.BenchmarkDotnet.Reporter
**Locally**
```bash
# Create a tool manifest file in the current directory
dotnet new tool-manifest
# Install the tool in the current directory
dotnet tool install PowerUtils.BenchmarkDotnet.Reporter
```
**Globally**
```bash
dotnet tool install --global PowerUtils.BenchmarkDotnet.Reporter
```
### Update Tool
**Locally**
```bash
dotnet tool update PowerUtils.BenchmarkDotnet.Reporter
```
**Update Tool**
```bash
dotnet tool update --global PowerUtils.BenchmarkDotnet.Reporter
```
### List installed tools
**Locally**
```bash
dotnet tool list
```
**Globally**
```bash
dotnet tool list --global
```
### Uninstall Tool
**Locally**
```bash
dotnet tool uninstall PowerUtils.BenchmarkDotnet.Reporter
```
**Globally**
```bash
dotnet tool uninstall --global PowerUtils.BenchmarkDotnet.Reporter
```
## Export reports using BenchmarkDotNet
To use the `PowerUtils.BenchmarkDotnet.Reporter` tool, you first need to generate benchmark reports using [BenchmarkDotNet](https://www.nuget.org/packages/BenchmarkDotNet) and export them in JSON format. You can do this by looking at the documentation of BenchmarkDotNet [here](https://benchmarkdotnet.org/articles/configs/exporters.html) or following the examples below:
> NOTE: Now the `PowerUtils.BenchmarkDotnet.Reporter` tool supports all the Json Formatters provided by BenchmarkDotNet, so you can choose the one that best suits your needs.
```csharp
// TheBenchmark.cs
using BenchmarkDotNet.Attributes;
[MemoryDiagnoser]
[JsonExporterAttribute.Full]
[JsonExporterAttribute.FullCompressed]
[JsonExporterAttribute.Brief]
[JsonExporterAttribute.BriefCompressed]
public class TheBenchmark
{
[Benchmark]
public void TheMethod()
{
// Your benchmark code here
}
}
// Program.cs
using BenchmarkDotNet.Running;
BenchmarkRunner.Run();
```
**Or**
```csharp
// TheBenchmark.cs
using BenchmarkDotNet.Attributes;
[MemoryDiagnoser]
public class TheBenchmark
{
[Benchmark]
public void TheMethod()
{
// Your benchmark code here
}
}
// Program.cs
using BenchmarkDotNet.Running;
var config = ManualConfig
.Create(DefaultConfig.Instance)
.AddExporter(JsonExporter.Full)
.AddExporter(JsonExporter.FullCompressed)
.AddExporter(JsonExporter.Brief)
.AddExporter(JsonExporter.BriefCompressed);
BenchmarkRunner.Run(config);
```
## How to use
### Run the tool
**Locally**
```bash
dotnet pbreporter [command] [options]
```
**Globally**
```bash
pbreporter [command] [options]
```
### Commands
#### `compare`
Compares two benchmark reports and generates a report with the differences.
**Example:**
```bash
pbreporter compare -b baseline-full.json -t target-full.json
```
##### Options:
* (`-b`, `--baseline`) ``: Path to the folder or file with Baseline report. **[Required]**
* (`-t`, `--target`) ``: Path to the folder or file with target reports. **[Required]**
* (`-tm`, `--threshold-mean`) ``: Throw an error when the mean threshold is met. Examples: 5%, 10ms, 10us, 100ns, 1s.
* (`-ta`, `--threshold-allocation`) ``: Throw an error when the allocation threshold is met. Examples: 5%, 10b, 10kb, 100mb, 1gb.
* (`-f`, `--format`) ``: Output format for the report. **[default: console]**
* (`-o`, `--output`) ``: Output directory to export the diff report. Default is current directory. **[default: ./BenchmarkReporter]**
* (`-fw`, `--fail-on-warnings`): Exit with error code when any threshold is hit during comparison. **[default: disabled]**
* (`-ft`, `--fail-on-threshold-hit`): Exit with error code when any threshold is hit during comparison. **[default: disabled]**
* (`-?`, `-h`, `--help`): Show help and usage information
###### Threshold Units
**Time (`-tm`):**
| Unit | Description | Example |
|------|-------------|---------|
| `ns` | Nanoseconds | `100ns` |
| `us` | Microseconds | `10us` |
| `ms` | Milliseconds | `10ms` |
| `s` | Seconds | `1s` |
| `%` | Percentage relative to baseline | `5%` |
**Memory (`-ta`):**
| Unit | Description | Example |
|------|-------------|---------|
| `b` | Bytes | `10b` |
| `kb` | Kilobytes | `10kb` |
| `mb` | Megabytes | `10mb` |
| `gb` | Gigabytes | `1gb` |
| `%` | Percentage relative to baseline | `5%` |
##### Example of usage:
**Simple usage**
```bash
pbreporter compare -b baseline-full.json -t target-full.json
```
**Passing folder paths**
```bash
pbreporter compare -b ./baseline-reports -t ./target-reports
```
> Note: You can pass a file path, a folder or mix both. The tool will automatically find the supported report files in the provided paths.
**With output format and directory**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -f json -f markdown -o ./out
```
**With thresholds**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -tm 5% -ta 12b
```
**With thresholds and output threshold report**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -tm 5% -f hit-txt
```
> Note: The `hit-txt` format will only generate when at least one threshold is hit.
**With console output**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -f console
```
> Note: The `console` format displays the comparison report directly in the terminal instead of creating a file.
**With Markdown output**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -f markdown
```
> Note: The `markdown` format is ideal for generating reports to upload to GitHub or other platforms that support Markdown rendering.
**With multiple formats**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -f json -f markdown -f console
```
##### Error Handling Options
The tool provides options to control exit codes for CI/CD integration and automated quality gates.
**Fail on warnings**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -fw
```
> Note: Exits with code 1 if any warnings are generated during comparison (e.g., environment differences between baseline and target).
**Fail on threshold hits**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -tm 5% -ta 10% -ft
```
> Note: Exits with code 2 if any performance thresholds are exceeded during comparison.
**Both error handling options**
```bash
pbreporter compare -b baseline-full.json -t target-full.json -tm 5% -fw -ft
```
> Note: If both conditions are met, warnings take priority and the tool exits with code 1.
###### Exit Codes
* **0**: Success - No issues detected
* **1**: Warnings detected (when `--fail-on-warnings` is enabled)
* **2**: Performance thresholds exceeded (when `--fail-on-threshold-hit` is enabled)
## GitHub Actions Setup
PowerUtils.BenchmarkDotnet.Reporter can be easily integrated into your CI/CD pipeline to automatically analyze and compare benchmark performance. For detailed instructions on setting up GitHub Actions workflows, see the [GitHub Actions Setup Guide](docs/github-actions-setup.md).
## Acknowledgments
This project was inspired by and based on the [ResultsComparer tool](https://github.com/dotnet/performance/tree/main/src/tools/ResultsComparer) from the .NET performance repository.
## Contribution
If you have any questions, comments, or suggestions, please open an [issue](https://github.com/TechNobre/PowerUtils.BenchmarkDotnet.Reporter/issues/new/choose) or create a [pull request](https://github.com/TechNobre/PowerUtils.BenchmarkDotnet.Reporter/compare).
Want to contribute to this tool? Check out the [Test Data Documentation](docs/test-data.md) to see sample benchmark reports available for testing your changes and developing new features.