https://github.com/gampleman/elm-bench
Rich Elm benchmarking with a powerful CLI runner
https://github.com/gampleman/elm-bench
benchmarking elm
Last synced: 2 days ago
JSON representation
Rich Elm benchmarking with a powerful CLI runner
- Host: GitHub
- URL: https://github.com/gampleman/elm-bench
- Owner: gampleman
- License: mit
- Created: 2026-05-14T11:17:07.000Z (about 2 months ago)
- Default Branch: main
- Last Pushed: 2026-05-20T14:46:15.000Z (about 2 months ago)
- Last Synced: 2026-06-17T20:09:31.432Z (19 days ago)
- Topics: benchmarking, elm
- Language: TypeScript
- Homepage: https://package.elm-lang.org/packages/gampleman/elm-bench/latest/
- Size: 180 KB
- Stars: 6
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# elm-bench
A complete benchmarking toolkit for Elm. Define benchmarks with automatic correctness verification, run them headlessly across multiple JS engines, and get rich terminal output with scaling charts.
## The Experience
**Define benchmarks with a clean API:**
```elm
module MyBenchmarks exposing (suite)
import Bench exposing (Benchmark)
suite : Benchmark
suite =
Bench.describe "Array.Extra"
[ Bench.rank "mapToList"
(\mapToList -> mapToList negate myArray)
[ ( "with foldr", MapToList.withFoldr )
, ( "with toIndexedList", MapToList.withListMap )
]
, Bench.scale "mapToList across sizes"
[ 10, 100, 1000, 10000 ]
(\n -> Array.fromList (List.range 1 n))
[ ( "with foldr", MapToList.withFoldr negate )
, ( "with toIndexedList", MapToList.withListMap negate )
]
]
```
**Run with a single command:**
```
$ elm-bench run --filter mapToList
```
**Get verification first, then results:**
```
Verifying benchmark correctness...
✓ All implementations produce consistent results
Seed: 1847293651 (reproduce with --seed 1847293651)
Running benchmarks in node...
Benchmark Results
Array.Extra / mapToList
───────────────────────────────────────────────────────
with foldr 980,124 runs/s (99.9%) fastest
with Array.toIndexedList 535,891 runs/s (99.8%) 1.83x slower
```
**See how algorithms scale with input size:**
```
Array.Extra / mapToList across sizes
30.2M ┼──╮───╮
17.5M ┤ ╰─────╮─────╮
10.1M ┤ ╰─────╮────╮
5.8M ┤ ╰────╮────╮
3.4M ┤ ╰────╮────╮
1.9M ┤ ╰────╮────╮
1.1M ┤ ╰────╮────╮
651.3K ┤ ╰────╮───╮
376.5K ┤ ╰───╮───╮
217.6K ┤ ╰───╮───╮
125.8K ┤ ╰───╮──╮
72.7K ┤ ╰──╮──╮
42.0K ┤ ╰──╮──╮
24.3K ┤ ╰──╮──╮
14.0K ┤ ╰──╮──
8.1K ┤ ╰──╮
4.7K ┤ ╰
(log scale)
n= 1 10 500 1000 10000
━━ with foldr (30.2M → 9.8K runs/s)
━━ with Array.toIndexedList (23.0M → 4.7K runs/s)
```
In terminals with inline image support (iTerm2, WezTerm, Ghostty, VS Code), a full-color PNG chart is rendered instead.
**Compare across JavaScript engines:**
```
$ elm-bench run -t node -t chromium -t firefox --filter mapToList
Array.Extra / indexedMapToList
node chromium firefox
────────────────────────────────────────────────────────────────────
with Array.foldr 608,609 runs/s 712,445 runs/s 838,702 runs/s
with Array.indexedMap 527,083 runs/s 634,221 runs/s 671,752 runs/s
with List.indexedMap 384,174 runs/s 445,890 runs/s 339,777 runs/s
with toIndexedList 302,367 runs/s 356,112 runs/s 330,534 runs/s
```
When rankings differ across targets, entries are highlighted in green (ranked higher than first target) or red (ranked lower) — immediately surfacing cross-engine performance characteristics.
## Quick Start
```bash
cd my-project
npx elm-bench init
```
This creates a `benchmarks/` directory with `elm.json` (with `gampleman/elm-bench` installed) and a starter `Benchmarks.elm` file.
Then run your benchmarks:
```bash
elm-bench run
```
## Installation
```bash
npm install -g elm-bench
```
Or as a project dependency:
```bash
npm install --save-dev elm-bench
```
## Key Features
### Automatic Correctness Verification
`Bench.rank`, `Bench.compare`, and `Bench.scale` automatically verify that all implementations produce the same result before benchmarking begins. This runs via `elm-test-rs` in seconds and catches broken optimizations before you waste time on misleading benchmark results.
```
Verifying benchmark correctness...
✗ Benchmark verification failed!
Some implementations produce different results:
↓ Array.Extra / reverse
✗ with List.reverse should match with push
Array.fromList [100,99,...,1]
╷
│ Expect.equal
╵
Array.fromList [1,2,...,100]
```
Use `Bench.skipEqualityCheck` for cases where implementations intentionally differ (e.g. different valid orderings).
### Optimization Workflow
Scaffold a head-to-head comparison for any function in your project:
```bash
elm-bench optimize MyModule.myFunction --arg "(List.range 1 1000)"
```
This:
1. Copies `MyModule` into `Baseline` and `Optimized` variants (with dead code removed)
2. Generates a `Bench.rank` benchmark comparing them
3. You edit the `Optimized` copy and iterate
Add more variants to compare:
```bash
elm-bench optimize MyModule.myFunction --add Experimental
```
Use `--watch` for instant feedback as you edit:
```bash
elm-bench run --watch
```
### Filtering
Only run what you need:
```bash
elm-bench run --filter mapToList
```
Filtering happens at the Elm level before benchmarks execute — so even with hundreds of benchmarks in your suite, filtered runs are fast.
### Fuzzer-Based Inputs
Use `elm-explorations/test` fuzzers for randomized benchmark input:
```elm
Bench.rankFuzz "mapToList"
(Fuzz.array (Fuzz.intRange 1 1000))
(\mapToList array -> mapToList negate array)
[ ( "with foldr", MapToList.withFoldr )
, ( "with toIndexedList", MapToList.withListMap )
]
```
The same fuzzer is used for both the correctness check (fuzz test) and to generate the benchmark input (seeded deterministically via `--seed`). The seed is printed with every run so you can reproduce results.
#### Debugging Fuzzer Inputs
If you see unexpected benchmark results and want to know what input the fuzzer generated, copy the seed from the output and use `Bench.sampleFuzzer` in `elm repl`:
```
> import Bench
> import Fuzz
> Bench.sampleFuzzer 1847293651 (Fuzz.list (Fuzz.intRange 1 100))
Just [42, 7, 93, ...] : Maybe (List Int)
```
This gives you the exact value the fuzzer produced during that run.
### Multi-Target Execution
Compare across JavaScript engines to ensure optimizations are consistent:
```bash
elm-bench run -t node -t chromium -t firefox -t webkit
```
Requires Playwright for browser targets: `npm install playwright`.
### Scaling Analysis
Understand algorithmic complexity with `Bench.scale`:
```elm
Bench.scale "sorting"
[ 10, 100, 1000, 10000 ]
(\n -> List.range 1 n |> List.reverse)
[ ( "List.sort", List.sort )
, ( "mergeSort", mergeSort )
]
```
Results are rendered as a chart with automatic log-scale detection.
### Output Formats
- **console** (default) — colored terminal output with progress bars and charts
- **json** — machine-readable for CI integration
- **markdown** — GitHub-flavored tables for easy paste into PRs
```bash
elm-bench run --reporter markdown
```
## CLI Reference
```
elm-bench init [options]
Initialize a benchmarks directory with elm-bench installed.
--project Base directory to create benchmarks/ in
elm-bench run [options] [globs...]
Run benchmarks.
-f, --filter Only run benchmarks matching pattern
-t, --target Execution target (repeatable): node, chromium, firefox, webkit
-r, --reporter Output: console (default), json, markdown
--compiler Path to elm binary
--project Path to benchmarks elm.json
--seed Random seed for fuzz-based inputs
--skip-test Skip correctness verification
--no-optimize Disable --optimize (for debugging)
-w, --watch Watch for changes and re-run affected benchmarks
elm-bench optimize [options]
Scaffold an optimization comparison benchmark.
--add Add a new variant to an existing optimization
--arg Argument expression for the runner (repeatable)
--project Path to elm.json
```
## Requirements
- Node.js >= 18
- Elm 0.19.1 (bundled with CLI)
- For browser targets: `npm install playwright`