https://github.com/objectionary/sabj25
Full-coverage JMH benchmarks for the Java 25 Stream API, run across six JDK vendors
https://github.com/objectionary/sabj25
benchmark java stream stream-processing
Last synced: 10 days ago
JSON representation
Full-coverage JMH benchmarks for the Java 25 Stream API, run across six JDK vendors
- Host: GitHub
- URL: https://github.com/objectionary/sabj25
- Owner: objectionary
- License: mit
- Created: 2026-06-05T11:44:49.000Z (about 1 month ago)
- Default Branch: master
- Last Pushed: 2026-06-19T06:22:25.000Z (19 days ago)
- Last Synced: 2026-06-19T08:23:16.841Z (19 days ago)
- Topics: benchmark, java, stream, stream-processing
- Language: Java
- Homepage:
- Size: 171 KB
- Stars: 3
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.txt
Awesome Lists containing this project
README
# Stream API Benchmarks for Java 25
[](https://github.com/objectionary/sabj25/actions/workflows/benchmark.yml)
[](https://github.com/objectionary/sabj25/actions/workflows/mvn.yml)
[](https://github.com/objectionary/sabj25/blob/master/LICENSE.txt)
This repository contains a set of benchmarks for [Stream API] in Java 25.
The motivation of this repository is the lack of benchmarks that would
cover the entire set of terminal and non-terminal methods of
Java Stream API, in their different combinations.
The benchmarks of [Biboudis et al.][biboudis2014] ([sources][cotl])
and [Møller et al.][moller2020] ([sources][streamliner])
only cover `map()` and `filter()` methods.
[Rosales et al.][rosales2023] go further with `StreamProf`, the first
dedicated stream profiler for the JVM, yet they target the runtime
overhead of `map()`, `filter()`, and `reduce()` rather than the
breadth of the API.
A few open-source projects on GitHub benchmark streams as well,
yet each of them stays narrow:
[keaz/java-stream-benchmark][keaz] measures only `sort()` and
`toString()` to tell when a parallel stream pays off,
[Pask423/stream-benchmarks][pask423] studies the internals of
parallel streams across a range of thread counts,
[phonty29/stream-benchmarks][phonty29] compares streams against
ordinary `for` loops,
and [Nikolas-Charalambidis/java-16-mapmulti-benchmark][mapmulti]
contrasts the `mapMulti()` method with `flatMap()`.
The [SoftwareMill][softwaremill] blog benchmarks a single
log-processing pipeline of `map()`, `filter()`, and
`collect(groupingBy())`, again touching only a handful of methods.
None of them exercise the full set of terminal and intermediate
methods, let alone their combinations.
## Principles
The pipelines are built to measure the Stream API itself, not the
cleverness of a particular JIT compiler. They follow a few rules:
- **No incidental repetition.** Where a pipeline sets out to cover the
API, each method appears exactly once, so it measures the operation in
combination with others rather than a loop of the same call.
Repetition shows up only when it is the subject itself, as in the
chain inside `stateless` that repeats `map()` and `filter()` on
purpose to turn the call sites megamorphic.
- **No easy optimization hotspots.** A `Blackhole` observation sits
after each boxing stage, because otherwise GraalVM's partial escape
analysis scalar-replaces the boxed `Long` and `Double` values and the
whole pipeline collapses to almost nothing, measuring elision instead
of work.
- **Lambdas do real arithmetic.** No lambda is an identity function;
every element flows through genuine computation at each step.
- **Every result is verified.** Each pipeline checks its result against a
precomputed constant, so a run that silently skipped work fails loudly
rather than reporting a fast but wrong number.
Order-insensitive pipelines fold into a plain sum, while pipelines
whose contract includes encounter order, such as `sorted()`,
`limit()`, and `forEachOrdered()`, fold through a rolling `31`-based
mix, so a result that holds the right elements in the wrong order
fails too.
- **The full API, in combination.** The pipelines span all four stream
types, `long`, `int`, `double`, and object, and weave terminal and
intermediate methods together, including `flatMap()` and `mapMulti()`.
- **One facet of the API per benchmark.** `stateless` covers the
stateless operations including the one-to-one scalar conversions,
`stateful` the operations that must remember state, `fold` the
reduction terminals, `ordered` the short-circuiting ones, and so on
through gatherers, collectors, sources, and parallel execution.
## Results
The numbers come from [JMH][jmh] 1.37, the Java Microbenchmark Harness,
driven from a JUnit test that builds the `Options` and runs the `Runner`
programmatically rather than from the command line.
Every method is measured in `AverageTime` mode and reported in
milliseconds per operation (`ms/op`), so each score is the mean wall-clock
time of a single invocation of one `@Benchmark` method.
Because each pipeline processes a different element count and operation mix,
the rows measure distinct workloads: compare JVMs down a single column,
not one benchmark against another across rows.
Each benchmark runs in two forks (each a freshly started JVM), with three
warmup iterations of one second each to let the JIT compiler settle,
followed by five measurement iterations of two seconds each; the reported
score is the arithmetic mean of all ten measurement iterations across the
two forks.
The state is thread-scoped and JMH drives each benchmark from a single
measurement thread, though the pipelines that call `parallel()` still fan
their work across the shared fork-join pool; a `Blackhole` consumes
intermediate boxed values to stop the JIT from eliminating the pipeline
as dead code.
JMH writes the raw results to `target/jmh-result.csv`, which the CI
pipeline parses into the table below.
The benchmarks run on every push to `master`, once per JVM, and the
table is regenerated automatically:
| Benchmark | Temurin 25 | Zulu 25 | Corretto 25 | GraalVM 25 | Oracle 25 | Semeru 25 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
| `collectors` | 1312.738 | 1221.746 | 1092.833 | 787.697 | 1095.761 | 1169.289 |
| `concurrent` | 265.958 | 206.017 | 214.983 | 179.688 | 207.396 | 288.608 |
| `fanout` | 85.001 | 91.858 | 78.938 | 35.033 | 70.783 | 149.798 |
| `fold` | 142.771 | 163.478 | 174.177 | 129.078 | 175.477 | 446.568 |
| `gatherers` | 86.445 | 86.965 | 89.281 | 73.876 | 90.003 | 171.074 |
| `materialize` | 117.761 | 142.358 | 156.501 | 81.877 | 159.956 | 170.113 |
| `ordered` | 71.211 | 84.394 | 97.637 | 29.163 | 99.031 | 109.319 |
| `overhead` | 0.000 | 0.000 | 0.000 | 0.000 | 0.000 | 0.001 |
| `sources` | 157.340 | 155.154 | 166.665 | 146.758 | 168.246 | 246.980 |
| `stateful` | 23.012 | 68.589 | 77.165 | 63.881 | 75.528 | 77.044 |
| `stateless` | 69.037 | 79.759 | 86.250 | 81.295 | 85.274 | 141.200 |
| `text` | 8.955 | 9.562 | 10.366 | 10.887 | 10.174 | 15.974 |
All scores are in milliseconds per operation (ms/op); lower is better.
The results were calculated in [this GHA job][benchmark-gha]
on 2026-06-12 at 07:38.
Each JVM ran on its own GitHub-hosted Linux runner,
so the scores across columns are indicative, not strictly comparable.
[biboudis2014]: https://arxiv.org/abs/1406.6631
[cotl]: https://github.com/biboudis/clashofthelambdas
[moller2020]: https://dl.acm.org/doi/abs/10.1145/3428236
[streamliner]: https://github.com/cs-au-dk/streamliner
[keaz]: https://github.com/keaz/java-stream-benchmark
[pask423]: https://github.com/Pask423/stream-benchmarks
[phonty29]: https://github.com/phonty29/stream-benchmarks
[mapmulti]: https://github.com/Nikolas-Charalambidis/java-16-mapmulti-benchmark
[rosales2023]: https://arxiv.org/abs/2302.10006
[softwaremill]: https://softwaremill.com/benchmarking-java-streams/
[benchmark-gha]: https://github.com/objectionary/sabj25/actions/runs/27401566362
[jmh]: https://github.com/openjdk/jmh
[Stream API]: https://docs.oracle.com/en/java/javase/25/docs/api/java.base/java/util/stream/package-summary.html