https://github.com/jgpc42/jmh-clojure
Seamless JMH benchmarking for Clojure
https://github.com/jgpc42/jmh-clojure
benchmarking clojure jmh performance
Last synced: about 1 month ago
JSON representation
Seamless JMH benchmarking for Clojure
- Host: GitHub
- URL: https://github.com/jgpc42/jmh-clojure
- Owner: jgpc42
- License: epl-1.0
- Created: 2017-09-27T22:24:14.000Z (over 7 years ago)
- Default Branch: master
- Last Pushed: 2025-01-11T13:58:06.000Z (6 months ago)
- Last Synced: 2025-01-11T14:44:36.057Z (6 months ago)
- Topics: benchmarking, clojure, jmh, performance
- Language: Clojure
- Size: 289 KB
- Stars: 133
- Watchers: 3
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
[](https://clojars.org/jmh-clojure)
[][ci]
### Dependency and version information
Click to show
[Leiningen][lein]
``` clojure
[jmh-clojure "0.4.1"]
```
[tools.deps][deps]
```clojure
{jmh-clojure/jmh-clojure {:mvn/version "0.4.1"}}
```
[Maven][maven]
``` xml
jmh-clojure
jmh-clojure
0.4.1
```
JDK versions 8 to 18 and Clojure versions 1.7 to 1.11 are currently [tested against][ci].
### What is it?
This library provides a data-oriented API to [JMH][jmh], the Java Microbenchmark Harness.
JMH is developed by OpenJDK JVM experts and goes to great lengths to ensure accurate benchmarks. Benchmarking on the JVM is a complex beast and, by extension, JMH takes a bit of effort to learn and use properly. That being said, JMH is very robust and configurable. If you are new to JMH, I would recommend browsing the [sample][samples] code and javadocs before using this library.
If you a need simpler, less strenuous tool, I would suggest looking at the popular [criterium][criterium] library.
### Quick start
As a simple example, let's say we want to benchmark our fn that gets the value at an arbitrary indexed type. Of course, the built in `nth` already does this, but we can't extend `nth` to existing types like `java.nio.ByteBuffer`, etc.
```clojure
(ns demo.core)
(defprotocol ValueAt
(value-at [x idx]))
(extend-protocol ValueAt
clojure.lang.Indexed
(value-at [i idx]
(.nth i idx))
CharSequence
(value-at [s idx]
(.charAt s idx))
#_...)
```
Benchmarks are [usually](#alternate-ways-to-run) described in data and are fully separated from definitions. The reason for this is twofold. First, decoupling is generally good design practice. And second, it allows us to easily take advantage of JMH process isolation (forking) for reliability and accurracy. More on this later.
For repeatability, we'll place the following data in a `benchmarks.edn` resource file in our project. (Note that using a file is not a requirement, we could also specify the same data in Clojure. The `:fn` key values would need to be quoted in that case, however.)
```clojure
{:benchmarks
[{:name :str, :fn demo.core/value-at, :args [:state/string, :state/index]}
{:name :vec, :fn demo.core/value-at, :args [:state/vector, :state/index]}]
:states
{:index {:fn (partial * 0.5), :args [:param/count]} ;; mid-point
:string {:fn demo.utils/make-str, :args [:param/count]}
:vector {:fn demo.utils/make-vec, :args [:param/count]}}
:params {:count 10}}
```
I have omitted showing the `demo.utils` namespace for brevity, it is defined [here][utils] if interested.
The above data should be fairly easy to understand. It is also a limited view of what can be specified. The [sample file][sample] provides a complete reference and explanation.
Now to run the benchmarks. We'll start a REPL in our project and evaluate the following. Note that we could instead use [lein-jmh][lein-jmh] or one of the other [supported tools](#tooling-support) to automate this entire process.
```clojure
(require '[jmh.core :as jmh]
'[clojure.java.io :as io]
'[clojure.edn :as edn])
(def bench-env
(-> "benchmarks.edn" io/resource slurp edn/read-string))
(def bench-opts
{:type :quick
:params {:count [31 100000]}
:profilers ["gc"]})
(jmh/run bench-env bench-opts)
;; => ({:name :str, :params {:count 31}, :score [1.44959801438209E8 "ops/s"], #_...}
;; {:name :str, :params {:count 100000}, :score [1.45485370497829E8 "ops/s"]}
;; {:name :vec, :params {:count 31}, :score [1.45550038851249E8 "ops/s"]}
;; {:name :vec, :params {:count 100000}, :score [8.5783753539823E7 "ops/s"]})
```
> **Note:** due to the way jmh-clojure [works][works], the `*compile-path*` directory should exist and be on your classpath before benchmarking. This is automated by tools like Leiningen. For `tools.deps`, see [here][task-cp].
The `run` fn takes a benchmark environment and an optional map. We select the `:quick` type: an [alias][alias-doc] for some common options. We override our default `:count` parameter sequence to measure our fn against small and large inputs. We also enable the gc profiler.
Notice how we have four results: one for each combination of parameter and benchmark fn. For this example, we have omitted lots of additional result map data, including the [profiler][profilers] information.
Note that the above results were taken from multiple [runs][result], which is always a good practice when benchmarking.
#### Alternate ways to run
Benchmarking expressions or fns manually without the data specification is also supported. For example, the `run-expr` macro provides an interface similar to criterium, and allows benchmarking of code that only resides in memory (that you are updating in a REPL, for example), rather than on disk (loadable via `require`). However, this forgoes JMH process isolation. For more on why benchmarking this way on the JVM can be sub-optimal, see [here][extended].
### Tooling support
This library can be used directly in a [bare REPL][bare], as shown above, or standalone via [tools.deps][cli]. For a more robust experience, see the [`jmh-clojure-task`][task] project. This companion library provides some additional convenience features like sorting, table output, easy uberjar creation, and more. It can be easily integrated with tools like [Leiningen][lein-jmh].
### More information
As previously mentioned, please see the [sample file][sample] for the complete benchmark environment reference. For `run` options, see the [docs][run-doc]. Also, see the [wiki][wiki] for additional examples and topics.
The materials for a talk I gave at a [London Clojurians][london] online meetup are also available [here][talk]. A video capture of the event can also be viewed on [YouTube][video].
### Running the tests
```bash
lein test
```
Or, `lein test-all` for all supported Clojure versions.
### License
Copyright © 2017-2025 Justin Conklin
Distributed under the Eclipse Public License, the same as Clojure.
[alias-doc]: https://jgpc42.github.io/jmh-clojure/doc/jmh.option.html#var-*type-aliases*
[bare]: https://github.com/jgpc42/jmh-clojure/wiki/Bare-REPL
[ci]: https://github.com/jgpc42/jmh-clojure/blob/master/.github/workflows/test.yml
[cli]: https://github.com/jgpc42/jmh-clojure/wiki/Clojure-1.9-CLI
[criterium]: https://github.com/hugoduncan/criterium
[deps]: https://github.com/clojure/tools.deps.alpha
[extended]: https://github.com/jgpc42/jmh-clojure/wiki/Extended
[jmh]: https://github.com/openjdk/jmh
[lein]: http://github.com/technomancy/leiningen
[lein-jmh]: https://github.com/jgpc42/lein-jmh
[london]: https://www.meetup.com/London-Clojurians/
[maven]: http://maven.apache.org
[profilers]: https://github.com/jgpc42/jmh-clojure/wiki/JMH-Profilers
[result]: https://gist.github.com/jgpc42/4d8a828f8d0739748afa71035f2b2c9c#file-results-edn
[run-doc]: https://jgpc42.github.io/jmh-clojure/doc/jmh.core.html#var-run
[sample]: https://github.com/jgpc42/jmh-clojure/blob/master/resources/sample.jmh.edn
[samples]: https://github.com/openjdk/jmh/tree/41548a7/jmh-samples/src/main/java/org/openjdk/jmh/samples
[talk]: https://github.com/jgpc42/london-clojurians-jmh-talk-2020
[task]: https://github.com/jgpc42/jmh-clojure-task
[task-cp]: https://github.com/jgpc42/jmh-clojure-task#usage
[utils]: https://gist.github.com/jgpc42/4d8a828f8d0739748afa71035f2b2c9c#file-utils-clj
[video]: https://www.youtube.com/watch?v=_6qVfFkBdWI
[wiki]: https://github.com/jgpc42/jmh-clojure/wiki
[works]: https://github.com/jgpc42/jmh-clojure/wiki/How-It-Works