https://github.com/hugoduncan/criterium
Benchmarking library for clojure
https://github.com/hugoduncan/criterium
Last synced: 3 days ago
JSON representation
Benchmarking library for clojure
- Host: GitHub
- URL: https://github.com/hugoduncan/criterium
- Owner: hugoduncan
- Created: 2009-12-16T15:40:35.000Z (over 15 years ago)
- Default Branch: develop
- Last Pushed: 2025-02-24T13:34:18.000Z (about 2 months ago)
- Last Synced: 2025-04-04T13:01:41.105Z (10 days ago)
- Language: Clojure
- Homepage: http://hugoduncan.github.io/criterium
- Size: 679 KB
- Stars: 1,203
- Watchers: 31
- Forks: 54
- Open Issues: 19
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-java - Criterium
README
# Criterium
Criterium measures the computation time of an expression. It is
designed to address some of the pitfalls of benchmarking, and benchmarking on
the JVM in particular.
This includes:
* statistical processing of multiple evaluations
* inclusion of a warm-up period, designed to allow the JIT compiler to
optimise its code
* purging of gc before testing, to isolate timings from GC state prior
to testing
* a final forced GC after testing to estimate impact of cleanup on the
timing results
## Installation
### Leiningen
Add the following to your `:dependencies`:
```clj
[criterium "0.4.6"]
```
### Maven
```xml
criterium
criterium
0.4.6
```
## Usage
The top level interface is in `criterium.core`.
(use 'criterium.core)
Use `bench` to run a benchmark in a simple manner.
```
(bench (Thread/sleep 1000))
=>
Execution time mean : 1.000803 sec
Execution time std-deviation : 328.501853 us
Execution time lower quantile : 1.000068 sec ( 2.5%)
Execution time upper quantile : 1.001186 sec (97.5%)
```
By default bench is quiet about its progress. Run `with-progress-reporting` to
get progress information on `*out*`.
```clj
(with-progress-reporting (bench (Thread/sleep 1000) :verbose))
(with-progress-reporting (quick-bench (Thread/sleep 1000) :verbose))
```
Lower level functions are available, that separate benchmark statistic
generation and reporting.
```clj
(report-result (benchmark (Thread/sleep 1000) {:verbose true}))
(report-result (quick-benchmark (Thread/sleep 1000)))
```
Note that results are returned to the user to prevent JIT from recognising that
the results are not used.
## Measurement Overhead Estimation
Criterium will automatically estimate a time for its measurement
overhead. The estimate is normally made once per session, and is
available in the `criterium.core/estimated-overhead-cache` var.
If the estimation is made while there is a lot of other processing
going on, then benchmarking quick functions may report small negative
times. You can force a recalculation of the overhead by calling
`criterium.core/estimated-overhead!`.
If you want consistency across JVM processes, it might be prudent to
explicitly set `criterium.core/estimated-overhead!` to a constant
value.
## References
[API Documentation](http://hugoduncan.github.io/criterium/0.4/api/)
[Annotated Source](http://hugoduncan.github.io/criterium/0.4/uberdoc.html)
See [Elliptic Group](http://www.ellipticgroup.com/html/benchmarkingArticle.html)
for a Java benchmarking library. The accompanying article describes many of the
JVM benchmarking pitfalls.
See [Criterion](http://hackage.haskell.org/package/criterion) for a Haskell
benchmarking library that applies many of the same statistical techniques.
## Todo
Serial correlation detection.
Multimodal distribution detection.
Use kernel density estimators?
## Releasing
To release, run the `release.sh` script. This requires that you have
git-flow enabled your git repository with `git flow init`, and that
you have configured your
[credentials for clojars](https://github.com/technomancy/leiningen/blob/stable/doc/DEPLOY.md).
## YourKit
YourKit is kindly supporting open source projects with its full-featured Java
Profiler.
YourKit, LLC is the creator of innovative and intelligent tools for profiling
Java and .NET applications. Take a look at YourKit's leading software products:
* YourKit Java Profiler and
* YourKit .NET Profiler.
## License
Licensed under [EPL](http://www.eclipse.org/legal/epl-v10.html)