Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/frenchy64/criterium-from-java
Use Clojure's criterium benchmarking library from other JVM languages
https://github.com/frenchy64/criterium-from-java
Last synced: 1 day ago
JSON representation
Use Clojure's criterium benchmarking library from other JVM languages
- Host: GitHub
- URL: https://github.com/frenchy64/criterium-from-java
- Owner: frenchy64
- License: other
- Created: 2024-04-05T19:16:30.000Z (6 months ago)
- Default Branch: main
- Last Pushed: 2024-04-08T02:03:34.000Z (6 months ago)
- Last Synced: 2024-09-07T10:17:54.795Z (26 days ago)
- Language: Java
- Size: 66.4 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Running Criterium benchmarks from Java and Maven
[](https://clojars.org/com.ambrosebs/criterium-from-java)
Clojure's [criterium](https://github.com/hugoduncan/criterium/) is a great library for microbenchmarking on the JVM.
This project packages it up for use from other JVM languages.
## API
The main public method is `Criterium.bench(Callable)`. It takes a callable that
executes the benchmark. You should initialize your benchmark (if appropriate) before returning the Callable.
[Example](test-project/java/example_benchmark/Main.java):
```java
package example_benchmark;
import com.ambrosebs.criterium_from_java.Criterium;
import java.util.concurrent.Callable;
import java.util.Map;
public class Main {
public static void main(String[] args) {
Map benchResults = Criterium.quickBench(new Callable() {
@Override
public Integer call() throws InterruptedException {
Thread.sleep(100);
return 10*2;
}
});
System.out.println("Results:");
System.out.println(benchResults.toString());
}
}
```
## Maven Dependency information
```xml
...
clojars
https://repo.clojars.org/
true
true
...
com.ambrosebs
criterium-from-java
1.1.0
...
```
## Example
There is an example benchmark that is run via Maven. The code looks like this:
```java
/**
* Example benchmark
*
* Demonstrates how to use bench(Callable) to kick off benchmarking.
**/
public static void main(String args[]) {
System.out.println("Running example benchmark ");
Callable myBench = new Callable() {
public Integer call() throws InterruptedException {
Thread.sleep(100);
return 10*2;
}
};
// takes a configuration map as described in Criterium class javadoc.
final Map config = new HashMap();
config.put("quick", true);
config.put("print-result", true);
config.put("verbose", true);
config.put("progress", true);
Map benchResults = bench(myBench, config);
System.out.println("\nReturns a map of results for programmatic manipulation.");
System.out.println("For example, benchmark run 5 return this value:");
List results = (List)benchResults.get("results");
System.out.println(results.get(4).toString());
System.out.println("\nWe can reproduce the same benchmark by copy-pasting the above serialized config string into the Java program.");
// also takes Clojure data, useful to reproduce previous results.
bench(myBench,
"{\"quick\" true, \"max-gc-attempts\" 100, \"samples\" 6, \"target-execution-time\" 100000000, \"warmup-jit-period\" 5000000000, \"tail-quantile\" 0.025, \"bootstrap-size\" 500, \"overhead\" 8.253210752367998E-9}");
}
```
Here is some example output:
```
./run.sh
+ mvn clean
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< com.ambrosebs:criterium-from-java >------------------
[INFO] Building criterium-from-java 1.1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- clean:3.2.0:clean (default-clean) @ criterium-from-java ---
[INFO] Deleting /Users/ambrose/Projects/criterium-from-java/target
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.271 s
[INFO] Finished at: 2024-04-07T01:12:38-05:00
[INFO] ------------------------------------------------------------------------
+ mvn compile
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< com.ambrosebs:criterium-from-java >------------------
[INFO] Building criterium-from-java 1.1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- build-helper:1.7:add-source (add-source) @ criterium-from-java ---
[INFO] Source directory: /Users/ambrose/Projects/criterium-from-java/java added.
[INFO]
[INFO] --- resources:3.3.1:resources (default-resources) @ criterium-from-java ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] Copying 0 resource from resources to target/classes
[INFO]
[INFO] --- compiler:3.11.0:compile (default-compile) @ criterium-from-java ---
[INFO] Changes detected - recompiling the module! :source
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform dependent!
[INFO] Compiling 1 source file with javac [debug target 1.8] to target/classes
[WARNING] bootstrap class path not set in conjunction with -source 8
[WARNING] source value 8 is obsolete and will be removed in a future release
[WARNING] target value 8 is obsolete and will be removed in a future release
[WARNING] To suppress warnings about obsolete options, use -Xlint:-options.
[INFO] /Users/ambrose/Projects/criterium-from-java/java/com/ambrosebs/criterium_from_java/Criterium.java: /Users/ambrose/Projects/criterium-from-java/java/com/ambrosebs/criterium_from_java/Criterium.java uses unchecked or unsafe operations.
[INFO] /Users/ambrose/Projects/criterium-from-java/java/com/ambrosebs/criterium_from_java/Criterium.java: Recompile with -Xlint:unchecked for details.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.330 s
[INFO] Finished at: 2024-04-07T01:12:41-05:00
[INFO] ------------------------------------------------------------------------
+ mvn exec:java -Dexec.mainClass=com.ambrosebs.criterium_from_java.Criterium
[INFO] Scanning for projects...
[INFO]
[INFO] -----------------< com.ambrosebs:criterium-from-java >------------------
[INFO] Building criterium-from-java 1.1.0-SNAPSHOT
[INFO] from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- exec:3.2.0:java (default-cli) @ criterium-from-java ---
Running example benchmark
Estimating sampling overhead
Warming up for JIT optimisations 10000000000 ...
compilation occurred before 454260 iterations
compilation occurred before 908441 iterations
compilation occurred before 1362622 iterations
compilation occurred before 1816803 iterations
compilation occurred before 3179346 iterations
compilation occurred before 5904432 iterations
compilation occurred before 8629518 iterations
compilation occurred before 11354604 iterations
compilation occurred before 11808785 iterations
compilation occurred before 20438224 iterations
compilation occurred before 24980034 iterations
compilation occurred before 25888396 iterations
compilation occurred before 27705120 iterations
compilation occurred before 28613482 iterations
compilation occurred before 34063654 iterations
compilation occurred before 48143265 iterations
compilation occurred before 49505808 iterations
compilation occurred before 56318523 iterations
compilation occurred before 57681066 iterations
compilation occurred before 65856324 iterations
compilation occurred before 73577401 iterations
compilation occurred before 80390116 iterations
compilation occurred before 85840288 iterations
compilation occurred before 86294469 iterations
compilation occurred before 114453691 iterations
compilation occurred before 114907872 iterations
compilation occurred before 115362053 iterations
compilation occurred before 116270415 iterations
compilation occurred before 145337999 iterations
compilation occurred before 148063085 iterations
compilation occurred before 173951402 iterations
compilation occurred before 177130669 iterations
compilation occurred before 177584850 iterations
compilation occurred before 183035022 iterations
compilation occurred before 183489203 iterations
compilation occurred before 190301918 iterations
compilation occurred before 232086570 iterations
compilation occurred before 253433077 iterations
compilation occurred before 257066525 iterations
compilation occurred before 257520706 iterations
compilation occurred before 298851177 iterations
compilation occurred before 299305358 iterations
compilation occurred before 322468589 iterations
compilation occurred before 433742934 iterations
compilation occurred before 448730907 iterations
compilation occurred before 456451984 iterations
compilation occurred before 457814527 iterations
compilation occurred before 498690817 iterations
compilation occurred before 525487496 iterations
compilation occurred before 563184519 iterations
compilation occurred before 609056800 iterations
compilation occurred before 759390711 iterations
compilation occurred before 1009190261 iterations
compilation occurred before 1023269872 iterations
compilation occurred before 1051429094 iterations
compilation occurred before 1068687972 iterations
compilation occurred before 1142265294 iterations
compilation occurred before 1159524172 iterations
compilation occurred before 1159978353 iterations
compilation occurred before 1169516154 iterations
compilation occurred before 1169970335 iterations
compilation occurred before 1170424516 iterations
compilation occurred before 1187229213 iterations
compilation occurred before 1187683394 iterations
Estimating execution count ...
Sampling ...
Final GC...
Checking GC...
Finding outliers ...
Bootstrapping ...
Checking outlier significance
Warming up for JIT optimisations 5000000000 ...
compilation occurred before 10 iterations
Estimating execution count ...
Sampling ...
Final GC...
Checking GC...
Finding outliers ...
Bootstrapping ...
Checking outlier significance
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 103.210107 ms
Execution time std-deviation : 1.784294 ms
Execution time lower quantile : 100.184480 ms ( 2.5%)
Execution time upper quantile : 104.768084 ms (97.5%)
Overhead used : 8.451731 ns
To reproduce via bench(Callable, String), use: "{\"progress\" true, \"target-execution-time\" 100000000, \"max-gc-attempts\" 100, \"print-result\" true, \"warmup-jit-period\" 5000000000, \"quick\" true, \"overhead\" 8.451731464212423E-9, \"bootstrap-size\" 500, \"tail-quantile\" 0.025, \"verbose\" true, \"samples\" 6}"
Result:
{"os-details"
{"arch" "x86_64",
"available-processors" 8,
"name" "Mac OS X",
"version" "14.3"},
"execution-count" 1,
"mean" [0.10321010683333333 [0.10135429133333333 0.104280358]],
"serialized-options"
"{\"progress\" true, \"target-execution-time\" 100000000, \"max-gc-attempts\" 100, \"print-result\" true, \"warmup-jit-period\" 5000000000, \"quick\" true, \"overhead\" 8.451731464212423E-9, \"bootstrap-size\" 500, \"tail-quantile\" 0.025, \"verbose\" true, \"samples\" 6}",
"total-time" 0.620001001,
"upper-q" [0.10476808400000001 [0.10388682562500001 0.104810285]],
"sample-count" 6,
"sample-mean"
[0.10333350016666668 [0.09809736797918299 0.10856963235415036]],
"results" [20 20 20 20 20 20],
"outliers"
{"low-severe" 0, "low-mild" 0, "high-mild" 0, "high-severe" 0},
"warmup-executions" 55,
"runtime-details"
{"input-arguments"
["-Dclassworlds.conf=/usr/local/Cellar/maven/3.9.6/libexec/bin/m2.conf"
"-Dmaven.home=/usr/local/Cellar/maven/3.9.6/libexec"
"-Dlibrary.jansi.path=/usr/local/Cellar/maven/3.9.6/libexec/lib/jansi-native"
"-Dmaven.multiModuleProjectDirectory=/Users/ambrose/Projects/criterium-from-java"],
"java-runtime-version" "21.0.1+12-LTS",
"clojure-version"
{"major" 1, "minor" 11, "incremental" 1, "qualifier" nil},
"vm-vendor" "Eclipse Adoptium",
"vm-name" "OpenJDK 64-Bit Server VM",
"spec-name" "Java Virtual Machine Specification",
"vm-version" "21.0.1+12-LTS",
"name" "[email protected]",
"java-version" "21.0.1",
"spec-vendor" "Oracle Corporation",
"sun-arch-data-model" "64",
"clojure-version-string" "1.11.1",
"spec-version" "21"},
"sample-variance" [3.046342253866968E-6 [0.0 0.0]],
"outlier-variance" 0.13888888888888873,
"final-gc-time" 76400143,
"lower-q" [0.10018448 [0.10018448 0.10257754300000001]],
"warmup-time" 5709716578,
"overhead" 8.451731464212423E-9,
"variance"
[3.1837051655162608E-6 [3.060973957574988E-7 5.718892606608664E-6]],
"options"
{"progress" true,
"target-execution-time" 100000000,
"max-gc-attempts" 100,
"print-result" true,
"warmup-jit-period" 5000000000,
"quick" true,
"overhead" 8.451731464212423E-9,
"bootstrap-size" 500,
"tail-quantile" 0.025,
"verbose" true,
"samples" 6},
"tail-quantile" 0.025,
"samples"
[100184480 103905666 104768084 102577543 104810285 103754943]}
Returns a map of results for programmatic manipulation.
For example, benchmark run 5 return this value:
20
We can reproduce the same benchmark by copy-pasting the above serialized config string into the Java program.
Evaluation count : 6 in 6 samples of 1 calls.
Execution time mean : 103.172682 ms
Execution time std-deviation : 1.125590 ms
Execution time lower quantile : 101.893942 ms ( 2.5%)
Execution time upper quantile : 104.607670 ms (97.5%)
Overhead used : 8.253211 ns
To reproduce via bench(Callable, String), use: "{\"quick\" true, \"max-gc-attempts\" 100, \"samples\" 6, \"target-execution-time\" 100000000, \"warmup-jit-period\" 5000000000, \"tail-quantile\" 0.025, \"bootstrap-size\" 500, \"overhead\" 8.253210752367998E-9}"
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 34.252 s
[INFO] Finished at: 2024-04-07T01:13:16-05:00
[INFO] ------------------------------------------------------------------------
```
## License
Copyright © 2024 Ambrose Bonnaire-Sergeant
This program and the accompanying materials are made available under the
terms of the Eclipse Public License 2.0 which is available at
http://www.eclipse.org/legal/epl-2.0.
This Source Code may also be made available under the following Secondary
Licenses when the conditions for such availability set forth in the Eclipse
Public License, v. 2.0 are satisfied: GNU General Public License as published by
the Free Software Foundation, either version 2 of the License, or (at your
option) any later version, with the GNU Classpath Exception which is available
at https://www.gnu.org/software/classpath/license.html.