{"id":15069473,"url":"https://github.com/sbt/sbt-jmh","last_synced_at":"2025-05-15T00:12:09.979Z","repository":{"id":17185725,"uuid":"19953200","full_name":"sbt/sbt-jmh","owner":"sbt","description":"\"Trust no one, bench everything.\" - sbt plugin for JMH (Java Microbenchmark Harness)","archived":false,"fork":false,"pushed_at":"2025-03-18T07:38:04.000Z","size":499,"stargazers_count":787,"open_issues_count":29,"forks_count":89,"subscribers_count":22,"default_branch":"main","last_synced_at":"2025-05-12T05:02:59.659Z","etag":null,"topics":["benchmark","openjdk","scala"],"latest_commit_sha":null,"homepage":"","language":"Scala","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":"swapagarwal/JARVIS-on-Messenger","license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/sbt.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2014-05-19T18:39:03.000Z","updated_at":"2025-03-18T07:38:02.000Z","dependencies_parsed_at":"2024-03-18T18:52:28.192Z","dependency_job_id":"9c209a03-d972-44ca-8791-9d4023851dc6","html_url":"https://github.com/sbt/sbt-jmh","commit_stats":null,"previous_names":["ktoso/sbt-jmh"],"tags_count":55,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbt%2Fsbt-jmh","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbt%2Fsbt-jmh/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbt%2Fsbt-jmh/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/sbt%2Fsbt-jmh/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/sbt","download_url":"https://codeload.github.com/sbt/sbt-jmh/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254249206,"owners_count":22039029,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","openjdk","scala"],"created_at":"2024-09-25T01:42:42.234Z","updated_at":"2025-05-15T00:12:04.970Z","avatar_url":"https://github.com/sbt.png","language":"Scala","funding_links":[],"categories":["Table of Contents"],"sub_categories":["Sbt plugins"],"readme":"sbt-jmh\n=======\n\nSBT plugin for running [OpenJDK JMH](http://openjdk.java.net/projects/code-tools/jmh/) benchmarks.\n\nJMH about itself:\n-----------------\n\nJMH is a Java harness for building, running, and analysing nano/micro/milli/macro benchmarks written in Java and other languages targeting the JVM.\n\nPlease read [nanotrusting nanotime](http://shipilev.net/blog/2014/nanotrusting-nanotime/) and other blog posts on micro-benchmarking (or why most benchmarks are wrong) and make sure your benchmark is valid,\nbefore you set out to implement your benchmarks.\n\nVersions\n--------\n\n| Plugin version                                                                             | Default JMH version                                        | Notes                           | \n|--------------------------------------------------------------------------------------------|------------------------------------------------------------|:-------------------------------:|\n| [`0.4.7`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.7)  (sbt 1.3.0+)              | [`1.37`](https://github.com/openjdk/jmh/releases/tag/1.37) |                                 | \n| [`0.4.6`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.6)  (sbt 1.3.0+)              | [`1.37`](https://github.com/openjdk/jmh/releases/tag/1.37) |                                 | \n| [`0.4.5`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.5)  (sbt 1.3.0+)              | [`1.36`](https://github.com/openjdk/jmh/releases/tag/1.36) |                                 | \n| [`0.4.4`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.4)  (sbt 1.3.0+)              | [`1.36`](https://github.com/openjdk/jmh/releases/tag/1.36) |                                 | \n| [`0.4.3`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.3)  (sbt 1.3.0+)              | [`1.32`](https://github.com/openjdk/jmh/releases/tag/1.32) |                                 | \n| [`0.4.2`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.2)  (sbt 1.3.0+)              | [`1.31`](https://github.com/openjdk/jmh/releases/tag/1.31) | JMH `-prof async` supports 2.x  | \n| [`0.4.1`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.1)  (sbt 1.3.0+)              | [`1.30`](https://github.com/openjdk/jmh/releases/tag/1.30) |                                 | \n| [`0.4.0`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.4.0)  (sbt 1.3.0+)              | [`1.25`](https://github.com/openjdk/jmh/releases/tag/1.25) | profilers now in JMH core       | \n| [`0.3.7`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.3.7)  (sbt 0.13.17 / sbt 1.1.4) | [`1.21`](https://github.com/openjdk/jmh/releases/tag/1.21) | support JDK 11                  | \n| [`0.3.6`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.3.6)  (sbt 0.13.17 / sbt 1.1.4) | [`1.21`](https://github.com/openjdk/jmh/releases/tag/1.21) | support JDK 11                  |\n| [`0.3.4`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.3.4)  (sbt 0.13.17 / sbt 1.1.4) | [`1.21`](https://github.com/openjdk/jmh/releases/tag/1.21) | support of GraalVM              |\n| [`0.3.3`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.3.3)  (sbt 0.13.17 / sbt 1.1.1) | [`1.20`](https://github.com/openjdk/jmh/releases/tag/1.20) | JMH bugfix release              |\n| [`0.3.2`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.3.2)  (sbt 0.13.16 / sbt 1.0)   | [`1.19`](https://github.com/openjdk/jmh/releases/tag/1.19) | minor bugfix release            |\n| [`0.3.1`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.3.1)  (sbt 0.13.16 / sbt 1.0)   | [`1.19`](https://github.com/openjdk/jmh/releases/tag/1.19) | minor bugfix release            |\n| [`0.3.0`](https://github.com/ktoso/sbt-jmh/releases/tag/v0.3.0)  (sbt 0.13.16 / sbt 1.0)   | [`1.19`](https://github.com/openjdk/jmh/releases/tag/1.19) | async profiler, flame-graphs    |\n| ...                                                                                        | ...                                                        |                                 |\n\nNot interesting versions are skipped in the above listing. Always use the newest which has the JMH version you need.\nYou should stick to the latest version at all times anyway of course.\n\nAdding to your project\n----------------------\n\nSince sbt-jmh is an **AutoPlugin** all you need to do in order to activate it in \nyour project is to add the below line to your `project/plugins.sbt` file:\n\n```scala\n// project/plugins.sbt\naddSbtPlugin(\"pl.project13.scala\" % \"sbt-jmh\" % \"0.4.7\")\n```\n\nand enable it in the projects where you want to (useful in multi-project builds, as you can enable it only where you need it):\n\n```scala\n// build.sbt\nenablePlugins(JmhPlugin)\n```\n\nIf you define your project in a `Build.scala`, you also need the following import:\n\n```scala\nimport pl.project13.scala.sbt.JmhPlugin\n```\n\nYou can read more about [auto plugins in sbt on it's documentation page](https://www.scala-sbt.org/1.x/docs/Using-Plugins.html#Enabling+and+disabling+auto+plugins).\n\nWrite your benchmarks in `src/main/scala`. They will be picked up and instrumented by the plugin.\n\nJMH has a very specific way of working (it generates loads of code), so you should prepare a separate project for your benchmarks. In it, just type `Jmh/run` in order to run your benchmarks.\nAll JMH options work as expected. For help type `Jmh/run -h`. Another example of running it is:\n\n```sbt\nJmh/run -i 3 -wi 3 -f1 -t1 .*FalseSharing.*\n```\n\nWhich means \"3 iterations\" \"3 warmup iterations\" \"1 fork\" \"1 thread\". Please note that benchmarks should be usually executed at least in 10 iterations (as a rule of thumb), but more is better.\n\n**For \"real\" results we recommend to at least warm up 10 to 20 iterations, and then measure 10 to 20 iterations again. Forking the JVM is required to avoid falling into specific optimisations (no JVM optimisation is really \"completely\" predictable)**\n\nIf your benchmark should be a module in a multimodule project and needs access to another modules test classes then you\nmight want to define your benchmarks in `src/test` as well (because [Intellij does not support \"compile-\u003etest\" dependencies](https://youtrack.jetbrains.com/issue/SCL-8396)).\nWhile this is not directly supported it can be achieved with some tweaks. Assuming the benchmarks live in a module `bench` and need access\n to test classes from `anotherModule`, you have to define this dependency in your `build.sbt`:\n```scala\nlazy val bench = project\n  .dependsOn(anotherModule % \"test-\u003etest\")\n  .enablePlugins(JmhPlugin)\n  .settings(\n     Jmh / sourceDirectory := (Test / sourceDirectory).value,\n     Jmh / classDirectory := (Test / classDirectory).value,\n     Jmh / dependencyClasspath := (Test / dependencyClasspath).value,\n     // rewire tasks, so that 'bench/Jmh/run' automatically invokes 'bench/Jmh/compile' (otherwise a clean 'bench/Jmh/run' would fail)\n     Jmh / compile := (Jmh / compile).dependsOn(Test / compile).value,\n     Jmh / run := (Jmh / run).dependsOn(Jmh / compile).evaluated\n  )\n```\n\nOptions\n-------\n\nPlease invoke `Jmh/run -h` to get a full list of run as well as output format options.\n\n**Useful hint**: If you plan to aggregate the collected data you should have a look at the available output formats (`-lrf`).\nFor example it's possible to keep the benchmark's results as csv or json files for later regression analysis.\n\nUsing Java Flight Recorder / async-profiler.\n----------------------------\n\n**NOTE**: `sbt-jmh`-s integration with async-profiler and Java Flight Recorder has been contributed to the\nJMH project as of JMH 1.25 and removed from this project. Please migrate to using `-prof jfr` / `-prof async`.\nUse `-prof jfr:help` / `-prof async:help` to list available options.\n\nExamples\n--------\nThe [examples are scala-fied examples from the original JMH repo](https://github.com/ktoso/sbt-jmh/tree/main/plugin/src/sbt-test/sbt-jmh/run/src/main/scala/org/openjdk/jmh/samples), check them out, and run them!\n\nThe results will look somewhat like this:\n\n```\n...\n\n[info] # Run progress: 92.86% complete, ETA 00:00:15\n[info] # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home/jre/bin/java\n[info] # VM options: \u003cnone\u003e\n[info] # Fork: 1 of 1\n[info] # Warmup: 2 iterations, single-shot each\n[info] # Measurement: 3 iterations, single-shot each\n[info] # Threads: 1 thread, will synchronize iterations\n[info] # Benchmark mode: Single shot invocation time\n[info] # Benchmark: org.openjdk.jmh.samples.JMHSample_02_BenchmarkModes.measureSingleShot\n[info] # Warmup Iteration   1: 100322.000 us\n[info] # Warmup Iteration   2: 100556.000 us\n[info] Iteration   1: 100162.000 us\n[info] Iteration   2: 100468.000 us\n[info] Iteration   3: 100706.000 us\n[info]\n[info] Result : 100445.333 ±(99.9%) 4975.198 us\n[info]   Statistics: (min, avg, max) = (100162.000, 100445.333, 100706.000), stdev = 272.707\n[info]   Confidence interval (99.9%): [95470.135, 105420.532]\n[info]\n[info]\n[info] # Run progress: 96.43% complete, ETA 00:00:07\n[info] # VM invoker: /Library/Java/JavaVirtualMachines/jdk1.7.0_60.jdk/Contents/Home/jre/bin/java\n[info] # VM options: \u003cnone\u003e\n[info] # Fork: 1 of 1\n[info] # Warmup: 2 iterations, single-shot each, 5000 calls per batch\n[info] # Measurement: 3 iterations, single-shot each, 5000 calls per batch\n[info] # Threads: 1 thread, will synchronize iterations\n[info] # Benchmark mode: Single shot invocation time\n[info] # Benchmark: org.openjdk.jmh.samples.JMHSample_26_BatchSize.measureRight\n[info] # Warmup Iteration   1: 15.344 ms\n[info] # Warmup Iteration   2: 13.499 ms\n[info] Iteration   1: 2.305 ms\n[info] Iteration   2: 0.716 ms\n[info] Iteration   3: 0.473 ms\n[info]\n[info] Result : 1.165 ±(99.9%) 18.153 ms\n[info]   Statistics: (min, avg, max) = (0.473, 1.165, 2.305), stdev = 0.995\n[info]   Confidence interval (99.9%): [-16.988, 19.317]\n[info]\n[info]\n[info] Benchmark                                                 Mode   Samples         Mean   Mean error    Units\n[info] o.o.j.s.JMHSample_22_FalseSharing.baseline               thrpt         3      692.034      179.561   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.baseline:reader        thrpt         3      199.185      185.188   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.baseline:writer        thrpt         3      492.850        7.307   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.contended              thrpt         3      706.532      293.880   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.contended:reader       thrpt         3      210.202      277.801   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.contended:writer       thrpt         3      496.330       78.508   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.hierarchy              thrpt         3     1751.941      222.535   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.hierarchy:reader       thrpt         3     1289.003      277.126   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.hierarchy:writer       thrpt         3      462.938       55.329   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.padded                 thrpt         3     1745.650       83.783   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.padded:reader          thrpt         3     1281.877       47.922   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.padded:writer          thrpt         3      463.773      104.223   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.sparse                 thrpt         3     1362.515      461.782   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.sparse:reader          thrpt         3      898.282      415.388   ops/us\n[info] o.o.j.s.JMHSample_22_FalseSharing.sparse:writer          thrpt         3      464.233       49.958   ops/us\n```\n\nAdvanced: Using custom Runners\n------------------------------\nIt is possible to hand over the running of JMH to an `App` implemented by you, which allows you to programmatically\naccess all test results and modify JMH arguments before you actually invoke it.\n\nTo use a custom runner class with `runMain`, simply use it: `Jmh/runMain com.example.MyRunner -i 10 .*` –\nan example for this is available in [plugin/src/sbt-test/sbt-jmh/runMain](plugin/src/sbt-test/sbt-jmh/runMain) (open the `test` file).\n\nTo replace the runner class which is used when you type `Jmh/run`, you can set the class in your build file –\nan example for this is available in [plugin/src/sbt-test/sbt-jmh/custom-runner](plugin/src/sbt-test/sbt-jmh/custom-runner) (open the `build.sbt` file).\n\nContributing\n============\n\nYes, pull requests and opening issues is very welcome!\n\nThe plugin is maintained at an best-effort basis -- submitting a PR is the best way of getting something done :-)\n\nYou can locally publish the plugin with:\n\n```\nsbt 'project plugin; ^publishLocal'\n```\n\nPlease test your changes by adding to the [scripted test suite][sbt-jmh/plugin/src/sbt-test/sbt-jmh/]\nwhich can be run with:\n\n```\n sbt 'project plugin; ^scripted'\n```\n\nSpecial thanks\n--------------\n\nSpecial thanks for contributing async-profiler and flame-graphs support and other improvements\ngo to [@retronym](https://github.com/retronym) of Lightbend's Scala team.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbt%2Fsbt-jmh","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsbt%2Fsbt-jmh","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsbt%2Fsbt-jmh/lists"}