{"id":32192177,"url":"https://github.com/tulos/boot-criterium","last_synced_at":"2025-10-22T02:00:05.420Z","repository":{"id":62435019,"uuid":"50773481","full_name":"tulos/boot-criterium","owner":"tulos","description":"Boot tasks that run your benchmarks!","archived":false,"fork":false,"pushed_at":"2016-02-25T11:08:27.000Z","size":21,"stargazers_count":6,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-02T20:34:15.291Z","etag":null,"topics":["benchmark","boot","boot-tasks","clojure","criterium"],"latest_commit_sha":null,"homepage":null,"language":"Clojure","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"epl-1.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tulos.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-01-31T12:32:44.000Z","updated_at":"2016-05-07T22:29:47.000Z","dependencies_parsed_at":"2022-11-01T20:46:01.974Z","dependency_job_id":null,"html_url":"https://github.com/tulos/boot-criterium","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/tulos/boot-criterium","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tulos%2Fboot-criterium","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tulos%2Fboot-criterium/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tulos%2Fboot-criterium/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tulos%2Fboot-criterium/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tulos","download_url":"https://codeload.github.com/tulos/boot-criterium/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tulos%2Fboot-criterium/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":280365593,"owners_count":26318385,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-21T02:00:06.614Z","response_time":58,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["benchmark","boot","boot-tasks","clojure","criterium"],"created_at":"2025-10-22T02:00:01.336Z","updated_at":"2025-10-22T02:00:05.414Z","avatar_url":"https://github.com/tulos.png","language":"Clojure","funding_links":[],"categories":[],"sub_categories":[],"readme":"# boot-criterium\n\n[](dependency)\n```clojure\n[tulos/boot-criterium \"0.3.0\"] ;; latest release\n```\n[](/dependency)\n\nA set of Boot tasks to benchmark your code using\n[Criterium](https://github.com/hugoduncan/criterium).\n\n## Motivation\n\nSo, you want to benchmark some code? What do you do? Set up a new Leiningen\nproject, add a Criterium dependency, start the REPL, create a source file,\nwrite your function, run the benchmark, stare at the output? What if you wanted\nto benchmark a simple one-liner? Now you can!\n\n```bash\n$ boot -d tulos/boot-criterium \\\n    bench -l sleep-1000 -g \"'(Thread/sleep 1000)\" -Q -- \\\n    bench -l sleep-500 -g \"'(Thread/sleep 500)\" -Q -- \\\n    report -f table -O\n\n| :benchmark/goal |       :mean |   :variance |    :upper-q |    :lower-q | :evaluation-count | :outlier-effect |\n|-----------------+-------------+-------------+-------------+-------------+-------------------+-----------------|\n|      sleep-1000 |  1.0007 sec | 438.2283 µs |  1.0011 sec |  1.0002 sec |                 6 |       :moderate |\n|       sleep-500 | 500.7362 ms | 449.0227 µs | 501.1333 ms | 500.2216 ms |                 6 |       :moderate |\n```\n\nVoila! We've just run a `criterium/quick-benchmark` with the default options\nand made sure that sleeping for 500 millis takes about half a second and\nsleeping for 1000 millis takes about twice as long.\n\nUsually we want to benchmark how different approaches to the same problem\nperform in comparison. Criterium doesn't provide an output format that would be\namenable to a quick comparison though. Neither does it provide the guidance on\nhow to setup the benchmark cases. Admittedly, it's just Clojure code, but it's\nstill Clojure code that you have to write.\n\nThere's the [Perforate](https://github.com/davidsantiago/perforate)\nLeiningen-based project that provides a benchmarking framework. You write\ngoals, usually in separate namespaces, and construct cases (variants) with\ndifferent methods of achieving these goals as multimethods. `boot-criterium`\nborrows the concept of a *goal* and the customizable output formats, nothing\nmore.\n\nCurrently `boot-criterium` will not provide any kind of structure above the\n`bench` task.  You'll have to write the Clojure code yourself. It's early days\nthough. We'll see if some tooling emerges through use that we'll want to\ninclude in this project.\n\n## Usage\n\nYou've already seen how a snippet of code can be run from the command line. The\nrest of this document will assume we're running in the Boot REPL.\n\nYou can run a single benchmark like so:\n\n```clojure\n(bench :goal 'my/function)\n```\n\nThis will add a file with the EDN result of the benchmark to the fileset (try\nrunning Criterium to see the result map for yourself, AFAIK it's not documented\nanywhere).  To make use of the result you either need to output the fileset to\nthe filesystem using the Boot's built-in `target` task or use the `report` task\nprovided here:\n\n```clojure\n(report :formatter 'table, :stdout true)\n```\n\nThe nice thing about `report` is that it operates on the results of all the\npreceding `bench` tasks:\n\n```clojure\n(boot\n  (bench :goal 'my/function-1)\n  (bench :goal 'my/function-2)\n  (report :formatter 'table)\n  (target))\n```\n\nnote that we didn't specify the `stdout` option to the `report` task. The above\nwill write the resulting fileset into the `target/` folder:\n\n```bash\n$ tree target\n\ntarget/\n├── criterium\n│   ├── my_function-1.edn\n│   ├── my_function-2.edn\n│   └── results.out\n\n$ cat target/criterium/results.out\n\n| :benchmark/goal |       :mean |   :variance |    :upper-q |    :lower-q | :evaluation-count | :outlier-effect |\n|-----------------+-------------+-------------+-------------+-------------+-------------------+-----------------|\n|   my/function-1 |  1.0007 sec | 349.2924 µs |  1.0011 sec |  1.0003 sec |                 6 |       :moderate |\n|   my/function-2 | 500.8664 ms | 376.1524 µs | 501.1621 ms | 500.3924 ms |                 6 |       :moderate |\n```\n\n### Dependencies\n\nNow let's try something interesting - suppose you want to benchmark some code\nwith different dependency versions. Let's try running a simple `(reduce +\n(range 1000))` across Clojure 1.6, 1.7 and 1.8:\n\n```clojure\n(def code '(reduce + (range 1000)))\n\n(boot\n  (bench :label \"1.6\", :goal code, :dependencies '[[org.clojure/clojure \"1.6.0\"]])\n  (bench :label \"1.7\", :goal code, :dependencies '[[org.clojure/clojure \"1.7.0\"]])\n  (bench :label \"1.8\", :goal code, :dependencies '[[org.clojure/clojure \"1.8.0\"]])\n  (report :formatter 'table, :stdout true))\n\n| :benchmark/goal |      :mean |   :variance |   :upper-q |   :lower-q | :evaluation-count | :outlier-effect |\n|-----------------+------------+-------------+------------+------------+-------------------+-----------------|\n|             1.6 | 37.9297 µs |   1.6069 µs | 41.9038 µs | 36.2114 µs |           1632780 |       :moderate |\n|             1.7 | 11.0225 µs | 275.3703 ns | 11.5597 µs | 10.5775 µs |           6743580 |       :moderate |\n|             1.8 |  9.7970 µs | 476.5232 ns | 10.8728 µs |  8.9932 µs |           6468900 |       :moderate |\n```\n\nYou'd have to start three REPLs to do the same without Boot!\n\n### Flags\n\nWhen running for the first time you will want to set the `-W` (warn) flag. This\nway Criterium will report whether you have any JVM options present that can\nimpeded the benchmark. You can then fix the JVM options by setting the\n`BOOT_JVM_OPTIONS` env variable.\n\n### Docs\n\nFor complete documentation see:\n\n```clojure\n(doc bench)\n```\n\nand\n\n```clojure\n(doc report)\n```\n\n## License\n\nCopyright © 2016 Tulos Capital\n\nDistributed under the Eclipse Public License.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftulos%2Fboot-criterium","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftulos%2Fboot-criterium","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftulos%2Fboot-criterium/lists"}