{"id":13507469,"url":"https://github.com/alco/benchfella","last_synced_at":"2025-05-15T14:05:18.001Z","repository":{"id":18482963,"uuid":"21678410","full_name":"alco/benchfella","owner":"alco","description":"Microbenchmarking tool for Elixir","archived":false,"fork":false,"pushed_at":"2023-06-18T01:32:27.000Z","size":143,"stargazers_count":516,"open_issues_count":20,"forks_count":23,"subscribers_count":10,"default_branch":"master","last_synced_at":"2025-05-10T15:51:52.014Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/alco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2014-07-10T03:51:55.000Z","updated_at":"2025-04-21T13:21:56.000Z","dependencies_parsed_at":"2024-05-01T15:27:41.174Z","dependency_job_id":"f75e292f-b7df-45a9-ac0a-5c408f81af84","html_url":"https://github.com/alco/benchfella","commit_stats":{"total_commits":124,"total_committers":11,"mean_commits":"11.272727272727273","dds":"0.14516129032258063","last_synced_commit":"60eaa41086f71c77bef916e6cbffca9805f84e84"},"previous_names":[],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alco%2Fbenchfella","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alco%2Fbenchfella/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alco%2Fbenchfella/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/alco%2Fbenchfella/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/alco","download_url":"https://codeload.github.com/alco/benchfella/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254355334,"owners_count":22057354,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T02:00:34.439Z","updated_at":"2025-05-15T14:05:12.991Z","avatar_url":"https://github.com/alco.png","language":"Elixir","readme":"Benchfella\n==========\n\n[![Module Version](https://img.shields.io/hexpm/v/benchfella.svg)](https://hex.pm/packages/benchfella)\n[![Hex Docs](https://img.shields.io/badge/hex-docs-lightgreen.svg)](https://hexdocs.pm/benchfella/)\n[![Total Download](https://img.shields.io/hexpm/dt/benchfella.svg)](https://hex.pm/packages/benchfella)\n[![License](https://img.shields.io/hexpm/l/benchfella.svg)](https://github.com/alco/benchfella/blob/master/LICENSE)\n[![Last Updated](https://img.shields.io/github/last-commit/alco/benchfella.svg)](https://github.com/alco/benchfella/commits/master)\n\nMicrobenchmarking tool for Elixir.\n\n## Overview\n\nWith Benchfella you can define small tests and it will intelligently run each individual one to\nobtain a more or less reliable estimate of the average run time for each test.\n\nThe key features of Benchfella:\n\n  * very easy to write and run microbenchmarks\n  * compare measurements between multiple runs\n  * plot results of multiple runs\n\nIf you are looking for a more elaborate treatment of the measurements, take a look at\n[bmark](https://github.com/joekain/bmark) which employs mathematical statistics to compare\nbenchmarking results and determine their credibility.\n\n\n## Installation\n\nAdd `:benchfella` as a dependency to your project:\n\n```elixir\n# in your mix.exs\n\ndefp deps do\n  [\n    {:benchfella, \"~\u003e 0.3.0\"}\n  ]\nend\n```\n\nThis will make the new tasks available in the root directory of your Mix project.\n\nAny project will do, so if you just want to measure a snippet of code quickly, create a bare-bones\nMix project with `mix new`, create a subdirectory called `bench` in it and put your benchmark\ndefinitions there. See examples below.\n\n## Usage\n\nTake a moment to study the output of running `mix help bench` and `mix help bench.cmp` inside your\nMix project to see all supported options.\n\nIn order to start writing tests, add a directory called `bench` and put files with names that match\nthe pattern `*_bench.exs` in it. Then run `mix bench` in the root directory of your project.\nBenchfella will then load each test and execute it for as many iterations as necessary so that the\ntotal running time is at least the specified duration.\n\nExample:\n\n```elixir\n# bench/basic_bench.exs\ndefmodule BasicBench do\n  use Benchfella\n\n  @list Enum.to_list(1..1000)\n\n  bench \"hello list\" do\n    Enum.reverse @list\n  end\nend\n```\n\n```sh\n$ mix bench\nSettings:\n  duration:      1.0 s\n\n## BasicBench\n[13:23:58] 0/1: hello list\n\nFinished in 3.15 seconds\n\n## BasicBench\nhello list      500000   5.14 µs/op\n```\n\nIn the end, the number of iterations and the average time of a single iteration are printed to the\nstandard output. Additionally, the output in machine format is written to a snapshot file in\n`bench/snapshots/`.\n\n\n### `setup_all` and `teardown_all`\n\n`setup_all/0` lets you perform some code before the first test in a module is run.\nIt takes no arguments and should return `{:ok, \u003ccontext\u003e}` where `\u003ccontext\u003e` is\nany term, it will be passed into `before_each_bench/1` and `teardown_all/1` if they are\ndefined. Returning any other value will raise an error and cause the whole\nmodule to be skipped.\n\n`teardown_all/1` lets you do some cleanup after the last test in a module has\nfinished running. It takes the context returned from `setup_all/0` (`nil` by\ndefault) as its argument.\n\n```elixir\n# bench/sys_bench.exs\ndefmodule SysBench do\n  use Benchfella\n\n  setup_all do\n    depth = :erlang.system_flag(:backtrace_depth, 100)\n    {:ok, depth}\n  end\n\n  teardown_all depth do\n    :erlang.system_flag(:backtrace_depth, depth)\n  end\n\n  @list Enum.to_list(1..10000)\n\n  bench \"list reverse\" do\n    Enum.reverse(@list)\n  end\nend\n```\n\n### `before_each_bench` and `after_each_bench`\n\n`before_each_bench/1` runs before each individual test is executed. It\ntakes the context returned from `setup_all/0` and should return `{:ok,\n\u003cbench_context\u003e}` where `\u003cbench_context\u003e` is any term. Returning any other value\nwill raise an error and cause the current test to be skipped.\n\n`\u003cbench_context\u003e` returned from `before_each_bench/1` will be available as the\n`bench_context` variable in each test.\n\n`after_each_bench/1` runs after each individual test has been executed. It\ntakes the context returned from `before_each_bench/1` as its argument.\n\n```elixir\n# bench/ets_bench.exs\ndefmodule ETSBench do\n  use Benchfella\n\n  before_each_bench _ do\n    tid = :ets.new(:my_table, [:public])\n    {:ok, tid}\n  end\n\n  after_each_bench tid do\n    IO.inspect length(:ets.tab2list(tid))\n    :ets.delete(tid)\n  end\n\n  bench \"ets insert\", [_unused: inspect_table(bench_context)] do\n    tid = bench_context\n    :ets.insert(tid, {:rand.uniform(1000), :x})\n    :ok\n  end\n\n  defp inspect_table(tid) do\n    IO.inspect :ets.info(tid)\n  end\nend\n```\n\n### Run time values\n\nWhen you need to generate inputs for tests at run time without affecting the measurements and you\ncan't use `before_each_bench/1` hook for that, the following trick can be used:\n\n```elixir\n# bench/string_bench.exs\ndefmodule StringBench do\n  use Benchfella\n\n  bench \"reverse string\", [str: gen_string()] do\n    String.reverse(str)\n  end\n\n  defp gen_string() do\n    String.duplicate(\"abc\", 10000)\n  end\nend\n```\n\n### `mix bench.cmp`\n\nTo compare results between multiple runs, use `mix bench.cmp`.\n\n```sh\n# Run 'mix bench' one more time.\n# Each run automatically saves a snapshot in bench/snapshots.\n$ mix bench\n...\n\n# 'mix bench.cmp' will read the two latest snapshots by default.\n# You could also pass the snapshot files to compare as arguments.\n$ mix bench.cmp -d percent\nbench/snapshots/2015-03-26_01-17-17.snapshot vs\nbench/snapshots/2015-03-26_01-19-30.snapshot\n\n## ListBench\nreverse list              -10.32%\n\n## StringBench\nreverse string dynamic    +2.26%\nreverse string            +3.33%\n```\n\n### `mix bench.graph`\n\nBenchfella can produce an HTML page with graphs providing various insights into\nthe raw data obtained from running `mix bench`.\n\n```sh\n# run the benchmarks twice\n$ mix bench\n...\n$ mix bench\n...\n\n# 'mix bench.graph' works similarly to 'mix bench.cmp' except it can display\n# all given snapshots on one graph.\n$ mix bench.graph\nWrote bench/graphs/index.html\n\n$ open bench/graphs/index.html\n```\n\n![Graph example](./assets/bench_graph.png \"Graph example\")\n\n## Copyright and License\n\nCopyright (c) 2014 Alexei Sholik\n\nThis software is licensed under [the MIT license](./LICENSE).\n","funding_links":[],"categories":["Benchmarking","Debugging"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falco%2Fbenchfella","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Falco%2Fbenchfella","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Falco%2Fbenchfella/lists"}