Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/alco/benchfella
Microbenchmarking tool for Elixir
https://github.com/alco/benchfella
Last synced: about 19 hours ago
JSON representation
Microbenchmarking tool for Elixir
- Host: GitHub
- URL: https://github.com/alco/benchfella
- Owner: alco
- License: mit
- Created: 2014-07-10T03:51:55.000Z (over 10 years ago)
- Default Branch: master
- Last Pushed: 2023-06-18T01:32:27.000Z (over 1 year ago)
- Last Synced: 2025-01-18T03:04:15.367Z (8 days ago)
- Language: Elixir
- Homepage:
- Size: 140 KB
- Stars: 513
- Watchers: 11
- Forks: 22
- Open Issues: 20
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
- freaking_awesome_elixir - Elixir - Benchmarking tool for Elixir. (Benchmarking)
- fucking-awesome-elixir - benchfella - Benchmarking tool for Elixir. (Benchmarking)
- awesome-elixir - benchfella - Benchmarking tool for Elixir. (Benchmarking)
README
Benchfella
==========
[](https://hex.pm/packages/benchfella)
[](https://hexdocs.pm/benchfella/)
[](https://hex.pm/packages/benchfella)
[](https://github.com/alco/benchfella/blob/master/LICENSE)
[](https://github.com/alco/benchfella/commits/master)
Microbenchmarking tool for Elixir.
## Overview
With Benchfella you can define small tests and it will intelligently run each individual one to
obtain a more or less reliable estimate of the average run time for each test.
The key features of Benchfella:
* very easy to write and run microbenchmarks
* compare measurements between multiple runs
* plot results of multiple runs
If you are looking for a more elaborate treatment of the measurements, take a look at
[bmark](https://github.com/joekain/bmark) which employs mathematical statistics to compare
benchmarking results and determine their credibility.
## Installation
Add `:benchfella` as a dependency to your project:
```elixir
# in your mix.exs
defp deps do
[
{:benchfella, "~> 0.3.0"}
]
end
```
This will make the new tasks available in the root directory of your Mix project.
Any project will do, so if you just want to measure a snippet of code quickly, create a bare-bones
Mix project with `mix new`, create a subdirectory called `bench` in it and put your benchmark
definitions there. See examples below.
## Usage
Take a moment to study the output of running `mix help bench` and `mix help bench.cmp` inside your
Mix project to see all supported options.
In order to start writing tests, add a directory called `bench` and put files with names that match
the pattern `*_bench.exs` in it. Then run `mix bench` in the root directory of your project.
Benchfella will then load each test and execute it for as many iterations as necessary so that the
total running time is at least the specified duration.
Example:
```elixir
# bench/basic_bench.exs
defmodule BasicBench do
use Benchfella
@list Enum.to_list(1..1000)
bench "hello list" do
Enum.reverse @list
end
end
```
```sh
$ mix bench
Settings:
duration: 1.0 s
## BasicBench
[13:23:58] 0/1: hello list
Finished in 3.15 seconds
## BasicBench
hello list 500000 5.14 µs/op
```
In the end, the number of iterations and the average time of a single iteration are printed to the
standard output. Additionally, the output in machine format is written to a snapshot file in
`bench/snapshots/`.
### `setup_all` and `teardown_all`
`setup_all/0` lets you perform some code before the first test in a module is run.
It takes no arguments and should return `{:ok, }` where `` is
any term, it will be passed into `before_each_bench/1` and `teardown_all/1` if they are
defined. Returning any other value will raise an error and cause the whole
module to be skipped.
`teardown_all/1` lets you do some cleanup after the last test in a module has
finished running. It takes the context returned from `setup_all/0` (`nil` by
default) as its argument.
```elixir
# bench/sys_bench.exs
defmodule SysBench do
use Benchfella
setup_all do
depth = :erlang.system_flag(:backtrace_depth, 100)
{:ok, depth}
end
teardown_all depth do
:erlang.system_flag(:backtrace_depth, depth)
end
@list Enum.to_list(1..10000)
bench "list reverse" do
Enum.reverse(@list)
end
end
```
### `before_each_bench` and `after_each_bench`
`before_each_bench/1` runs before each individual test is executed. It
takes the context returned from `setup_all/0` and should return `{:ok,
}` where `` is any term. Returning any other value
will raise an error and cause the current test to be skipped.
`` returned from `before_each_bench/1` will be available as the
`bench_context` variable in each test.
`after_each_bench/1` runs after each individual test has been executed. It
takes the context returned from `before_each_bench/1` as its argument.
```elixir
# bench/ets_bench.exs
defmodule ETSBench do
use Benchfella
before_each_bench _ do
tid = :ets.new(:my_table, [:public])
{:ok, tid}
end
after_each_bench tid do
IO.inspect length(:ets.tab2list(tid))
:ets.delete(tid)
end
bench "ets insert", [_unused: inspect_table(bench_context)] do
tid = bench_context
:ets.insert(tid, {:rand.uniform(1000), :x})
:ok
end
defp inspect_table(tid) do
IO.inspect :ets.info(tid)
end
end
```
### Run time values
When you need to generate inputs for tests at run time without affecting the measurements and you
can't use `before_each_bench/1` hook for that, the following trick can be used:
```elixir
# bench/string_bench.exs
defmodule StringBench do
use Benchfella
bench "reverse string", [str: gen_string()] do
String.reverse(str)
end
defp gen_string() do
String.duplicate("abc", 10000)
end
end
```
### `mix bench.cmp`
To compare results between multiple runs, use `mix bench.cmp`.
```sh
# Run 'mix bench' one more time.
# Each run automatically saves a snapshot in bench/snapshots.
$ mix bench
...
# 'mix bench.cmp' will read the two latest snapshots by default.
# You could also pass the snapshot files to compare as arguments.
$ mix bench.cmp -d percent
bench/snapshots/2015-03-26_01-17-17.snapshot vs
bench/snapshots/2015-03-26_01-19-30.snapshot
## ListBench
reverse list -10.32%
## StringBench
reverse string dynamic +2.26%
reverse string +3.33%
```
### `mix bench.graph`
Benchfella can produce an HTML page with graphs providing various insights into
the raw data obtained from running `mix bench`.
```sh
# run the benchmarks twice
$ mix bench
...
$ mix bench
...
# 'mix bench.graph' works similarly to 'mix bench.cmp' except it can display
# all given snapshots on one graph.
$ mix bench.graph
Wrote bench/graphs/index.html
$ open bench/graphs/index.html
```
## Copyright and License
Copyright (c) 2014 Alexei Sholik
This software is licensed under [the MIT license](./LICENSE).