https://github.com/tkf/benchmarkconfigsweeps.jl

https://github.com/tkf/benchmarkconfigsweeps.jl

Last synced: 11 months ago
JSON representation

• Host: GitHub
• URL: https://github.com/tkf/benchmarkconfigsweeps.jl
• Owner: tkf
• License: mit
• Created: 2021-10-19T21:01:11.000Z (about 4 years ago)
• Default Branch: master
• Last Pushed: 2021-12-07T22:55:25.000Z (about 4 years ago)
• Last Synced: 2025-01-20T16:56:23.841Z (12 months ago)
• Language: Julia
• Size: 24.4 KB
• Stars: 2
• Watchers: 3
• Forks: 0
• Open Issues: 0
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

README

          
# BenchmarkConfigSweeps

BenchmarkConfigSweeps.jl can be used for running benchmarks over various

`--nthreads` settings, environment variables, and `julia` versions.

Each configuration is specified as a combination of configuration specifiers

such as `BenchmarkConfigSweeps.nthreads`, `BenchmarkConfigSweeps.env`, and

`BenchmarkConfigSweeps.julia`.  The set of configurations to run benchmarks is

simply an iterable of (tuples of) such configuration specifiers:

```JULIA

using BenchmarkConfigSweeps

nthreads_list = [1, 4, 8, 16]

configs = Iterators.product(

    zip(

        BenchmarkConfigSweeps.nthreads.(nthreads_list),

        BenchmarkConfigSweeps.env.("OPENBLAS_NUM_THREADS" .=> nthreads_list),

    ),

    BenchmarkConfigSweeps.env.(

        "JULIA_PROJECT" .=> ["baseline", "target"],

        "JULIA_LOAD_PATH" => "@",  # not varied

    ),

    BenchmarkConfigSweeps.julia.(["julia1.5", "julia1.6"]),

)

```

It can be then passed to `BenchmarkConfigSweeps.run`:

```JULIA

BenchmarkConfigSweeps.run("build", "benchmarks.jl", configs)

```

where `"build"` is the directory to store the result and `"benchmarks.jl"` is

the Julia script that defines the variable `SUITE :: BenchmarkGroup` at the

top-level.

The result can be loaded using

```JULIA

sweepresult = BenchmarkConfigSweeps.load("build")

```

The `sweepresult` object implements the Tables.jl interface. For example, it can

be easily converted into a `DataFrame` by

```JULIA

using DataFrames

df = DataFrame(sweepresult)

```

Note that the default table conversion is rather too DWIM-y in that it tries to

guess the meaning of `BenchmarkGroup` keys.  For more information, see

`BenchmarkConfigSweeps.flattable` for how it works.  Use

`BenchmarkConfigSweeps.simpletable` for programmatic processing.