Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/awadell1/pkgjogger.jl
Take your packages for a jog!
https://github.com/awadell1/pkgjogger.jl
benchmark benchmark-framework benchmarking github-actions julia julia-language
Last synced: 5 days ago
JSON representation
Take your packages for a jog!
- Host: GitHub
- URL: https://github.com/awadell1/pkgjogger.jl
- Owner: awadell1
- License: gpl-3.0
- Created: 2021-08-11T01:15:26.000Z (about 3 years ago)
- Default Branch: main
- Last Pushed: 2024-07-16T20:52:52.000Z (4 months ago)
- Last Synced: 2024-10-19T03:25:04.525Z (19 days ago)
- Topics: benchmark, benchmark-framework, benchmarking, github-actions, julia, julia-language
- Language: Julia
- Homepage: https://awadell1.github.io/PkgJogger.jl/stable/
- Size: 908 KB
- Stars: 23
- Watchers: 2
- Forks: 0
- Open Issues: 10
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# PkgJogger
[](https://awadell1.github.io/PkgJogger.jl/stable)
[](https://awadell1.github.io/PkgJogger.jl/dev)
[](https://github.com/awadell1/PkgJogger.jl/actions)
[](https://coveralls.io/github/awadell1/PkgJogger.jl?branch=main)
[](https://juliahub.com/ui/Packages/PkgJogger/AaLEJ)
[](https://juliaci.github.io/NanosoldierReports/pkgeval_badges/report.html)
[](https://juliapkgstats.com/pkg/PkgJogger)
PkgJogger provides a framework for running suites of
[BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl) benchmarks
without the boilerplate.
## Just write benchmarks
Create a `benchmark/bench_*.jl` file, define a
[BenchmarkTools.jl](https://github.com/JuliaCI/BenchmarkTools.jl) `suite` and
go!
```julia
using BenchmarkTools
using AwesomePkg
suite = BenchmarkGroup()
suite["fast"] = @benchmarkable fast_code()
```
PkgJogger will wrap each `benchmark/bench_*.jl` in a module and bundle them into `JogAwesomePkg`
```julia
using AwesomePkg
using PkgJogger
# Creates the JogAwesomePkg module
@jog AwesomePkg
# Warmup, tune, and run all of AwesomePkg's benchmarks
JogAwesomePkg.benchmark()
```
## Benchmark, Revise, and Benchmark Again!
PkgJogger uses [Revise.jl](https://github.com/timholy/Revise.jl) to track
changes to your `benchmark/bench_*.jl` files and reload your suite as you edit.
No more waiting for benchmarks to precompile!
Tracked Changes:
- Changing your benchmarked function
- Changing benchmarking parameters (i.e. `seconds` or `samples`)
- Adding new benchmarks
Current Limitations:
- New benchmark files are not tracked
- Deleted benchmarks will stick around
- Renamed benchmarks will create a new benchmark and retain the old name
To get around the above, run `@jog PkgName` to get an updated jogger.
> Note: Revise must be loaded first in order to track changes to your benchmark files.
## Continuous Benchmarking Baked In!
Install PkgJogger, run benchmarks, and save results to a `*.bson.gz` with a
one-line command.
```shell
julia -e 'using Pkg; Pkg.add("PkgJogger"); using PkgJogger; PkgJogger.ci()'
```
What gets done:
- Constructs a temporary
[benchmarking environment](https://awadell1.github.io/PkgJogger.jl/stable/ci/#Isolated-Benchmarking-Environment)
from `Project.toml` and `benchmark/Project.toml`.
- Creates a [jogger](https://awadell1.github.io/PkgJogger.jl/stable/jogger/)
to run the package's benchmarks.
- Warmup, tune and run all benchmarks.
- Save Benchmarking results and more to a compressed `*.bson.gz` file.
Or for a more lightweight option, use
[`@test_bechmarks`](https://awadell1.github.io/PkgJogger.jl/stable/ci/#Testing-Benchmarks)
to run each benchmark once (No Warmup, tuning, etc.), as a smoke test
against benchmarking regressions.