https://github.com/mlr-org/mlr3batchmark

Connector between mlr3 and batchtools
https://github.com/mlr-org/mlr3batchmark

batchtools cluster-computing high-performance-computing hpc mlr3 r r-package

Last synced: about 2 months ago
JSON representation

Connector between mlr3 and batchtools

• Host: GitHub
• URL: https://github.com/mlr-org/mlr3batchmark
• Owner: mlr-org
• Created: 2020-08-13T09:02:22.000Z (over 4 years ago)
• Default Branch: main
• Last Pushed: 2024-04-25T07:54:09.000Z (8 months ago)
• Last Synced: 2024-05-01T09:37:21.613Z (8 months ago)
• Topics: batchtools, cluster-computing, high-performance-computing, hpc, mlr3, r, r-package
• Language: R
• Homepage:
• Size: 122 KB
• Stars: 5
• Watchers: 6
• Forks: 3
• Open Issues: 4
• Metadata Files:
  • Readme: README.Rmd

Awesome Lists containing this project

README

        
---

output: github_document

---

# mlr3batchmark

[![r-cmd-check](https://github.com/mlr-org/mlr3batchmark/actions/workflows/r-cmd-check.yml/badge.svg)](https://github.com/mlr-org/mlr3batchmark/actions/workflows/r-cmd-check.yml)

[![CRAN status](https://www.r-pkg.org/badges/version/mlr3batchmark)](https://CRAN.R-project.org/package=mlr3batchmark)

[![StackOverflow](https://img.shields.io/badge/stackoverflow-mlr3-orange.svg)](https://stackoverflow.com/questions/tagged/mlr3)

[![Mattermost](https://img.shields.io/badge/chat-mattermost-orange.svg)](https://lmmisld-lmu-stats-slds.srv.mwn.de/mlr_invite/)

A connector between [mlr3](https://github.com/mlr-org/mlr3) and [batchtools](https://mllg.github.io/batchtools/).

This allows to run large-scale benchmark experiments on scheduled high-performance computing clusters.

The package comes with two core functions for switching between `mlr3` and `batchtools` to perform a benchmark:

* After creating a `design` object (as required for `mlr3`'s `benchmark()` function), instead of `benchmark()` call `batchmark()` which populates

  an `ExperimentRegistry` for the computational jobs of the benchmark.

  You are now in the world of `batchtools` where you can selectively submit jobs with different resources, monitor the progress or resubmit as needed.

* After the computations are finished, collect the results with `reduceResultsBatchmark()` to return to `mlr3`.

  The resulting object is a regular `BenchmarkResult`.

## Example

```{r}

library("mlr3")

library("batchtools")

library("mlr3batchmark")

tasks = tsks(c("iris", "sonar"))

learners = lrns(c("classif.featureless", "classif.rpart"))

resamplings = rsmp("cv", folds = 3)

design = benchmark_grid(

  tasks = tasks,

  learners = learners,

  resamplings = resamplings

)

reg = makeExperimentRegistry(NA)

ids = batchmark(design, reg = reg)

submitJobs()

getStatus()

reduceResultsBatchmark()

```

## Resources

* The *Large-Scale Benchmarking* chapter of the [mlr3 book](https://mlr3book.mlr-org.com/)