Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/mlr-org/mlr3tuningspaces
Collection of search spaces for hyperparameter optimization in the mlr3 ecosystem
https://github.com/mlr-org/mlr3tuningspaces
automl hyperparameter-tuning machine-learning mlr3 r r-package tune tuning
Last synced: 2 days ago
JSON representation
Collection of search spaces for hyperparameter optimization in the mlr3 ecosystem
- Host: GitHub
- URL: https://github.com/mlr-org/mlr3tuningspaces
- Owner: mlr-org
- Created: 2021-05-03T11:21:09.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2024-08-18T15:56:08.000Z (3 months ago)
- Last Synced: 2024-10-28T17:24:01.097Z (11 days ago)
- Topics: automl, hyperparameter-tuning, machine-learning, mlr3, r, r-package, tune, tuning
- Language: R
- Homepage: https://mlr3tuningspaces.mlr-org.com
- Size: 5.84 MB
- Stars: 13
- Watchers: 7
- Forks: 3
- Open Issues: 5
-
Metadata Files:
- Readme: README.Rmd
- Changelog: NEWS.md
Awesome Lists containing this project
README
---
output: github_document
bibliography: references.bib
---
```{r, include = FALSE}
library(mlr3misc)
library(utils)
library(mlr3tuningspaces)
library(data.table)
source("R/bibentries.R")
writeLines(toBibtex(bibentries), "references.bib")
lgr::get_logger("mlr3")$set_threshold("warn")
lgr::get_logger("bbotk")$set_threshold("warn")
set.seed(1)
options(
datatable.print.nrows = 10,
datatable.print.class = FALSE,
datatable.print.keys = FALSE,
width = 100)
```
# mlr3tuningspaces 
Package website: [release](https://mlr3tuningspaces.mlr-org.com/) | [dev](https://mlr3tuningspaces.mlr-org.com/dev/)
[](https://github.com/mlr-org/mlr3tuningspaces/actions/workflows/r-cmd-check.yml)
[](https://cran.r-project.org/package=mlr3tuningspaces)
[](https://stackoverflow.com/questions/tagged/mlr3)
[](https://lmmisld-lmu-stats-slds.srv.mwn.de/mlr_invite/)
*mlr3tuningspaces* is a collection of search spaces for hyperparameter optimization in the [mlr3](https://github.com/mlr-org/mlr3/) ecosystem.
It features ready-to-use search spaces for many popular machine learning algorithms.
The search spaces are from scientific articles and work for a wide range of data sets.
Currently, we offer tuning spaces from three publications.
| Publication | Learner | n Hyperparameter |
| ------------ | ------- | ---------------- |
| @bischl_2021 | glmnet | 2 |
| | kknn | 3 |
| | ranger | 4 |
| | rpart | 3 |
| | svm | 4 |
| | xgboost | 8 |
| @kuehn_2018 | glmnet | 2 |
| | kknn | 1 |
| | ranger | 8 |
| | rpart | 4 |
| | svm | 5 |
| | xgboost | 13 |
| @binder_2020 | glmnet | 2 |
| | kknn | 1 |
| | ranger | 6 |
| | rpart | 4 |
| | svm | 4 |
| | xgboost | 10 |
## Resources
There are several sections about hyperparameter optimization in the [mlr3book](https://mlr3book.mlr-org.com).
* Getting started with the [book](https://mlr3book.mlr-org.com/chapters/chapter4/hyperparameter_optimization.html#sec-tuning-spaces) section on mlr3tuningspaces.
* Learn about [search space](https://mlr3book.mlr-org.com/chapters/chapter4/hyperparameter_optimization.html#sec-learner-search-space).
The [gallery](https://mlr-org.com/gallery-all-optimization.html) features a collection of case studies and demos about optimization.
* [Tune](https://mlr-org.com/gallery/optimization/2021-07-06-introduction-to-mlr3tuningspaces/) a classification tree with the default tuning space from @bischl_2021.
## Installation
Install the last release from CRAN:
```{r eval = FALSE}
install.packages("mlr3tuningspaces")
```
Install the development version from GitHub:
```{r eval = FALSE}
remotes::install_github("mlr-org/mlr3tuningspaces")
```
## Example
### Quick Tuning
A learner passed to the `lts()` function arguments the learner with the default tuning space from @bischl_2021.
```{r}
library(mlr3tuningspaces)
learner = lts(lrn("classif.rpart"))
# tune learner on pima data set
instance = tune(
tnr("random_search"),
task = tsk("pima"),
learner = learner,
resampling = rsmp("holdout"),
measure = msr("classif.ce"),
term_evals = 10
)
# best performing hyperparameter configuration
instance$result
```
### Tuning Search Spaces
The `mlr_tuning_spaces` dictionary contains all tuning spaces.
```{r, eval=FALSE}
library("data.table")
# print keys and tuning spaces
as.data.table(mlr_tuning_spaces)
```
A key passed to the `lts()` function returns the `TuningSpace`.
```{r}
tuning_space = lts("classif.rpart.rbv2")
tuning_space
```
Get the learner with tuning space.
```{r}
tuning_space$get_learner()
```
### Pipelines
Tuning spaces can be applied to the learners in a pipeline.
```{r}
library(mlr3pipelines)
# set default tuning space
graph_learner = as_learner(po("subsample") %>>%
lts(lrn("classif.rpart")))
# set rbv2 tuning space
tuning_space = lts("classif.rpart.rbv2")
graph_learner$graph$pipeops$classif.rpart$param_set$set_values(.values = tuning_space$values)
```
### Adding New Tuning Spaces
We are looking forward to new collections of tuning spaces from peer-reviewed articles.
You can suggest new tuning spaces in an issue or contribute a new collection yourself in a pull request.
Take a look at an already implemented collection e.g. our [default tuning spaces](https://github.com/mlr-org/mlr3tuningspaces/blob/main/R/tuning_spaces_default.R) from @bischl_2021.
A `TuningSpace` is added to the ` mlr_tuning_spaces` dictionary with the `add_tuning_space()` function.
Create a tuning space for each variant of the learner e.g. for `LearnerClassifRpart` and `LearnerRegrRpart`.
```{r, eval=FALSE}
vals = list(
minsplit = to_tune(2, 64, logscale = TRUE),
cp = to_tune(1e-04, 1e-1, logscale = TRUE)
)
add_tuning_space(
id = "classif.rpart.example",
values = vals,
tags = c("default", "classification"),
learner = "classif.rpart",
label = "Classification Tree Example"
)
```
Choose a name that is related to the publication and adjust the documentation.
The reference is added to the `bibentries.R` file
```{r, eval=FALSE}
bischl_2021 = bibentry("misc",
key = "bischl_2021",
title = "Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges",
author = "Bernd Bischl and Martin Binder and Michel Lang and Tobias Pielok and Jakob Richter and Stefan Coors and Janek Thomas and Theresa Ullmann and Marc Becker and Anne-Laure Boulesteix and Difan Deng and Marius Lindauer",
year = "2021",
eprint = "2107.05847",
archivePrefix = "arXiv",
primaryClass = "stat.ML",
url = "https://arxiv.org/abs/2107.05847"
)
```
We are happy to help you with the pull request if you have any questions.
## References