Ecosyste.ms: Awesome

An open API service indexing awesome lists of open source software.

Awesome Lists | Featured Topics | Projects

https://github.com/analythium/rconfig

Manage R Configuration at the Command Line
https://github.com/analythium/rconfig

cli config json r rstats yaml

Last synced: 3 months ago
JSON representation

Manage R Configuration at the Command Line

Awesome Lists containing this project

README 

        
---

output:

  md_document:

    variant: gfm

---



```{r, echo = FALSE}

knitr::opts_chunk$set(

  collapse = TRUE,

  comment = "#",

  fig.path = "README-"

)

set.seed(0)

knitr::opts_knit$set(root.dir = 'inst/examples')

```



# rconfig



> Manage R Configuration at the Command Line



[![Build status](https://github.com/analythium/rconfig/actions/workflows/check.yml/badge.svg)](https://github.com/analythium/rconfig/actions)

[![CRAN version](https://www.r-pkg.org/badges/version/rconfig)](https://CRAN.R-project.org/package=rconfig)

[![CRAN RStudio mirror downloads](https://cranlogs.r-pkg.org/badges/grand-total/rconfig)](https://hub.analythium.io/rconfig/)



Manage R configuration using files (YAML, JSON, INI, TXT)

JSON strings and command line arguments. Command line arguments

can be used to override configuration. Period-separated command line

flags are parsed as hierarchical lists. Environment variables, R global variables,

  and configuration values can be substituted.



_Try rconfig in your browser: click the Gitpod button, then `cd inst/examples` in the VS Code terminal to run the `Rscript` example from this README!_



[![Open in Gitpod](https://gitpod.io/button/open-in-gitpod.svg)](https://gitpod.io/#https://github.com/analythium/rconfig)



- [Install](#install)

- [Another config package?](#another-config-package)

- [Usage](#usage)

  - [R command line usage](#r-command-line-usage)

  - [Variable substitution](#variable-substitution)

  - [Using with Rscript](#using-with-rscript)

  - [Shiny](#shiny)

  - [Plumber](#plumber)

- [License](#license)



## Install



```{r eval=FALSE}

# CRAN version

install.packages("rconfig")



# Development version from R-universe

install.packages("rconfig", repos = "https://analythium.r-universe.dev")

```



## Another config package?



There are other R packages to manage configs:



- [config](https://rstudio.github.io/config/) has nice inheritance rules, and it even scans parent directories for YAML config files

- [configr](https://CRAN.R-project.org/package=configr) has nice substitution/interpolation features and supports YAML, JSON, TOML, and INI file formats



These package are fantastic if you are managing deployments at different stages of the life cycle, i.e. testing/staging/production.



However, when you use `Rscript` from the command line, you often do not want to manage too many configuration files, but want a quick way to override some of the default settings.



The rconfig package provides various ways to override defaults, and instead of changing the active configuration (as in the config package), you can merge lists in order to arrive at a final configuration. These are very similar concepts, but not quite the same.



The rconfig package has the following features:



- uses default configuration file

- file based override with the `-f` or `--file` flags (accepts JSON, YAML, INI, and plain text files)

- JSON string based override with the `-j` or `--json` flags

- other command line arguments are merged too, e.g. `--cores 4`

- heuristic rules are used to coerce command line values to the right type

- R expressions starting with `!expr` are evaluated by default, this behavior can be turned off (same feature can be found in the yaml and config packages, but here it works with plain text and JSON too)

- period-separated command line arguments are parsed as hierarchical lists, e.g. `--user.name Joe` will be added as `user$name` to the config list

- nested configurations can also be flattened 

- command line flags without a value will evaluate to `TRUE`, e.g. `--verbose`

- environment variables (`${VALUE}`), R global variables (`@{VALUE}`), and configuration values (`#{VALUE}`) can be substituted

- differentiates verb/noun syntax, where verbs are sub-commands following the R script file name and preceding the command line flags (starting with `-` or `--`)



This looks very similar to what [littler](https://CRAN.R-project.org/package=littler), [getopt](https://CRAN.R-project.org/package=getopt), and [optparse](https://CRAN.R-project.org/package=optparse) are supposed to do. You are right. These packages offer amazing command line experience once you have a solid interface. In an iterative and evolving research and development situation, however, rconfig gives you agility.



Moreover, the rconfig package offers various ways for substituting environment variables, R global variables, and even substituting configuration values. The [GetoptLong](https://CRAN.R-project.org/package=GetoptLong/vignettes/variable_interpolation.html) package has similar functionality but its focus is on command line interfaces and not configuration. Other tools, such as `sprintf`, [glue](https://CRAN.R-project.org/package=glue), [rprintf](https://CRAN.R-project.org/package=rprintf), and [whiskers](https://CRAN.R-project.org/package=whisker) are aimed at substituting values from R expressions.



If you are not yet convinced, here is a quick teaser.

This is the content of the default configuration file, `rconfig.yml`:



```{verbatim lang="yaml", file="rconfig.yml"}

```



Let's use a simple R script to print out the configs:



```{verbatim lang="r", file="test.R"}

```



Now you can override the default configuration using another file, a JSON string, and some other flags. Notice the variable substitution for user name!



```{bash}

export USER=Jane



Rscript --vanilla test.R deploy \

  -f rconfig-prod.yml \

  -j '{"trials":30,"dataset":"full-data.csv"}' \

  --user.name $USER \

  --verbose

```



The package was inspired by the config package, docker-compose/kubectl/caddy and other CLI tools, and was motivated by some real world need when managing background processing on cloud instances.



## Usage



### R command line usage



Open the project in RStudio or set the work directory to the folder root after cloning/downloading the repository.



```{r}

str(rconfig::rconfig())



str(rconfig::rconfig(

    file = "rconfig-prod.yml"))



str(rconfig::rconfig(

    file = c("rconfig.json",

             "rconfig-prod.txt"),

    list = list(user = list(name = "Jack"))))



str(rconfig::rconfig(

    file = c("rconfig.json",

             "rconfig-prod.txt"),

    list = list(user = list(name = "Jack")),

    flatten = TRUE))

```



Set defaults in case some values are undefined:



```{r}

CONFIG <- rconfig::rconfig(

    file = "rconfig-prod.yml")



rconfig::value(CONFIG$cores, 2L)   # set to 1L

rconfig::value(CONFIG$test)        # unset

rconfig::value(CONFIG$test, FALSE) # use default

```



The default values are used to ensure type safety:



```{r}

str(rconfig::value(CONFIG$trials, 0L))    # integer

str(rconfig::value(CONFIG$trials, 0))     # numeric

str(rconfig::value(CONFIG$trials, "0"))   # character

str(rconfig::value(CONFIG$trials, FALSE)) # logical

```



Using alongside of the config package:



```{r}

conf <- config::get(

    config = "production",

    file = "config.yml",

    use_parent = FALSE)



str(rconfig::rconfig(

    file = "rconfig.yml",

    list = conf))

```



### Variable substitution



The rconfig package interprets 3 kinds of substitution patterns:



- environment variables (`${VALUE}`): these variables are already present when the configurations is read from the calling environment or from `.Renviron` file in the project specific or home folder, set variables can be null or not-null

- R global variables (`@{VALUE}`): the rconfig package looks for variables in the global environment at the time of configuration evaluation, however, expressions are not evaluated (unlike the `!expr` option for values)

- configuration values (`#{VALUE}`): the configuration level variables are evaluated last, thus these values can refer to existing keys that are already substituted



The substitution pattern can set defaults or error messages, following [bash](https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02) and [Docker](https://docs.docker.com/compose/environment-variables/) conventions:



|              | `a` is set & not null            | `a` set but null                |  `a` is unset         |

|--------------|--------------------------------|-------------------------------|---------------------|

| `${a:-b}`     | substitute `a`                   | substitute `b`                  |  substitute `b`       |

| `${a-b}`      | substitute `a`                   | substitute null               |  substitute `b`       |

| `${a:?b}`     | substitute `a`                   | error, exit                   |  error, exit        |

| `${a?b}`      | substitute `a`                   | substitute null               |  error, exit        |



The following YAML example has all 3 kinds of variable substitution pattern:



```{verbatim lang="yaml", file="rconfig-sub.yml"}

```



Set the following variables:

```{r}

Sys.setenv(USER="Adele")

Sys.setenv(ACCESS="admin")

Type <- "simple"

Lang <- "HU"

```



This is the substituted version:



```{r echo=FALSE, results="asis"}

# variables only looked for in global env

Type <<- "simple"

Lang <<- "HU"

Sys.setenv(R_RCONFIG_FILE = "rconfig-sub.yml")

# x <- rconfig::rconfig(file = "rconfig-sub.yml")

x <- rconfig::rconfig()

Sys.unsetenv("R_RCONFIG_FILE")

cat("```yaml\n", yaml::as.yaml(x), "```", sep = "")

```



### Using with Rscript



Set the work directory to the `inst/examples` folder cloning/downloading the repo.



Default config if found (the script has debug mode on):



```{bash}

Rscript test.R

```



Default with debug mode off:



```{bash}

R_RCONFIG_DEBUG="FALSE" Rscript test.R

```



Change default config file:



```{bash}

R_RCONFIG_FILE="rconfig-prod.yml" Rscript test.R

```



Change default config file and debug off:



```{bash}

R_RCONFIG_FILE="rconfig-prod.yml" R_RCONFIG_DEBUG="FALSE" Rscript test.R

```



Use file and other props to override default:



```{bash}

Rscript test.R -f rconfig-prod.yml --user.name "unreal_Zh5z*$#="

```



Use JSON string and other props to override default:



```{bash}

Rscript test.R \

  -j '{"trials":30,"dataset":"full-data.csv","user":{"name": "real_We4$#z*="}}' \

  --user.name "unreal_Zh5z*$#="

```



rconfig also interprets verb/noun syntax, where

verbs are sub-commands following the R script file name

and preceding the command line flags (starting with `-` or `--`):



```{bash}

Rscript test.R deploy --user.name "unreal_Zh5z*$#="

```



For a more realistic but still small example, let's use the `iris` data and get summaries by species using command line arguments:



```{bash}

Rscript iris.R --species virginica

```



```{bash}

Rscript iris.R --species setosa --verbose

```



```{bash error=TRUE}

Rscript iris.R --species maxima --verbose

```



```{bash error=TRUE}

Rscript iris.R

```



Check out the [`iris_rlog.R`](inst/examples/iris_rlog.R) file to see an example with proper logging.



Another illustration using the `mtcars` data set to fit linear models to different variables:



```{bash}

Rscript mtcars.R

```



```{bash}

Rscript mtcars.R --verbose --vars cyl

```



```{bash error=TRUE}

Rscript mtcars.R --verbose --vars cal

```



```{bash}

Rscript mtcars.R --vars cyl disp hp

```



Let's see how to use sub-commands:



```{bash}

## This will print messages:

Rscript commands.R model



## This will not print messages:

Rscript commands.R model --silent

```



```{bash}

Rscript commands.R predict

```



```{bash error=TRUE}

Rscript commands.R fit

```



```{bash error=TRUE}

Rscript commands.R

```



Here is how to make the R script executable on Linux (of course you'll need rconfig installed for this to work):



```{bash eval=FALSE}

sudo cp ./inst/examples/commands.R /usr/local/bin/

sudo chmod +x /usr/local/bin/commands.R

```



Make sure that the R script has the shebang (`#!/usr/bin/env Rscript`) as the 1st line, and now can drop the `Rscript` part and use the script as `commands.R model`.



### Shiny



An example to configure a [Shiny app](https://shiny.posit.co/) with command line flags:



```bash

Rscript shiny/app.R



Rscript shiny/app.R \

  --test \

  --value 1000 \

  --color 'pink' \

  --title 'Only Testing'

```



An example to configure a Shiny app using the [golem](https://golemverse.org/) with command line flags:



```r

# app.R

CONFIG <- rconfig()

yourpkg::run_app(

  title = value(CONFIG$title, "Hello Shiny!"),

  test = value(CONFIG$test, FALSE),

  color = value(CONFIG$color, "purple"),

  options = list(port = value(CONFIG$port, 8080)))

```



```bash

## then in terminal

Rscript app.R \

  --test \

  --value 1000 \

  --color 'pink' \

  --title 'Only Testing' \

  --port 3838

```



### Plumber



An example to configure a [Plumber API](https://www.rplumber.io/) with command line flags:



```bash

cd plumber



Rscript index.R



# httr::POST("http://127.0.0.1:8080/echo?msg=Cool") |> httr::content()

# httr::GET("http://127.0.0.1:8080/test") |> httr::content()



Rscript index.R \

  --test \

  --port 8000 \

  --title 'The echoed message is'



# httr::POST("http://127.0.0.1:8000/echo?msg=Cool") |> httr::content()

# httr::GET("http://127.0.0.1:8000/test") |> httr::content()

```



## License



[MIT License](./LICENSE)

© 2022 Peter Solymos and Analythium Solutions Inc.