https://github.com/mrc-ide/rrq

:runner::runner::runner: Lightweight Redis queues
https://github.com/mrc-ide/rrq

cluster infrastructure

Last synced: about 1 month ago
JSON representation

:runner::runner::runner: Lightweight Redis queues

• Host: GitHub
• URL: https://github.com/mrc-ide/rrq
• Owner: mrc-ide
• License: other
• Created: 2016-06-01T11:32:35.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2024-10-18T12:43:50.000Z (about 2 months ago)
• Last Synced: 2024-11-02T16:08:20.784Z (about 1 month ago)
• Topics: cluster, infrastructure
• Language: R
• Homepage: https://mrc-ide.github.io/rrq/
• Size: 3.91 MB
• Stars: 24
• Watchers: 6
• Forks: 4
• Open Issues: 7
• Metadata Files:
  • Readme: README.Rmd
  • Changelog: NEWS.md
  • License: LICENSE

Awesome Lists containing this project

• jimsghstars - mrc-ide/rrq - :runner::runner::runner: Lightweight Redis queues (R)

README

        


```{r, include = FALSE}

knitr::opts_chunk$set(

  collapse = TRUE,

  error = FALSE,

  comment = "#>",

  fig.path = "man/figures/README-",

  out.width = "100%"

)

```

# rrq

[![Project Status: WIP - Initial development is in progress, but there has not yet been a stable, usable release suitable for the public.](https://www.repostatus.org/badges/latest/wip.svg)](https://www.repostatus.org/#wip)

[![R-CMD-check](https://github.com/mrc-ide/rrq/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/mrc-ide/rrq/actions/workflows/R-CMD-check.yaml)

[![codecov.io](https://codecov.io/github/mrc-ide/rrq/coverage.svg?branch=master)](https://codecov.io/github/mrc-ide/rrq?branch=master)

[![CodeFactor](https://www.codefactor.io/repository/github/mrc-ide/rrq/badge)](https://www.codefactor.io/repository/github/mrc-ide/rrq)

Task queues for R, implemented using Redis.

## Getting started

```{r}

library(rrq)

```

Create an `rrq_controller` object

```{r}

obj <- rrq_controller("rrq:readme")

rrq_default_controller_set(obj)

```

Submit work to the queue:

```{r}

t <- rrq_task_create_expr(runif(10))

t

```

Query task process:

```{r}

rrq_task_status(t)

```

Run tasks on workers in the background

```{r}

rrq_worker_spawn()

```

Wait for tasks to complete

```{r}

rrq_task_wait(t)

```

Retrieve results from a task

```{r}

rrq_task_result(t)

```

Query what workers have done

```{r}

rrq_worker_log_tail(n = Inf)

```

For more information, see `vignette("rrq")`

```{r, include = FALSE}

rrq_destroy(timeout_worker_stop = 10)

```

## Installation

Install from the mrc-ide R universe package repository:

```r

install.packages(

  "rrq",

  repos = c("https://mrc-ide.r-universe.dev", "https://cloud.r-project.org"))

```

Alternatively, install with `remotes`:

```r

remotes::install_github("mrc-ide/rrq", upgrade = FALSE)

```

## Testing

To test, we need a redis server that can be automatically connected to using the `redux` defaults.  This is satisfied if you have an unauthenticated redis server running on localhost, otherwise you should update the environment variable `REDIS_URL` to point at a redis server.  Do not use a production server, as the package will create and delete a lot of keys.

A suitable redis server can be started using docker with

```

./scripts/redis start

```

(and stopped with `./scripts/redis stop`)

Alternatively, on Windows, a native (i.e., not depending on CygWin, MSys, or WSL) port of Redis 5.0.14.1 can be installed from [here](https://github.com/tporadowski/redis/releases), and will run out of the box.

## Testing the pkgdown site with examples

The documentation includes many executed code examples. To preview the documentation with the results of these, you'll need a Redis server running, and the rrq package installed. Then, in an R terminal:-

```

rrq::rrq_worker$new("rrq:example")$loop()

```

to run a worker. Then in another R terminal,

```

pkgdown::build_site()

```

and the pkgdown site will be built in the `docs/` folder.

## License

MIT © Imperial College of Science, Technology and Medicine