Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/r-lib/coro
Coroutines for R
https://github.com/r-lib/coro
async coroutines generator iterator promises r reticulate
Last synced: about 8 hours ago
JSON representation
Coroutines for R
- Host: GitHub
- URL: https://github.com/r-lib/coro
- Owner: r-lib
- License: other
- Created: 2017-09-27T08:37:42.000Z (about 7 years ago)
- Default Branch: main
- Last Pushed: 2024-03-11T11:52:47.000Z (8 months ago)
- Last Synced: 2024-06-11T17:10:03.161Z (5 months ago)
- Topics: async, coroutines, generator, iterator, promises, r, reticulate
- Language: R
- Homepage: https://coro.r-lib.org/
- Size: 5.31 MB
- Stars: 151
- Watchers: 7
- Forks: 7
- Open Issues: 6
-
Metadata Files:
- Readme: README.Rmd
- License: LICENSE
- Code of conduct: .github/CODE_OF_CONDUCT.md
Awesome Lists containing this project
- jimsghstars - r-lib/coro - Coroutines for R (R)
README
---
output: github_document
always_allow_html: yes
---
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README",
out.width = "100%"
)
```
# coro
[](https://cran.r-project.org/package=coro)
[](https://github.com/r-lib/coro/actions/workflows/R-CMD-check.yaml)
## Overview
coro implements __coroutines__ for R, i.e. functions that can be suspended and resumed later on. There are two kinds:
- __Async__ functions, which make it straightforward to program concurrently
- __Generators__ for iterating over complex sequences
Supported features:
- Suspending within loops and if/else branches
- Suspending within `tryCatch()`
- `on.exit()` expressions and stack-based cleanup such as provided by `local_` functions in the [withr](https://github.com/r-lib/withr/) package
- Step-debugging and `browser()` within coroutines
Compatibility with:
- Python iterators from the [reticulate](https://rstudio.github.io/reticulate/) package
- Async operations from the [promises](https://github.com/rstudio/promises/) package
- Parallel computations from the [future](https://github.com/HenrikBengtsson/future) package
Attach the package to follow the examples:
```{r}
library(coro)
```
### Async/await functions
Concurrent programming is made straightforward by async-await functions. Whenever you are waiting for a result that may take a while (downloading a file, computing a value in an external process), use `await()`. The argument to `await()` must return a promise from the [promises](https://github.com/rstudio/promises/) package.
Concurrent code based on promises can quickly become hard to write and follow. In the following artificial example, we wait for a download to complete, then decide to launch a computation in an external process depending on a property of the downloaded data. We also handle some errors specifically.
```{r}
my_async <- function() {
async_download() %>%
then(function(data) {
if (ncol(data) > 10) {
then(future::future(fib(30)), function(fib) {
data / fib
})
} else {
data
}
}, onRejected = function(err) {
if (inherits(err, "download_error")) {
NULL
} else {
stop(err)
}
})
}
```
Rewriting this function with async/await greatly simplifies the code:
```{r}
my_async <- async(function() {
data <- tryCatch(
await(async_download()),
download_error = function(err) NULL
)
if (is.null(data)) {
return(NULL)
}
if (ncol(data) > 10) {
fib <- await(future::future(fib(30)))
data <- data /fib
}
data
})
```
### Generators
Generators are based on a simple iteration protocol:
- Iterators are functions.
- They can be advanced by calling the function. The new value is returned.
- An exhausted iterator returns the sentinel symbol `exhausted`.
The `generator()` function creates a generator factory which returns generator instances:
```{r}
# Create a generator factory
generate_abc <- generator(function() {
for (x in letters[1:3]) {
yield(x)
}
})
# Create a generator instance
abc <- generate_abc()
```
A generator instance is an iterator function which yields values:
```{r}
abc
abc()
```
Collect all remaining values from an iterator with `collect()`:
```{r}
collect(abc)
```
Iterate over an iterator with `loop()`:
```{r}
loop(for (x in generate_abc()) {
print(toupper(x))
})
```
See `vignette("generator")` for more information.
### Compatibility with the reticulate package
Python iterators imported with the [reticulate](https://rstudio.github.io/reticulate/) package are compatible with `loop()` and `collect()`:
```{r}
suppressMessages(library(reticulate))
py_run_string("
def first_n(n):
num = 1
while num <= n:
yield num
num += 1
")
loop(for (x in py$first_n(3)) {
print(x * 2)
})
```
They can also be composed with coro generators:
```{r}
times <- generator(function(it, n) for (x in it) yield(x * n))
composed <- times(py$first_n(3), 10)
collect(composed)
```
## Limitations
`yield()` and `await()` can be used in loops, if/else branches, `tryCatch()` expressions, or any combinations of these. However they can't be used as function arguments. These will cause errors:
```{r, eval = FALSE}
generator(function() {
list(yield("foo"))
})
async(function() {
list(await(foo()))
})
```
Fortunately it is easy to rewrite the code to work around this limitation:
```{r, eval = FALSE}
generator(function() {
x <- yield("foo")
list(x)
})
async(function() {
x <- await(foo())
list(x)
})
```
## How does it work
Coroutines are an [abstraction for state machines](https://eli.thegreenplace.net/2009/08/29/co-routines-as-an-alternative-to-state-machines) in languages that support them. Conversely, you can implement coroutines by rewriting the code source provided by the user as a state machine. Pass `internals = TRUE` to the print methods of coroutines to reveal the state machine that is running under the hood:
```{r}
print(generate_abc, internals = TRUE)
```
Despite this transformation of source code, `browser()` and step-debugging still work as you would expect. This is because coro keeps track of the source references from the original code.
## Acknowledgements
- The [regenerator](https://facebook.github.io/regenerator/) Javascript package which uses a similar transformation to implement generators and async functions in older versions of Javascript.
- Gabor Csardi for many interesting discussions about concurrency and the design of coro.
## Installation
Install the development version from github with:
```r
# install.packages("devtools")
devtools::install_github("r-lib/coro", build_vignettes = TRUE)
```
## Code of Conduct
Please note that the coro project is released with a [Contributor Code of Conduct](https://coro.r-lib.org/CODE_OF_CONDUCT.html). By contributing to this project, you agree to abide by its terms.