https://github.com/lecardozo/rworker

Celery worker for R tasks! Celery :left_right_arrow: R
https://github.com/lecardozo/rworker

celery message-queue r

Last synced: 3 months ago
JSON representation

Celery worker for R tasks! Celery :left_right_arrow: R

• Host: GitHub
• URL: https://github.com/lecardozo/rworker
• Owner: lecardozo
• License: other
• Created: 2017-07-22T18:58:19.000Z (over 7 years ago)
• Default Branch: master
• Last Pushed: 2021-05-06T16:31:18.000Z (over 3 years ago)
• Last Synced: 2024-06-05T01:44:34.050Z (5 months ago)
• Topics: celery, message-queue, r
• Language: R
• Homepage:
• Size: 176 KB
• Stars: 27
• Watchers: 3
• Forks: 11
• Open Issues: 3
• Metadata Files:
  • Readme: README.md
  • License: LICENSE

Awesome Lists containing this project

• jimsghstars - lecardozo/rworker - Celery worker for R tasks! Celery :left_right_arrow: R (R)

README

        

# rworker [![Travis-CI Build Status](https://travis-ci.org/lecardozo/rworker.svg?branch=master)](https://travis-ci.org/lecardozo/rworker)

Celery worker for R tasks.

## Motivation

The main motivation for this package was the need for executing long running R functions

triggered by [Celery](https://github.com/celery/celery) (asynchronous task queue package for Python).

![Data flow](https://raw.githubusercontent.com/lecardozo/rworker/master/img/rworker.png)

## Usage

##### Start consuming tasks from R

```R

library(rworker)

library(magrittr)

# Broker url

redis_url <- 'redis://localhost:6379'

# Instantiate Rworker object

consumer <- rworker(name='celery', workers=2, queue=redis_url, backend=redis_url)

# Register tasks

(function(){

    Sys.sleep(5)

}) %>% consumer$task(name='long_running_task')

# Start consuming messages

consumer$consume()

```

The `rworker` function returns a `Rworker` object. This object is responsible for registering tasks, listening for messages coming from the message queue and triggering tasks execution on background processes

##### Send tasks from Python

```python

from celery import Celery

worker = Celery('app', broker="redis://localhost:6379/0", backend="redis://localhost:6379/0")

worker.send_task('long_running_task')

```

#### Tutorial

You can find more information about usage [here](https://lecardozo.github.io/rworker).