Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/rusty-celery/rusty-celery
🦀 Rust implementation of Celery for producing and consuming background tasks
https://github.com/rusty-celery/rusty-celery
celery celery-workers rabbitmq redis rust
Last synced: about 2 months ago
JSON representation
🦀 Rust implementation of Celery for producing and consuming background tasks
- Host: GitHub
- URL: https://github.com/rusty-celery/rusty-celery
- Owner: rusty-celery
- License: apache-2.0
- Created: 2020-01-08T21:36:10.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-03-06T13:48:57.000Z (7 months ago)
- Last Synced: 2024-04-13T17:57:08.029Z (5 months ago)
- Topics: celery, celery-workers, rabbitmq, redis, rust
- Language: Rust
- Homepage: https://rusty-celery.github.io
- Size: 2.18 MB
- Stars: 711
- Watchers: 17
- Forks: 82
- Open Issues: 47
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
Awesome Lists containing this project
README
A Rust implementation of Celery for producing and consuming asynchronous tasks with a distributed message queue.
We welcome contributions from everyone regardless of your experience level with Rust. For complete beginners, see [HACKING_QUICKSTART.md](https://github.com/rusty-celery/rusty-celery/blob/main/HACKING_QUICKSTART.md).
If you already know the basics of Rust but are new to Celery, check out the [Rusty Celery Book](https://rusty-celery.github.io/) or the original Python [Celery Project](http://www.celeryproject.org/).
## Quick start
Define tasks by decorating functions with the [`task`](https://docs.rs/celery/*/celery/attr.task.html) attribute.
```rust
use celery::prelude::*;
#[celery::task]
fn add(x: i32, y: i32) -> TaskResult {
Ok(x + y)
}
```
Create an app with the [`app`](https://docs.rs/celery/*/celery/macro.celery_app.html) macro
and register your tasks with it:
```rust
let my_app = celery::app!(
broker = AMQPBroker { std::env::var("AMQP_ADDR").unwrap() },
tasks = [add],
task_routes = [
"*" => "celery",
],
).await?;
```
Then send tasks to a queue with
```rust
my_app.send_task(add::new(1, 2)).await?;
```
And consume tasks as a worker from a queue with
```rust
my_app.consume().await?;
```
## Examples
The [`examples/`](https://github.com/rusty-celery/rusty-celery/tree/main/examples) directory contains:
- a simple Celery app implemented in Rust using an AMQP broker ([`examples/celery_app.rs`](https://github.com/rusty-celery/rusty-celery/blob/main/examples/celery_app.rs)),
- the same Celery app implemented in Python ([`examples/celery_app.py`](https://github.com/rusty-celery/rusty-celery/blob/main/examples/celery_app.py)),
- and a Beat app implemented in Rust ([`examples/beat_app.rs`](https://github.com/rusty-celery/rusty-celery/blob/main/examples/beat_app.rs)).
### Prerequisites
If you already have an AMQP broker running you can set the environment variable `AMQP_ADDR` to your broker's URL (e.g., `amqp://localhost:5672//`, where
the second slash at the end is the name of the [default vhost](https://www.rabbitmq.com/access-control.html#default-state)).
Otherwise simply run the helper script:
```bash
./scripts/brokers/amqp.sh
```
This will download and run the official [RabbitMQ](https://www.rabbitmq.com/) image (RabbitMQ is a popular AMQP broker).
### Run the examples
#### Run Rust Celery app
You can consume tasks with:
```bash
cargo run --example celery_app consume
```
And you can produce tasks with:
```bash
cargo run --example celery_app produce [task_name]
```
Current supported tasks for this example are: `add`, `buggy_task`, `long_running_task` and `bound_task`
#### Run Python Celery app
Similarly, you can consume or produce tasks from Python by running
```bash
python examples/celery_app.py consume [task_name]
```
or
```bash
python examples/celery_app.py produce
```
You'll need to have Python 3 installed, along with the requirements listed in the `requirements.txt` file. You'll also have to provide a task name. This example implements 4 tasks: `add`, `buggy_task`, `long_running_task` and `bound_task`
#### Run Rust Beat app
You can start the Rust beat with:
```bash
cargo run --example beat_app
```
And then you can consume tasks from Rust or Python as explained above.
## Road map and current state
✅ = Supported and mostly stable, although there may be a few incomplete features.
⚠️ = Partially implemented and under active development.
🔴 = Not supported yet but on-deck to be implemented soon.
### Core
| | Status | Tracking |
| ---------------- |:-------:| --------- |
| Protocol | ⚠️ | [](https://github.com/rusty-celery/rusty-celery/issues?q=is%3Aissue+label%3A%22Protocol+Feature%22+is%3Aopen) |
| Producers | ✅ | |
| Consumers | ✅ | |
| Brokers | ✅ | |
| Beat | ✅ | |
| Backends | 🔴 | |
| [Baskets](https://github.com/rusty-celery/rusty-celery/issues/53) | 🔴 | |
### Brokers
| | Status | Tracking |
| ----- |:------:| -------- |
| AMQP | ✅ | [](https://github.com/rusty-celery/rusty-celery/labels/Broker%3A%20AMQP) |
| Redis | ✅ | [](https://github.com/rusty-celery/rusty-celery/labels/Broker%3A%20Redis) |
### Backends
| | Status | Tracking |
| ----------- |:------:| -------- |
| RPC | 🔴 | [](https://github.com/rusty-celery/rusty-celery/labels/Backend%3A%20RPC) |
| Redis | 🔴 | [](https://github.com/rusty-celery/rusty-celery/labels/Backend%3A%20Redis) |