Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/ovh/celery-director
Simple and rapid framework to build workflows with Celery
https://github.com/ovh/celery-director
celery python workflow
Last synced: 7 days ago
JSON representation
Simple and rapid framework to build workflows with Celery
- Host: GitHub
- URL: https://github.com/ovh/celery-director
- Owner: ovh
- License: bsd-3-clause
- Created: 2020-01-30T13:02:40.000Z (almost 5 years ago)
- Default Branch: master
- Last Pushed: 2024-06-18T00:43:34.000Z (5 months ago)
- Last Synced: 2024-10-29T22:56:41.228Z (14 days ago)
- Topics: celery, python, workflow
- Language: Python
- Homepage: https://ovh.github.io/celery-director/
- Size: 3.13 MB
- Stars: 536
- Watchers: 26
- Forks: 58
- Open Issues: 18
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Codeowners: .github/CODEOWNERS
- Authors: AUTHORS
Awesome Lists containing this project
README
----------------
Director is a simple and rapid framework used to manage tasks and build workflows using Celery.
The objective is to make Celery easier to use by providing :
- a WebUI, an API and a CLI to manage and execute the workflows,
- a WebUI to track the tasks states,
- a YAML syntax used to combine tasks into workflows,
- the ability to periodically launch a whole workflow,
- and many others.
See how to use Director with the quickstart and guides in the [documentation](https://ovh.github.io/celery-director/).
## Installation
Install the latest version of Director with pip (requires at least `Python 3.7`):
```bash
pip install celery-director
```
## Usage
### Write your code in Python
```python
# tasks/orders.py
from director import task
from .utils import Order, Mail
@task(name="ORDER_PRODUCT")
def order_product(*args, **kwargs):
order = Order(
user=kwargs["payload"]["user"],
product=kwargs["payload"]["product"]
).save()
return {"id": order.id}
@task(name="SEND_MAIL")
def send_mail(*args, **kwargs):
order_id = args[0]["id"]
mail = Mail(
title=f"Your order #{order_id} has been received",
user=kwargs["payload"]["user"]
)
mail.send()
```
### Build your workflows in YAML
```yaml
# workflows.yml
product.ORDER:
tasks:
- ORDER_PRODUCT
- SEND_MAIL
```
### Run it
You can simply test your workflow in local :
```bash
$ director workflow run product.ORDER '{"user": 1234, "product": 1000}'
```
And run it in production using the director API :
```bash
$ curl --header "Content-Type: application/json" \
--request POST \
--data '{"project": "product", "name": "ORDER", "payload": {"user": 1234, "product": 1000}}' \
http://localhost:8000/api/workflows
```
Read the [documentation](https://ovh.github.io/celery-director/) to try the quickstart and see advanced usages of Celery Director.
## Project layout
.env # The configuration file.
workflows.yml # The workflows definition.
tasks/
example.py # A file containing some tasks.
... # Other files containing other tasks.
## Commands
* `director init [path]` - Create a new project.
* `director celery [worker|beat|flower]` - Start Celery daemons.
* `director webserver` - Start the webserver.
* `director workflow [list|show|run]` - Manage your project workflows.
## License
See https://github.com/ovh/celery-director/blob/master/LICENSE