https://github.com/pyprogrammerblog/arend
A simple producer consumer library for Beanstalkd queue.
https://github.com/pyprogrammerblog/arend
beanstalkd beanstalkd-monitor fastapi producer-consumer rabbitmq redis sqlmodel
Last synced: 2 months ago
JSON representation
A simple producer consumer library for Beanstalkd queue.
- Host: GitHub
- URL: https://github.com/pyprogrammerblog/arend
- Owner: pyprogrammerblog
- License: mit
- Created: 2021-05-14T19:13:54.000Z (over 4 years ago)
- Default Branch: master
- Last Pushed: 2022-10-02T13:57:45.000Z (about 3 years ago)
- Last Synced: 2025-06-24T12:19:13.249Z (3 months ago)
- Topics: beanstalkd, beanstalkd-monitor, fastapi, producer-consumer, rabbitmq, redis, sqlmodel
- Language: Python
- Homepage:
- Size: 22.5 KB
- Stars: 1
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
Arend
========
A simple producer-consumer library for the Beanstalkd queue.
Installation
--------------
Hit the command:
```shell
pip install arend
```
Basic Usage
--------------
In your code:
```python
from arend import arend_task
from arend.backends.mongo import MongoSettings
from arend.brokers import BeanstalkdSettings
from arend.settings import ArendSettings
from arend.worker import consumer
settings = ArendSettings(
beanstalkd=BeanstalkdSettings(host="beanstalkd", port=11300),
backend=MongoSettings(
mongo_connection="mongodb://user:pass@mongo:27017",
mongo_db="db",
mongo_collection="Tasks"
),
)
@arend_task(queue="my_queue", settings=settings)
def double(num: int):
return 2 * num
double(2) # returns 4
task = double.apply_async(args=(4,)) # It is sent to the queue
consumer(queue="my_queue", settings=settings) # consume tasks from the queue
Task = settings.get_backend() # you can check your backend for the result
task = Task.get(uuid=task.uuid)
assert task.result == 4
```
Backends
-------------------
The available backends to store logs are **Mongo** and **Redis**.
Please read the [docs](https://arend.readthedocs.io/en/latest/)
for further information.
Setting your backend with environment variables
--------------------------------------------------
You can set your backend by defining env vars.
The `AREND__` prefix indicates that it belongs to the Arend.
```shell
# Redis
AREND__REDIS_HOST='redis'
AREND__REDIS_DB='1'
AREND__REDIS_PASSWORD='pass'
...
# Mongo
AREND__MONGO_CONNECTION='mongodb://user:pass@mongo:27017'
AREND__MONGO_DB='db'
AREND__MONGO_COLLECTION='logs'
...
```
In your code:
```python
from arend import arend_task
from arend.worker import consumer
@arend_task(queue="my_queue")
def double(num: int):
return 2 * num
double.apply_async(args=(4,)) # It is sent to the queue
consumer(queue="my_queue")
```
Documentation
--------------
Please visit this [link](https://arend.readthedocs.io/en/latest/) for documentation.