https://github.com/mopeyjellyfish/rhubarb
An Event Bus library that simplifies interacting with multiple queue backends into a single API
https://github.com/mopeyjellyfish/rhubarb
asyncio eventbus kafka message-queue postgres python rabbitmq redis websocket websockets
Last synced: about 1 year ago
JSON representation
An Event Bus library that simplifies interacting with multiple queue backends into a single API
- Host: GitHub
- URL: https://github.com/mopeyjellyfish/rhubarb
- Owner: mopeyjellyfish
- License: mit
- Created: 2021-11-04T23:10:04.000Z (over 4 years ago)
- Default Branch: main
- Last Pushed: 2024-03-15T14:57:31.000Z (over 2 years ago)
- Last Synced: 2025-05-12T14:11:51.500Z (about 1 year ago)
- Topics: asyncio, eventbus, kafka, message-queue, postgres, python, rabbitmq, redis, websocket, websockets
- Language: Python
- Homepage:
- Size: 1.22 MB
- Stars: 6
- Watchers: 1
- Forks: 0
- Open Issues: 54
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Security: SECURITY.md
Awesome Lists containing this project
README
# Rhubarb
[](https://img.shields.io/github/v/release/mopeyjellyfish/rhubarb)
[](https://github.com/mopeyjellyfish/rhubarb/actions?query=workflow%3Abuild)
[](https://pypi.org/project/rhubarb-py)
[](https://github.com/mopeyjellyfish/rhubarb/pulls?utf8=%E2%9C%93&q=is%3Apr%20author%3Aapp%2Fdependabot)
[](https://codecov.io/gh/mopeyjellyfish/rhubarb)
[](https://rhubarb-py.readthedocs.io/en/latest/?badge=latest)
[](https://github.com/psf/black)
[](https://github.com/PyCQA/bandit)
[](https://github.com/mopeyjellyfish/rhubarb/blob/master/.pre-commit-config.yaml)
[](https://github.com/mopeyjellyfish/rhubarb/releases)
[](https://img.shields.io/github/commit-activity/m/mopeyjellyfish/rhubarb)
[](https://github.com/mopeyjellyfish/rhubarb/blob/master/LICENSE)
Rhubarb is a library that simplifies realtime streaming of events for a number of backends in to a single API. Currently supports [`Postgres`](https://github.com/MagicStack/asyncpg), [`kafka`](https://github.com/aio-libs/aiokafka), [`RabbitMQ`](https://github.com/mosquito/aio-pika), [`redis`](https://github.com/aio-libs/aioredis-py) as well as an internal memory backend useful for testing.
## Installation
There are a number of backends that can be used with Rhubarb:
| Kafka | Postgres | Redis | RabbitMQ |
| --------------------------------------------------------------------- | --------------------------------------------------------------------------------- | --------------------------------------------------------------------------------- |--------------------------------------------------------------------------------- |
|
| 
| 
| 
|
| `pip install rhubarb-py[kafka]` | `pip install rhubarb-py[postgres]` | `pip install rhubarb-py[redis]` | `pip install rhubarb-py[rabbitmq]` |
## Backends
- `Rhubarb("redis://localhost:6379/0")`
- `Rhubarb("kafka://localhost:9092")`
- `Rhubarb("postgres://postgres:postgres@localhost:5432/rhubarb")`
- `Rhubarb("amqp://guest:guest@localhost/")`
- `Rhubarb("memory://")`
## Quick start
### Simple event consumer
```python
async with Rhubarb("redis://localhost:6379/0") as events:
async with events.subscribe(channel="CHATROOM") as subscriber:
async for event in subscriber:
await websocket.send_text(event.message)
```
### Simple event producer
```python
async with Rhubarb("redis://localhost:6379/0") as events:
await events.publish("test message")
```
### History retrieval
```python
async with Rhubarb("redis://localhost:6379/0") as events:
async with events.subscribe(channel="CHATROOM", history=10) as subscriber: # read the last 10 events published to the channel
async for event in subscriber:
await websocket.send_text(event.message)
```
### Custom serializer & deserializer
```python
async with Rhubarb("redis://localhost:6379/0", serializer=json.dumps, deserializer=json.loads) as events:
async with events.subscribe(channel="CHATROOM", history=10) as subscriber: # read the last 10 events published to the channel
async for event in subscriber:
await websocket.send_text(event.message)
```
### Group subscribing (at-most-once processing)
```python
async with Rhubarb("redis://localhost:6379/0", serializer=json.dumps, deserializer=json.loads) as events:
async with events.subscribe(
"TEST-GROUP-CHANNEL", group_name="TEST_GROUP", consumer_name="sub_1"
) as subscriber_1:
async for event in subscriber:
await process_job(event)
```
## Example
A minimal working example can be found in [example](https://github.com/mopeyjellyfish/rhubarb/blob/main/example/app.py) directory.
```python
import os
from starlette.applications import Starlette
from starlette.concurrency import run_until_first_complete
from starlette.responses import HTMLResponse
from starlette.routing import Route, WebSocketRoute
from rhubarb import Rhubarb
URL = os.environ.get("URL", "redis://localhost:6379/0")
events = Rhubarb(URL)
html = """
Chat
WebSocket Chat
Send
var ws = new WebSocket("ws://localhost:8000/ws");
ws.onmessage = function(event) {
var messages = document.getElementById('messages')
var message = document.createElement('li')
var content = document.createTextNode(event.data)
message.appendChild(content)
messages.appendChild(message)
};
function sendMessage(event) {
var input = document.getElementById("messageText")
ws.send(input.value)
input.value = ''
event.preventDefault()
}
"""
async def homepage(_):
return HTMLResponse(html)
async def room_consumer(websocket):
async for message in websocket.iter_text():
await events.publish(channel="chatroom", message=message)
async def room_producer(websocket):
async with events.subscribe(channel="chatroom") as subscriber:
async for event in subscriber:
await websocket.send_text(event.message)
async def ws(websocket):
await websocket.accept()
await run_until_first_complete(
(room_consumer, {"websocket": websocket}),
(room_producer, {"websocket": websocket}),
)
routes = [
Route("/", homepage),
WebSocketRoute("/ws", ws, name="chatroom_ws"),
]
app = Starlette(
routes=routes,
on_startup=[events.connect],
on_shutdown=[events.disconnect],
)
```
## 🛡 License
[](https://github.com/mopeyjellyfish/rhubarb/blob/master/LICENSE)
This project is licensed under the terms of the `MIT` license. See [LICENSE](https://github.com/mopeyjellyfish/rhubarb/blob/master/LICENSE) for more details.