https://github.com/tastyware/streaq
Fast, async, fully-typed distributed task queue via Redis streams
https://github.com/tastyware/streaq
anyio asyncio distributed-systems fastapi redis task-queue trio typed
Last synced: 7 days ago
JSON representation
Fast, async, fully-typed distributed task queue via Redis streams
- Host: GitHub
- URL: https://github.com/tastyware/streaq
- Owner: tastyware
- License: mit
- Created: 2025-02-05T00:52:19.000Z (about 1 year ago)
- Default Branch: master
- Last Pushed: 2026-02-20T16:48:28.000Z (9 days ago)
- Last Synced: 2026-02-20T21:09:59.778Z (8 days ago)
- Topics: anyio, asyncio, distributed-systems, fastapi, redis, task-queue, trio, typed
- Language: Python
- Homepage: https://streaq.rtfd.io
- Size: 980 KB
- Stars: 125
- Watchers: 2
- Forks: 11
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing.rst
- Funding: .github/FUNDING.yml
- License: LICENSE
Awesome Lists containing this project
README
[](https://streaq.readthedocs.io/en/latest/?badge=latest)
[](https://pypi.org/project/streaq)
[](https://pepy.tech/project/streaq)
[](https://github.com/tastyware/streaq/releases)
[](https://matrix.to/#/#tastyware:gitter.im)
# streaQ
Fast, async, fully-typed distributed task queue via Redis streams
## Features
- Up to [5x faster](https://github.com/tastyware/streaq/tree/master/benchmarks) than `arq`
- Fully typed
- Comprehensive documentation
- Support for delayed/scheduled tasks
- Cron jobs
- Task middleware
- Task dependency graph
- Pipelining
- Priority queues
- Support for synchronous tasks (run in separate threads)
- Redis Sentinel & Cluster support for production
- Built-in web UI for monitoring tasks
- Built with structured concurrency on `anyio`, supports both `asyncio` and `trio`
> [!TIP]
> Sick of `redis-py`? Check out [coredis](https://coredis.readthedocs.io/en/latest/), a fast, fully-typed Redis client that supports Trio!
## Installation
```console
$ pip install streaq
```
## Getting started
To start, you'll need to create a `Worker` object:
```python
from streaq import Worker
worker = Worker(redis_url="redis://localhost:6379", anyio_backend="trio")
```
You can then register async tasks with the worker like this:
```python
import trio
@worker.task
async def sleeper(time: int) -> int:
await trio.sleep(time)
return time
@worker.cron("* * * * mon-fri") # every minute on weekdays
async def cronjob() -> None:
print("Nobody respects the spammish repetition!")
```
Finally, let's use the worker's async context manager to queue up some tasks:
```python
async with worker:
await sleeper.enqueue(3)
# enqueue returns a task object that can be used to get results/info
task = await sleeper.enqueue(1).start(delay=3)
print(await task.info())
print(await task.result(timeout=5))
```
Putting this all together gives us [example.py](https://github.com/tastyware/streaq/blob/master/example.py). Let's spin up a worker:
```
$ streaq run example:worker
```
and queue up some tasks like so:
```
$ python example.py
```
Let's see what the output looks like:
```
[INFO] 2025-09-23 02:14:30: starting worker 3265311d for 2 functions
[INFO] 2025-09-23 02:14:35: task sleeper □ cf0c55387a214320bd23e8987283a562 → worker 3265311d
[INFO] 2025-09-23 02:14:38: task sleeper ■ cf0c55387a214320bd23e8987283a562 ← 3
[INFO] 2025-09-23 02:14:40: task sleeper □ 1de3f192ee4a40d4884ebf303874681c → worker 3265311d
[INFO] 2025-09-23 02:14:41: task sleeper ■ 1de3f192ee4a40d4884ebf303874681c ← 1
[INFO] 2025-09-23 02:15:00: task cronjob □ 2a4b864e5ecd4fc99979a92f5db3a6e0 → worker 3265311d
Nobody respects the spammish repetition!
[INFO] 2025-09-23 02:15:00: task cronjob ■ 2a4b864e5ecd4fc99979a92f5db3a6e0 ← None
```
```python
TaskInfo(fn_name='sleeper', enqueue_time=1751508876961, tries=0, scheduled=datetime.datetime(2025, 7, 3, 2, 14, 39, 961000, tzinfo=datetime.timezone.utc), dependencies=set(), dependents=set())
TaskResult(fn_name='sleeper', enqueue_time=1751508876961, success=True, start_time=1751508880500, finish_time=1751508881503, tries=1, worker_id='ca5bd9eb', _result=1)
```
For more examples, check out the [documentation](https://streaq.readthedocs.io/en/latest/).