https://github.com/danfimov/taskiq-postgres
PostgreSQL integration for Taskiq with support for asyncpg, psycopg, psqlpy and aiopg drivers.
https://github.com/danfimov/taskiq-postgres
postgresql taskiq taskiq-broker taskiq-result-backend
Last synced: 5 months ago
JSON representation
PostgreSQL integration for Taskiq with support for asyncpg, psycopg, psqlpy and aiopg drivers.
- Host: GitHub
- URL: https://github.com/danfimov/taskiq-postgres
- Owner: danfimov
- License: mit
- Created: 2025-05-11T10:34:51.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2026-01-19T16:58:04.000Z (5 months ago)
- Last Synced: 2026-01-19T22:54:06.903Z (5 months ago)
- Topics: postgresql, taskiq, taskiq-broker, taskiq-result-backend
- Language: Python
- Homepage: https://danfimov.github.io/taskiq-postgres/
- Size: 4.43 MB
- Stars: 14
- Watchers: 1
- Forks: 2
- Open Issues: 4
-
Metadata Files:
- Readme: README.md
- Contributing: docs/contributing.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://pypi.org/project/taskiq-postgres/)
[](https://pypi.org/project/taskiq-postgres/)
[&style=for-the-badge)](https://github.com/danfimov/taskiq-postgres)
PostgreSQL integration for Taskiq with support for asyncpg, psqlpy, psycopg and aiopg drivers.
## Features
- **PostgreSQL Broker** - high-performance message broker using PostgreSQL LISTEN/NOTIFY;
- **Result Backend** - persistent task result storage with configurable retention;
- **Scheduler Source** - cron-like task scheduling with PostgreSQL persistence;
- **Multiple Drivers** - support for asyncpg, psycopg3, psqlpy and aiopg;
- **Flexible Configuration** - customizable table names, field types, and connection options;
- **Multiple Serializers** - support for different serialization methods (Pickle, JSON, etc.).
See usage guide in [documentation](https://danfimov.github.io/taskiq-postgres/) or explore examples in [separate directory](https://github.com/danfimov/taskiq-postgres/examples).
## Installation
Depending on your preferred PostgreSQL driver, you can install this library with the corresponding extra:
```bash
# with asyncpg
pip install taskiq-postgres[asyncpg]
# with psqlpy
pip install taskiq-postgres[psqlpy]
# with psycopg3
pip install taskiq-postgres[psycopg]
# with aiopg
pip install taskiq-postgres[aiopg]
```
## Quick start
### Basic task processing
1. Define your broker with [asyncpg](https://github.com/MagicStack/asyncpg):
```python
# broker_example.py
import asyncio
from taskiq_pg.asyncpg import AsyncpgBroker, AsyncpgResultBackend
dsn = "postgres://taskiq_postgres:look_in_vault@localhost:5432/taskiq_postgres"
broker = AsyncpgBroker(dsn).with_result_backend(AsyncpgResultBackend(dsn))
@broker.task("solve_all_problems")
async def best_task_ever() -> None:
"""Solve all problems in the world."""
await asyncio.sleep(2)
print("All problems are solved!")
async def main():
await broker.startup()
task = await best_task_ever.kiq()
print(await task.wait_result())
await broker.shutdown()
if __name__ == "__main__":
asyncio.run(main())
```
2. Start a worker to process tasks (by default taskiq runs two instances of worker):
```bash
taskiq worker broker_example:broker
```
3. Run `broker_example.py` file to send a task to the worker:
```bash
python broker_example.py
```
Your experience with other drivers will be pretty similar. Just change the import statement and that's it.
### Task scheduling
1. Define your broker and schedule source:
```python
# scheduler_example.py
import asyncio
from taskiq import TaskiqScheduler
from taskiq_pg.asyncpg import AsyncpgBroker, AsyncpgScheduleSource
dsn = "postgres://taskiq_postgres:look_in_vault@localhost:5432/taskiq_postgres"
broker = AsyncpgBroker(dsn)
scheduler = TaskiqScheduler(
broker=broker,
sources=[AsyncpgScheduleSource(
dsn=dsn,
broker=broker,
)],
)
@broker.task(
task_name="solve_all_problems",
schedule=[
{
"cron": "*/1 * * * *", # type: str, either cron or time should be specified.
"cron_offset": None, # type: str | None, can be omitted. For example "Europe/Berlin".
"time": None, # type: datetime | None, either cron or time should be specified.
"args": [], # type list[Any] | None, can be omitted.
"kwargs": {}, # type: dict[str, Any] | None, can be omitted.
"labels": {}, # type: dict[str, Any] | None, can be omitted.
},
],
)
async def best_task_ever() -> None:
"""Solve all problems in the world."""
await asyncio.sleep(2)
print("All problems are solved!")
```
2. Start worker processes:
```bash
taskiq worker scheduler_example:broker
```
3. Run scheduler process:
```bash
taskiq scheduler scheduler_example:scheduler
```
## Motivation
There are too many libraries for PostgreSQL and Taskiq integration. Although they have different view on interface and different functionality.
To address this issue I created this library with a common interface for most popular PostgreSQL drivers that handle similarity across functionality of result backends, brokers and schedule sources.