https://github.com/kpn/arq-prometheus
Prometheus metrics for arq job queues
https://github.com/kpn/arq-prometheus
async-task asyncio prometheus-metrics
Last synced: about 1 year ago
JSON representation
Prometheus metrics for arq job queues
- Host: GitHub
- URL: https://github.com/kpn/arq-prometheus
- Owner: kpn
- License: apache-2.0
- Created: 2022-07-21T13:04:34.000Z (almost 4 years ago)
- Default Branch: main
- Last Pushed: 2023-08-04T14:19:45.000Z (almost 3 years ago)
- Last Synced: 2025-04-04T21:09:47.471Z (about 1 year ago)
- Topics: async-task, asyncio, prometheus-metrics
- Language: Python
- Homepage:
- Size: 28.3 KB
- Stars: 9
- Watchers: 6
- Forks: 1
- Open Issues: 3
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
Awesome Lists containing this project
README
# Arq-prometheus
[](https://pypi.org/project/arq-prometheus/)
[](https://pypi.org/project/arq-prometheus/)
[](https://pypi.org/project/arq-prometheus/)
[](https://codecov.io/gh/kpn/arq-prometheus)
Prometheus metrics for [arq](https://github.com/samuelcolvin/arq)
⚠️ WARNING! This is a project in alpha phase ⚠️
## Installation
[Pip](https://pip.pypa.io/en/stable/)
```sh
pip install -U arq-prometheus
```
[Poetry](https://python-poetry.org/)
```sh
poetry add arq-prometheus
```
## Description
The metrics exposed are the same as the health check.
| Metric name | Description |
| ----------------------- | -------------------------------- |
| `arq_jobs_completed` | The number of jobs completed |
| `arq_jobs_failed` | The total number of errored jobs |
| `arq_jobs_retried` | The total number of retried jobs |
| `arq_ongoing_jobs` | The number of jobs in progress |
| `arq_queued_inprogress` | The number of jobs in progress |
When working with `arq` I found some limitations, it was specially hard to get access to
the worker in order to retrieve information like the `queue_name` or `health_check_key`.
The startup and shutdown functions only make available a `ctx` with the redis connection.
This means that if you provide a custom `queue_name` or `health_check_key`, you will
also have to provide them to `ArqPrometheusMetrics`.
## Usage
````python
# example_worker.py
from arq_prometheus import ArqPrometheusMetrics
async def startup(ctx):
arq_prometheus = ArqPrometheusMetrics(ctx, delay=delay)
ctx["arq_prometheus"] = await arq_prometheus.start()
async def shutdown(ctx):
await ctx["arq_prometheus"].stop()
class WorkerSettings:
on_startup = startup
on_shutdown = shutdown
function = [] # your arq jobs
... # other settings
````
Start your arq worker,
```sh
arq example_worker.WorkerSettings
```
Make request to `localhost:8081` (or open in browser).
```sh
curl localhost:8081
```
## Arguments
- `ctx: dict`: arq context
- `queue_name: str = default_queue_name`: name of the arq queue
- `health_check_key: Optional[str] = None`: arq health key
- `delay: datetime.timedelta = datetime.timedelta(seconds=5)`: a datetime.timedelta
- `enable_webserver: bool = True`: set to True if you want a web server exposing the metrics
- `addr: str = "0.0.0.0"`: webserver address
- `port: int = 8081`: webserver port
- `registry: prom.CollectorRegistry = prom.REGISTRY`: the prometheus registry, usually you do not have to override this