{"id":28089359,"url":"https://github.com/first-digital-finance/pyrmq","last_synced_at":"2025-05-13T12:57:35.265Z","repository":{"id":37023636,"uuid":"271849441","full_name":"first-digital-finance/pyrmq","owner":"first-digital-finance","description":"Python with RabbitMQ—simplified so you won't have to.","archived":false,"fork":false,"pushed_at":"2025-05-02T12:31:25.000Z","size":162,"stargazers_count":19,"open_issues_count":15,"forks_count":9,"subscribers_count":3,"default_branch":"master","last_synced_at":"2025-05-02T13:43:08.005Z","etag":null,"topics":["consumer","messages","pika","publisher","pyrabbit","python","python3","queues","rabbitmq"],"latest_commit_sha":null,"homepage":"https://pyrmq.readthedocs.io","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/first-digital-finance.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2020-06-12T17:01:49.000Z","updated_at":"2025-04-25T13:09:24.000Z","dependencies_parsed_at":"2023-10-14T19:29:48.109Z","dependency_job_id":"ee13a0ce-4efd-45ac-a83f-5a71ace33548","html_url":"https://github.com/first-digital-finance/pyrmq","commit_stats":null,"previous_names":[],"tags_count":28,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-digital-finance%2Fpyrmq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-digital-finance%2Fpyrmq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-digital-finance%2Fpyrmq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/first-digital-finance%2Fpyrmq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/first-digital-finance","download_url":"https://codeload.github.com/first-digital-finance/pyrmq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":253948349,"owners_count":21988953,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consumer","messages","pika","publisher","pyrabbit","python","python3","queues","rabbitmq"],"created_at":"2025-05-13T12:57:34.705Z","updated_at":"2025-05-13T12:57:35.249Z","avatar_url":"https://github.com/first-digital-finance.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\u003c!--suppress HtmlDeprecatedAttribute --\u003e\n\u003cdiv align=\"center\"\u003e\n  \u003ch1\u003ePyRMQ\u003c/h1\u003e\n  \u003ca href=\"https://github.com/first-digital-finance/pyrmq/actions/workflows/testing.yml\"\u003e\u003cimg alt=\"GitHub Workflow Status\" src=\"https://img.shields.io/github/actions/workflow/status/first-digital-finance/pyrmq/testing.yml?branch=master\u0026style=for-the-badge\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/PyRMQ/\"\u003e\u003cimg alt=\"PyPI\" src=\"https://img.shields.io/pypi/v/pyrmq?style=for-the-badge\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pyrmq.readthedocs.io\"\u003e\u003cimg src='https://readthedocs.org/projects/pyrmq/badge/?version=latest\u0026style=for-the-badge' alt='Documentation Status' /\u003e\u003c/a\u003e\n  \u003ca href=\"https://codecov.io/gh/first-digital-finance/pyrmq\"\u003e\u003cimg alt=\"Codecov\" src=\"https://img.shields.io/codecov/c/github/first-digital-finance/pyrmq/master.svg?style=for-the-badge\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://pypi.org/project/PyRMQ/\"\u003e\u003cimg alt=\"Supports Python \u003e= 3.11\" src=\"https://img.shields.io/pypi/pyversions/pyrmq?style=for-the-badge\"/\u003e\u003c/a\u003e\n  \u003ca href=\"https://mit-license.org\" target=\"_blank\"\u003e\u003cimg src=\"https://img.shields.io/badge/license-MIT-blue.svg?longCache=true\u0026style=for-the-badge\" alt=\"License\"\u003e\u003c/a\u003e \n  \u003ca href=\"https://github.com/psf/black\"\u003e\u003cimg alt=\"Code style: black\" src=\"https://img.shields.io/badge/code%20style-black-000000.svg?longCache=true\u0026style=for-the-badge\"\u003e\u003c/a\u003e\n  \u003ca href=\"https://github.com/PyCQA/isort\"\u003e\u003cimg alt=\"Imports: isort\" src=\"https://img.shields.io/badge/%20imports-isort-%231674b1?style=for-the-badge\u0026labelColor=ef8336)](https://pycqa.github.io/isort/\"\u003e\u003c/a\u003e\n  \u003cp\u003ePython with RabbitMQ—simplified so you won't have to.\u003c/p\u003e\n\u003c/div\u003e\n\n## Features\nStop worrying about boilerplating and implementing retry logic for your queues. PyRMQ already\ndoes it for you.\n- Use out-of-the-box `Consumer` and `Publisher` classes created from `pika` for your projects and tests.\n- Custom DLX-DLK-based retry logic for message consumption.\n- Message priorities\n- Works with Python 3.11-3.13.\n- Production ready\n\n## Getting Started\n### Installation\nPyRMQ is available at PyPi.\n```shell script\npip install pyrmq\n```\n### Usage\n#### Publishing\nJust instantiate the feature you want with their respective settings.\nPyRMQ already works out of the box with RabbitMQ's [default initialization settings](https://hub.docker.com/_/rabbitmq).\n\n\u003e **Note:** The Publisher class only verifies that exchanges exist and does not create queues or exchanges.\n\u003e Exchanges must be created by a Consumer before a Publisher can use them.\n\n```python\nfrom pyrmq import Publisher\npublisher = Publisher(\n    exchange_name=\"exchange_name\",\n    queue_name=\"queue_name\",\n    routing_key=\"routing_key\",\n)\npublisher.publish({\"pyrmq\": \"My first message\"})\n```\n#### Publish message with priorities\nPyRMQ supports two ways to prioritize messages:\n\n1. **Quorum queues (recommended)**: Use the `is_priority` flag to set a priority of 5 (high priority).\n   ```python\n   from pyrmq import Publisher\n   publisher = Publisher(\n       exchange_name=\"exchange_name\",\n       queue_name=\"queue_name\",\n       routing_key=\"routing_key\",\n   )\n   publisher.publish({\"pyrmq\": \"High priority message\"}, is_priority=True)  # Priority 5\n   publisher.publish({\"pyrmq\": \"Normal message\"})  # Default priority 0\n   ```\n   \n   In quorum queues, messages with priority 5-255 are considered high priority, and those with priority 0-4 are normal priority. When both types exist in the queue, RabbitMQ maintains a 2:1 ratio, delivering at least 2 high priority messages for every 1 normal priority message.\n\n2. **Classic queues**: For finer-grained control with numeric priorities, configure your Consumer with the `x-max-priority` \n   argument and use message properties when publishing.\n   ```python\n   # When setting up the Consumer\n   consumer = Consumer(\n       exchange_name=\"exchange_name\",\n       queue_name=\"queue_name\",\n       routing_key=\"routing_key\",\n       queue_args={\"x-queue-type\": \"classic\", \"x-max-priority\": 5},\n       callback=callback\n   )\n   \n   # When publishing\n   publisher.publish({\"pyrmq\": \"Priority message\"}, message_properties={\"priority\": 3})\n   ```\n\nRead more about message priorities [here](https://www.rabbitmq.com/docs/priority).\n\n| :warning: Warning                                                                                  |\n|:---------------------------------------------------------------------------------------------------|\nAdding arguments on an existing queue is not possible. If you wish to add queue arguments, you will need to either\ndelete the existing queue then recreate the queue with arguments or simply make a new queue with the arguments.\n\n#### Consuming\nInstantiating a `Consumer` automatically starts it in its own thread making it\nnon-blocking by default. When run after the code from before, you should be\nable to receive the published data.\n```python\nfrom pyrmq import Consumer\n\ndef callback(data):\n    print(f\"Received {data}!\")\n\nconsumer = Consumer(\n    exchange_name=\"exchange_name\",\n    queue_name=\"queue_name\",\n    routing_key=\"routing_key\",\n    callback=callback\n)\nconsumer.start()\n```\n\n#### DLX-DLK Retry Logic\nWhat if you wanted to retry a failure on a consumed message? PyRMQ offers a custom solution that keeps your message\nin queues while retrying periodically for a set amount of times.\n\nThis approach uses [dead letter exchanges and queues](https://www.rabbitmq.com/dlx.html) to republish a message to your\noriginal queue once it has expired. PyRMQ creates this \"retry\" queue for you with the default naming convention of\nappending your original queue with `.retry`.\n\n```python\nfrom pyrmq import Consumer\n\ndef callback(data):\n    print(f\"Received {data}!\")\n    raise Exception\n\nconsumer = Consumer(\n    exchange_name=\"exchange_name\",\n    queue_name=\"queue_name\",\n    routing_key=\"routing_key\",\n    callback=callback,\n    is_dlk_retry_enabled=True,\n)\nconsumer.start()\n```\n\nThis will start a loop of passing your message between the original queue and the retry queue until it reaches\nthe default number of `max_retries`.\n\n##### DLX-DLK Retry backoff vs Periodic retries\nSince [RabbitMQ does not remove expired messages that aren't at the head of the queue](https://www.rabbitmq.com/ttl.html#per-message-ttl-caveats),\nthis leads to a congestion of the retry queue that is bottlenecked with an unexpired message\nat the head. As such, as of 3.3.0, PyRMQ will be using a simple periodic retry.\n\n#### Using other exchange types\nYou can use another exchange type just by simply specifying it in the Publisher class. The default is\n`direct`. \n\n```python\nfrom pyrmq import Publisher\n\nqueue_args = {\"routing.sample\": \"sample\", \"x-match\": \"all\"}\n\npublisher = Publisher(\n    exchange_name=\"exchange_name\",\n    exchange_type=\"headers\",\n    queue_args=queue_args\n)\n\nmessage_properties = {\"headers\": {\"routing.sample\": \"sample\"}}\npublisher.publish({\"pyrmq\": \"My first message\"}, message_properties=message_properties)\n```\n\nThis is an example of how to publish to a headers exchange that will get routed\nbased on its headers.\n\n#### Binding an exchange to another exchange\nBy default, the `exchange_name` you pass when initializing a `Consumer` is declared and bound to the passed\n`queue_name`. What if you want to bind and declare this exchange to another exchange as well?\n\nThis is done by using `bound_exchange`. This parameter accepts an object with two keys: `name` of your exchange and its\n`type`. Let's take a look at an example to see this in action.\n\n```py\nfrom pyrmq import Consumer\n\ndef callback(data):\n    print(f\"Received {data}!\")\n    raise Exception\n\nconsumer = Consumer(\n    exchange_name=\"direct_exchange\",\n    queue_name=\"direct_queue\",\n    routing_key=\"routing_key\",\n    bound_exchange={\"name\": \"headers_exchange_name\", \"type\": \"headers\"},\n    callback=callback,\n    is_dlk_retry_enabled=True,\n)\nconsumer.start()\n```\n\nIn the example above, we want to consume from an exchange called `direct_exchange` that is directly bound to queue\n`direct_queue`. We want `direct_exchange` to get its messages from another exchange called `headers_exchange_name` of\ntype `headers`. By using `bound_exchange`, PyRMQ declares `direct_exchange` and `direct_queue` along with any queue or\nexchange arguments you may have _first_ then declares the bound exchange next and binds them together. This is done\nto alleviate the need to declare your bound exchange manually.\n\n| :warning: Important                                                                                |\n|:---------------------------------------------------------------------------------------------------|\nSince this method uses [e2e bindings](https://www.rabbitmq.com/e2e.html), if you're using a headers exchange to bind your consumer to, they _and_ your publisher must all have the same routing key to route the messages properly. This is not needed for exchange to queue bindings as the routing key is optional for those.\n\n#### Declaring _classic_ queues\n\nThe default queue type when declaring is quorum which has the advantage of data replication and being highly available. Though these features fit better for highly distributed enterprise systems, it may not fit your certain requirements. You may read more about the differences between the classic and quorum queue types in the official [documentation](https://www.rabbitmq.com/docs/quorum-queues).\n\nTo configure your queue to classic, simply instantiate your queue with the queue argument `x-queue-type` and set its value to `classic`.\n```python\nfrom pyrmq import Publisher\npublisher = Publisher(\n    exchange_name=\"exchange_name\",\n    queue_name=\"queue_name\",\n    routing_key=\"routing_key\",\n    queue_args={\"x-queue-type\": \"classic\"},\n)\n```\n\n\n## Documentation\nVisit https://pyrmq.readthedocs.io for the most up-to-date documentation.\n\n\n## Testing\nFor development, just run:\n```shell script\npytest\n```\nTo test for all the supported Python versions using UV:\n```shell script\nuv tool install tox --with tox-uv \ntox\n```\nTo test for a specific Python version:\n```shell script\ntox -e py311\n```\n\n## Development with UV\n\nThis project uses [UV](https://github.com/astral-sh/uv), a fast Python package installer and resolver written in Rust.\n\n### Basic Setup\n\n```shell script\n# Install UV\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n\n# Create a virtual environment (uses current Python version)\nuv venv\n\n# Activate the virtual environment\nsource .venv/bin/activate  # Linux/macOS\n.venv\\Scripts\\activate     # Windows\n\n# Install the package with development dependencies\nuv add -e .[dev]\n\n# Run tests\nuv run pytest\n```\n\n### Working with Multiple Python Versions\n\n```shell script\n# List available Python versions\nuv python list\n\n# Install a specific Python version\nuv python install 3.8.19\n\n# Create a virtual environment with a specific Python version\nuv venv --python 3.8\n\n# Build with a specific Python version\nuv build --python 3.9\n\n# Run tests with a specific Python version\nuv run --python 3.11 pytest\n```\n\n### Building and Publishing\n\n```shell script\n# Build the package\nuv build\n\n# Publish to PyPI (requires a token)\nuv publish --token YOUR_PYPI_TOKEN\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirst-digital-finance%2Fpyrmq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffirst-digital-finance%2Fpyrmq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffirst-digital-finance%2Fpyrmq/lists"}