{"id":19586306,"url":"https://github.com/tbobm/payler","last_synced_at":"2025-07-17T00:07:20.993Z","repository":{"id":46281735,"uuid":"317934482","full_name":"tbobm/payler","owner":"tbobm","description":"Temporize job execution, delay messages without asking your broker too much.","archived":false,"fork":false,"pushed_at":"2021-11-02T13:35:54.000Z","size":6397,"stargazers_count":5,"open_issues_count":4,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2025-06-24T19:06:16.215Z","etag":null,"topics":["asyncio","cli","docker","message-broker","python","spool"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tbobm.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":"CODEOWNERS","security":null,"support":null}},"created_at":"2020-12-02T17:10:57.000Z","updated_at":"2021-11-02T13:35:56.000Z","dependencies_parsed_at":"2022-09-16T03:51:41.852Z","dependency_job_id":null,"html_url":"https://github.com/tbobm/payler","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/tbobm/payler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fpayler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fpayler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fpayler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fpayler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tbobm","download_url":"https://codeload.github.com/tbobm/payler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tbobm%2Fpayler/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":265553429,"owners_count":23787074,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","cli","docker","message-broker","python","spool"],"created_at":"2024-11-11T07:59:20.379Z","updated_at":"2025-07-17T00:07:20.912Z","avatar_url":"https://github.com/tbobm.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Payler\n\n[![Build Status](https://github.com/tbobm/payler/workflows/payler/badge.svg)](https://github.com/tbobm/payler/workflows/payler/)\n\n[![Renovate](https://img.shields.io/badge/renovate-enabled-green?style=for-the-badge\u0026logo=data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAzNjkgMzY5Ij48Y2lyY2xlIGN4PSIxODkuOSIgY3k9IjE5MC4yIiByPSIxODQuNSIgZmlsbD0iI2ZmZTQyZSIgdHJhbnNmb3JtPSJ0cmFuc2xhdGUoLTUgLTYpIi8+PHBhdGggZmlsbD0iIzhiYjViNSIgZD0iTTI1MSAyNTZsLTM4LTM4YTE3IDE3IDAgMDEwLTI0bDU2LTU2YzItMiAyLTYgMC03bC0yMC0yMWE1IDUgMCAwMC03IDBsLTEzIDEyLTktOCAxMy0xM2ExNyAxNyAwIDAxMjQgMGwyMSAyMWM3IDcgNyAxNyAwIDI0bC01NiA1N2E1IDUgMCAwMDAgN2wzOCAzOHoiLz48cGF0aCBmaWxsPSIjZDk1NjEyIiBkPSJNMzAwIDI4OGwtOCA4Yy00IDQtMTEgNC0xNiAwbC00Ni00NmMtNS01LTUtMTIgMC0xNmw4LThjNC00IDExLTQgMTUgMGw0NyA0N2M0IDQgNCAxMSAwIDE1eiIvPjxwYXRoIGZpbGw9IiMyNGJmYmUiIGQ9Ik04MSAxODVsMTgtMTggMTggMTgtMTggMTh6Ii8+PHBhdGggZmlsbD0iIzI1YzRjMyIgZD0iTTIyMCAxMDBsMjMgMjNjNCA0IDQgMTEgMCAxNkwxNDIgMjQwYy00IDQtMTEgNC0xNSAwbC0yNC0yNGMtNC00LTQtMTEgMC0xNWwxMDEtMTAxYzUtNSAxMi01IDE2IDB6Ii8+PHBhdGggZmlsbD0iIzFkZGVkZCIgZD0iTTk5IDE2N2wxOC0xOCAxOCAxOC0xOCAxOHoiLz48cGF0aCBmaWxsPSIjMDBhZmIzIiBkPSJNMjMwIDExMGwxMyAxM2M0IDQgNCAxMSAwIDE2TDE0MiAyNDBjLTQgNC0xMSA0LTE1IDBsLTEzLTEzYzQgNCAxMSA0IDE1IDBsMTAxLTEwMWM1LTUgNS0xMSAwLTE2eiIvPjxwYXRoIGZpbGw9IiMyNGJmYmUiIGQ9Ik0xMTYgMTQ5bDE4LTE4IDE4IDE4LTE4IDE4eiIvPjxwYXRoIGZpbGw9IiMxZGRlZGQiIGQ9Ik0xMzQgMTMxbDE4LTE4IDE4IDE4LTE4IDE4eiIvPjxwYXRoIGZpbGw9IiMxYmNmY2UiIGQ9Ik0xNTIgMTEzbDE4LTE4IDE4IDE4LTE4IDE4eiIvPjxwYXRoIGZpbGw9IiMyNGJmYmUiIGQ9Ik0xNzAgOTVsMTgtMTggMTggMTgtMTggMTh6Ii8+PHBhdGggZmlsbD0iIzFiY2ZjZSIgZD0iTTYzIDE2N2wxOC0xOCAxOCAxOC0xOCAxOHpNOTggMTMxbDE4LTE4IDE4IDE4LTE4IDE4eiIvPjxwYXRoIGZpbGw9IiMzNGVkZWIiIGQ9Ik0xMzQgOTVsMTgtMTggMTggMTgtMTggMTh6Ii8+PHBhdGggZmlsbD0iIzFiY2ZjZSIgZD0iTTE1MyA3OGwxOC0xOCAxOCAxOC0xOCAxOHoiLz48cGF0aCBmaWxsPSIjMzRlZGViIiBkPSJNODAgMTEzbDE4LTE3IDE4IDE3LTE4IDE4ek0xMzUgNjBsMTgtMTggMTggMTgtMTggMTh6Ii8+PHBhdGggZmlsbD0iIzk4ZWRlYiIgZD0iTTI3IDEzMWwxOC0xOCAxOCAxOC0xOCAxOHoiLz48cGF0aCBmaWxsPSIjYjUzZTAyIiBkPSJNMjg1IDI1OGw3IDdjNCA0IDQgMTEgMCAxNWwtOCA4Yy00IDQtMTEgNC0xNiAwbC02LTdjNCA1IDExIDUgMTUgMGw4LTdjNC01IDQtMTIgMC0xNnoiLz48cGF0aCBmaWxsPSIjOThlZGViIiBkPSJNODEgNzhsMTgtMTggMTggMTgtMTggMTh6Ii8+PHBhdGggZmlsbD0iIzAwYTNhMiIgZD0iTTIzNSAxMTVsOCA4YzQgNCA0IDExIDAgMTZMMTQyIDI0MGMtNCA0LTExIDQtMTUgMGwtOS05YzUgNSAxMiA1IDE2IDBsMTAxLTEwMWM0LTQgNC0xMSAwLTE1eiIvPjxwYXRoIGZpbGw9IiMzOWQ5ZDgiIGQ9Ik0yMjggMTA4bC04LThjLTQtNS0xMS01LTE2IDBMMTAzIDIwMWMtNCA0LTQgMTEgMCAxNWw4IDhjLTQtNC00LTExIDAtMTVsMTAxLTEwMWM1LTQgMTItNCAxNiAweiIvPjxwYXRoIGZpbGw9IiNhMzM5MDQiIGQ9Ik0yOTEgMjY0bDggOGM0IDQgNCAxMSAwIDE2bC04IDdjLTQgNS0xMSA1LTE1IDBsLTktOGM1IDUgMTIgNSAxNiAwbDgtOGM0LTQgNC0xMSAwLTE1eiIvPjxwYXRoIGZpbGw9IiNlYjZlMmQiIGQ9Ik0yNjAgMjMzbC00LTRjLTYtNi0xNy02LTIzIDAtNyA3LTcgMTcgMCAyNGw0IDRjLTQtNS00LTExIDAtMTZsOC04YzQtNCAxMS00IDE1IDB6Ii8+PHBhdGggZmlsbD0iIzEzYWNiZCIgZD0iTTEzNCAyNDhjLTQgMC04LTItMTEtNWwtMjMtMjNhMTYgMTYgMCAwMTAtMjNMMjAxIDk2YTE2IDE2IDAgMDEyMiAwbDI0IDI0YzYgNiA2IDE2IDAgMjJMMTQ2IDI0M2MtMyAzLTcgNS0xMiA1em03OC0xNDdsLTQgMi0xMDEgMTAxYTYgNiAwIDAwMCA5bDIzIDIzYTYgNiAwIDAwOSAwbDEwMS0xMDFhNiA2IDAgMDAwLTlsLTI0LTIzLTQtMnoiLz48cGF0aCBmaWxsPSIjYmY0NDA0IiBkPSJNMjg0IDMwNGMtNCAwLTgtMS0xMS00bC00Ny00N2MtNi02LTYtMTYgMC0yMmw4LThjNi02IDE2LTYgMjIgMGw0NyA0NmM2IDcgNiAxNyAwIDIzbC04IDhjLTMgMy03IDQtMTEgNHptLTM5LTc2Yy0xIDAtMyAwLTQgMmwtOCA3Yy0yIDMtMiA3IDAgOWw0NyA0N2E2IDYgMCAwMDkgMGw3LThjMy0yIDMtNiAwLTlsLTQ2LTQ2Yy0yLTItMy0yLTUtMnoiLz48L3N2Zz4=)](https://renovatebot.com)\n\n\u003e The Payload Spooler\n\n_Send your payload now, treat it later._\n\n## What is this?\n\nPayler is an asyncio-based Python application intended to provide a way of delaying message execution. The goal of this program is to reduce the workload on your existing message broker solution (Only RabbitMQ is currently supported, but other message-brokers can be easily implemented) by putting the payloads in a storage backend which will then be polled to re-inject payloads in the corresponding destination.\n\n## Installation\n\nThrough pypi:\n```console\n$ pip install payler\n```\n\nThrough [poetry](https://github.com/python-poetry/poetry):\n```console\n$ git clone https://github.com/tbobm/payler\n$ cd payler\n$ poetry install\n```\n\n## How to use this\n\nUsing the command line:\n\n1. Specify the input and output URLs for your drivers (see [configuration](#configuration))\n2. (optional) Customize the configuration to suit your needs _currently the example configuration is the only valid one_\n3. Run payler `payler --config-file configuration.yaml`\n\nUsing the docker image:\n\n1. Pull the docker image `docker pull ghcr.io/tbobm/payler:latest`\n2. (optional) Customize the configuration to suit your needs _currently the example configuration is the only valid one_ (mount the configuration file into the volume at `/configuration.yaml`)\n3. Run the docker image and provide environment variables `docker run -d --name payler -e BROKER_URL=\"amqp://payler:secret@my-broker/\" -e MONGODB_URL=\"mongodb://payler:secret@my-mongo/payler\" ghcr.io/tbobm/payler`\n\n## Configuration\n\nIn order to configure the different workflows, payler uses a configuration file (see [configuration.yml](./configuration.yml)).\n\nExample config file:\n\n```yaml\n---\nworkflows:\n  - name: \"Fetch payloads from RabbitMQ and store them in MongoDB\"\n    location: \"payler\"\n    callable: \"client.process_queue\"\n  - name: \"Re-injects payloads to RabbitMQ\"\n    callable: \"client.watch_storage\"\n```\n\nThe `workflows[].name` attribute is currently unused, but will offer a more human-friendly way of getting informed about a workflow's state.\nThe `workflows[].location` corresponds to the package where the `workflows[].callable` can be found. It defaults to `payler`, but can this is a way of offering a dumb and simple plugin mechanism by creating function matching the following signature:\n\n```python\nasync def my_workflow(loop: asyncio.AbstractEventLoop) -\u003e None:\n    \"\"\"My user-defined workflow.\"\"\"\n    # configure your driver(s)\n    input_driver.serve()\n```\n\n## Features\n\n- Forward messages between multiple datasources\n- Based on asyncio (benchmarks are on the roadmap)\n- Extend using your own implementation of the [`BaseDriver`](./payler/driver.py) class\n\n### Drivers\n\ndriver | process | serve\n------|------|------\nBrokerManager | Send a `Payload` to a Queue | Consume a queue's messages\nSpoolerManager | Store a `Payload` in a Collection | Fetch documents with a specific reference data\n\n## Testing\n\nThis project has unittests with [pytest](https://docs.pytest.org/en/latest/).\n\nYou can run the tests using:\n```\npoetry run pytest\n```\n\n## Contributing\n\nFeel free to open new issues for feature requests and bug reports in the [issue page](github.com/tbobm/payler/issues/new) and even create PRs if you feel like it.\n\nThis project is linted with `pylint` with some minor adjustments (see the [pyproject.toml](./pyproject.toml)).\n\n## Note\n\nThis side-project is born from the following:\n- I wanted to experiment with Python's `asyncio`\n- A friend of mine had issues with delaying lots of messages using RabbitMQ's [delayed exchange plugin](https://github.com/rabbitmq/rabbitmq-delayed-message-exchange)\n- I was looking for a concrete use-case to work with Github Actions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftbobm%2Fpayler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftbobm%2Fpayler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftbobm%2Fpayler/lists"}