{"id":26178682,"url":"https://github.com/loadsmart/django-jaiminho","last_synced_at":"2025-04-14T22:26:57.921Z","repository":{"id":96042540,"uuid":"491269899","full_name":"loadsmart/django-jaiminho","owner":"loadsmart","description":"A broker agnostic implementation of outbox and other message resilience patterns for Django apps.","archived":false,"fork":false,"pushed_at":"2025-04-07T22:44:34.000Z","size":473,"stargazers_count":31,"open_issues_count":0,"forks_count":3,"subscribers_count":47,"default_branch":"master","last_synced_at":"2025-04-07T23:29:10.164Z","etag":null,"topics":["django","dualwrite","orm","outbox","outbox-pattern"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/loadsmart.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":".github/CODEOWNERS","security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-05-11T20:57:18.000Z","updated_at":"2025-04-07T22:43:55.000Z","dependencies_parsed_at":"2024-02-15T20:28:02.654Z","dependency_job_id":"d65a2ecd-c271-47c9-a8bf-4d5b2afea06e","html_url":"https://github.com/loadsmart/django-jaiminho","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loadsmart%2Fdjango-jaiminho","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loadsmart%2Fdjango-jaiminho/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loadsmart%2Fdjango-jaiminho/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/loadsmart%2Fdjango-jaiminho/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/loadsmart","download_url":"https://codeload.github.com/loadsmart/django-jaiminho/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248970705,"owners_count":21191473,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["django","dualwrite","orm","outbox","outbox-pattern"],"created_at":"2025-03-11T21:42:06.379Z","updated_at":"2025-04-14T22:26:57.875Z","avatar_url":"https://github.com/loadsmart.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"# jaiminho\n\n[![CircleCI](https://dl.circleci.com/status-badge/img/gh/loadsmart/django-jaiminho/tree/master.svg?style=svg)](https://dl.circleci.com/status-badge/redirect/gh/loadsmart/django-jaiminho/tree/master)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/python/black)\n\nA broker agnostic implementation of the outbox and other message resilience patterns for Django apps. \n\n![Jaiminho](https://github.com/loadsmart/django-jaiminho/blob/master/assets/jaiminho.jpg?raw=true)\n\n## Getting Started\n\nTo use jaiminho with your project, you just need to do 6 steps:\n\n### 1 - Install it\n\n```sh\npython -m pip install jaiminho\n```\n\n### 2 - Add jaiminho to the INSTALLED_APPS \n\n```python\nINSTALLED_APPS = [\n    ...\n    \"jaiminho\"\n]\n```\n\n### 3 - Run migrations\n\n```sh\npython manage.py migrate\n```\n\n### 4 - Configure jaiminho options in Django settings.py:\n```python\nJAIMINHO_CONFIG = {\n    \"PERSIST_ALL_EVENTS\": False,\n    \"DELETE_AFTER_SEND\": True,\n    \"DEFAULT_ENCODER\": DjangoJSONEncoder,\n    \"PUBLISH_STRATEGY\": \"publish-on-commit\",\n}\n\n```\n\n### 5 - Decorate your functions with @save_to_outbox\n```python\nfrom jaiminho.send import save_to_outbox\n\n@save_to_outbox\ndef any_external_call(**kwargs):\n    # do something\n    return\n```\n\n### 6 - Run the relay events command\n\n```\npython manage.py events_relay --run-in-loop --loop-interval 1\n\n```\n\nIf you don't use `--run-in-loop` option, the relay command will run only 1 time. This is useful in case you want to configure it as a cronjob.\n\n\n## Details\n\nJaiminho `@save_to_outbox` decorator will **intercept** decorated function and **persist** it in a **database table** in the same **transaction** that is active in the decorated function context. The event relay **command**, is a **separated process** that fetches the rows from this table and execute the functions. When an outage happens, the event relay command will **keep retrying until it succeeds**. This way, **eventual consistency is ensured** by design.\n\n### Configuration options\n\n- `PUBLISH_STRATEGY` - Strategy used to publish events (publish-on-commit, keep-order)\n- `PERSIST_ALL_EVENTS` - Saves all events and not only the ones that fail, default is `False`. Only applicable for `{ \"PUBLISH_STRATEGY\": \"publish-on-commit\" }` since all events needs to be stored on keep-order strategy. \n- `DELETE_AFTER_SEND` - Delete the event from the outbox table immediately, after a successful send\n- `DEFAULT_ENCODER` - Default Encoder for the payload (overwritable in the function call)\n\n### Strategies\n\n#### Keep Order\nThis strategy is similar to transactional outbox [described by Chris Richardson](https://microservices.io/patterns/data/transactional-outbox.html). The decorated function intercepts the function call and saves it on the local DB to be executed later. A separate command relayer will keep polling local DB and executing those functions in the same order it was stored. \nBe carefully with this approach, **if any execution fails, the relayer will get stuck** as it would not be possible to guarantee delivery order.  \n\n#### Publish on commit\n\nThis strategy will always execute the decorated function after current transaction commit. With this approach, we don't depend on a relayer (separate process / cronjob) to execute the decorated function and deliver the message. Failed items will only be retried\nthrough relayer. Although this solution has a better performance as only failed items is delivered by the relay command, **we cannot guarantee delivery order**.\n\n\n### Relay Command\nWe already provide a command to relay items from DB, [EventRelayCommand](https://github.com/loadsmart/django-jaiminho/blob/master/jaiminho/management/commands/events_relay.py). The way you should configure depends on the strategy you choose. \nFor example, on **Publish on Commit Strategy** you can configure a cronjob to run every a couple of minutes since only failed items are published by the command relay. If you are using **Keep Order Strategy**, you should run relay command in loop mode as all items will be published by the command, e.g `call_command(events_relay.Command(), run_in_loop=True, loop_interval=0.1)`.  \n\n\n### How to clean older events\n\nYou can use Jaiminho's [EventCleanerCommand](https://github.com/loadsmart/django-jaiminho/blob/master/jaiminho/management/commands/event_cleaner.py) in order to do that. It will query for all events that were sent before a given time interval (e.g. last 7 days) and will delete them from the outbox table.\n\nThe default time interval is `7 days`. You can use the `TIME_TO_DELETE` setting to change it. It should be added to `JAIMINHO_CONFIG` and must be a valid [timedelta](https://docs.python.org/3/library/datetime.html#timedelta-objects).\n\n### Running as cron jobs\n\nYou can run those commands in a cron job. Here are some config examples:\n\n```yaml\n  - name: relay-failed-outbox-events\n    schedule: \"*/15 * * * *\"\n    suspend: false\n    args:\n      - ddtrace-run\n      - python\n      - manage.py\n      - events_relay\n    resources:\n      requests:\n        cpu: 1\n      limits:\n        memory: 384Mi\n\n  - name: delete-old-outbox-events\n    schedule: \"0 5 * * *\"\n    suspend: false\n    args:\n      - ddtrace-run\n      - python\n      - manage.py\n      - event_cleaner\n    resources:\n      requests:\n        cpu: 1\n      limits:\n        memory: 384Mi\n```\n\n### Relay per stream and Overwrite publish strategy\n\nDifferent streams can have different requirements. You can save separate events per streams by using the `@save_to_outbox_stream` decorator:\n\n````python\n@save_to_outbox_stream(\"my-stream\")\ndef any_external_call(payload, **kwargs):\n    # do something\n    pass\n````\n\nyou can also overwrite publish strategy configure on settings:\n\n````python\n@save_to_outbox_stream(\"my-stream\", PublishStrategyType.KEEP_ORDER)\ndef any_external_call(payload, **kwargs):\n    # do something\n    pass\n````\n\nAnd then, run relay command with stream filter option\n````shell\npython manage.py relay_event True 0.1 my-stream\n````\n\nIn the example above, `True` is the option for run_in_loop; `0.1` for loop_interval; and `my_stream` is the name of the stream.\n\n### Signals\n\nJaiminho triggers the following Django signals:\n\n| Signal                  | Description                                                                     |\n|-------------------------|---------------------------------------------------------------------------------|\n| event_published         | Triggered when an event is sent successfully                                    |\n| event_failed_to_publish | Triggered when an event is not sent, being added to the Outbox table queue      |\n\n\n### How to collect metrics from Jaiminho?\n\nYou could use the Django signals triggered by Jaiminho to collect metrics. \nConsider the following code as example:\n\n````python\nfrom django.dispatch import receiver\n\n@receiver(event_published)\ndef on_event_sent(sender, event_payload, **kwargs):\n    metrics.count(f\"event_sent_successfully {event_payload.get('type')}\")\n\n@receiver(event_failed_to_publish)\ndef on_event_send_error(sender, event_payload, **kwargs):\n    metrics.count(f\"event_failed {event_payload.get('type')}\")\n\n````\n\n## Development\n\nCreate a virtualenv\n\n```bash\nvirtualenv venv\npip install -r requirements-dev.txt\ntox -e py39\n```\n## Collaboration\n\nIf you want to improve or suggest improvements, check our [CONTRIBUTING.md](https://github.com/loadsmart/django-jaiminho/blob/master/CONTRIBUTING.md) file.\n\n\n## License\n\nThis project is licensed under MIT License.\n\n## Security\n\nIf you have any security concern or report feel free to reach out to security@loadsmart.com;\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floadsmart%2Fdjango-jaiminho","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Floadsmart%2Fdjango-jaiminho","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Floadsmart%2Fdjango-jaiminho/lists"}