{"id":28440585,"url":"https://github.com/allisson/pysqsx","last_synced_at":"2025-06-27T14:31:28.577Z","repository":{"id":212410661,"uuid":"731094674","full_name":"allisson/pysqsx","owner":"allisson","description":"A simple task processor for Amazon SQS","archived":false,"fork":false,"pushed_at":"2025-05-03T12:52:48.000Z","size":111,"stargazers_count":6,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-06-06T04:06:11.244Z","etag":null,"topics":["hacktoberfest","sqs","sqs-consumer","sqs-queue"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/allisson.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2023-12-13T10:48:29.000Z","updated_at":"2025-05-03T12:51:49.000Z","dependencies_parsed_at":"2025-06-06T20:01:11.708Z","dependency_job_id":null,"html_url":"https://github.com/allisson/pysqsx","commit_stats":null,"previous_names":["allisson/pysqsx"],"tags_count":9,"template":false,"template_full_name":null,"purl":"pkg:github/allisson/pysqsx","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allisson%2Fpysqsx","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allisson%2Fpysqsx/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allisson%2Fpysqsx/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allisson%2Fpysqsx/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/allisson","download_url":"https://codeload.github.com/allisson/pysqsx/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/allisson%2Fpysqsx/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":262273381,"owners_count":23285669,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["hacktoberfest","sqs","sqs-consumer","sqs-queue"],"created_at":"2025-06-06T04:06:08.942Z","updated_at":"2025-06-27T14:31:28.571Z","avatar_url":"https://github.com/allisson.png","language":"Python","readme":"# sqsx\n[![Tests](https://github.com/allisson/pysqsx/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/allisson/pysqsx/actions/workflows/tests.yml)\n![PyPI - Version](https://img.shields.io/pypi/v/sqsx)\n![PyPI - Python Version](https://img.shields.io/pypi/pyversions/sqsx)\n![GitHub License](https://img.shields.io/github/license/allisson/pysqsx)\n\nA simple task processor for Amazon SQS.\n\n## Quickstart\n\nFor this demonstration we will use elasticmq locally using docker:\n\n```bash\ndocker run --name pysqsx-elasticmq -p 9324:9324 -d softwaremill/elasticmq-native\n```\n\nInstall the package:\n\n```bash\npip install sqsx\n```\n\n### Working with sqsx.Queue\n\nWe use sqsx.Queue when we need to work with scheduling and consuming tasks.\n\nNow let's create a script that will create a new task and we will consume them:\n\n```python\n# file script.py\nimport logging\n\nimport boto3\n\nfrom sqsx import Queue\n\n# configure the logging\nlogging.basicConfig(level=logging.DEBUG)\nlogging.getLogger('botocore').setLevel(logging.CRITICAL)\nlogging.getLogger('urllib3').setLevel(logging.CRITICAL)\n\n# create the sqs_client\nqueue_url = \"http://localhost:9324/000000000000/tests\"\nqueue_name = \"tests\"\nsqs_client = boto3.client(\n    \"sqs\",\n    endpoint_url=\"http://localhost:9324\",\n    region_name=\"elasticmq\",\n    aws_secret_access_key=\"x\",\n    aws_access_key_id=\"x\",\n    use_ssl=False,\n)\n\n# create the new sqs queue\nsqs_client.create_queue(QueueName=queue_name)\n\n# create the sqsx.Queue\nqueue = Queue(url=queue_url, sqs_client=sqs_client)\n\n# add a new task\nqueue.add_task(\"my_task\", a=1, b=2, c=3)\n\n# create the task handler, which must be a simple function like this\ndef task_handler(context: dict, a: int, b: int, c: int):\n    print(f\"context={context}, a={a}, b={b}, c={c}\")\n\n# add a new task handler\nqueue.add_task_handler(\"my_task\", task_handler)\n\n# start the consumption of messages, to stop press ctrl+c to exit gracefully\nqueue.consume_messages()\n```\n\nRunning the script:\n\n```bash\npython script.py\nINFO:sqsx.queue:Starting consuming tasks, queue_url=http://localhost:9324/000000000000/tests\ncontext={'queue_url': 'http://localhost:9324/000000000000/tests', 'task_name': 'my_task', 'sqs_message': {'MessageId': '42513c2d-ac93-4701-bb63-83b45e6fe2ca', 'ReceiptHandle': '42513c2d-ac93-4701-bb63-83b45e6fe2ca#6eb5443b-a2eb-454e-8619-86f6d2e67561', 'MD5OfBody': '8087eb7436895841c5d646156a8a469f', 'Body': 'eyJrd2FyZ3MiOiB7ImEiOiAxLCAiYiI6IDIsICJjIjogM319', 'Attributes': {'SentTimestamp': '1702573178736', 'ApproximateReceiveCount': '1', 'ApproximateFirstReceiveTimestamp': '1702573178740', 'SenderId': '127.0.0.1'}, 'MD5OfMessageAttributes': '5346f2cd7c539a880febaf9112a86921', 'MessageAttributes': {'TaskName': {'StringValue': 'my_task', 'DataType': 'String'}}}}, a=1, b=2, c=3\nDEBUG:sqsx.queue:Waiting some seconds because no message was received, seconds=10, queue_url=http://localhost:9324/000000000000/tests\nDEBUG:sqsx.queue:Waiting some seconds because no message was received, seconds=10, queue_url=http://localhost:9324/000000000000/tests\n^CINFO:sqsx.queue:Starting graceful shutdown process\nINFO:sqsx.queue:Stopping consuming tasks, queue_url=http://localhost:9324/000000000000/tests\n```\n\n### Working with sqsx.RawQueue\n\nWe use sqsx.RawQueue when we need to work with one handler consuming all the queue messages.\n\nNow let's create a script that will create a new message and we will consume them:\n\n```python\n# file raw_script.py\nimport logging\n\nimport boto3\n\nfrom sqsx import RawQueue\n\n# configure the logging\nlogging.basicConfig(level=logging.DEBUG)\nlogging.getLogger('botocore').setLevel(logging.CRITICAL)\nlogging.getLogger('urllib3').setLevel(logging.CRITICAL)\n\n# create the sqs_client\nqueue_url = \"http://localhost:9324/000000000000/tests\"\nqueue_name = \"tests\"\nsqs_client = boto3.client(\n    \"sqs\",\n    endpoint_url=\"http://localhost:9324\",\n    region_name=\"elasticmq\",\n    aws_secret_access_key=\"x\",\n    aws_access_key_id=\"x\",\n    use_ssl=False,\n)\n\n# create the new sqs queue\nsqs_client.create_queue(QueueName=queue_name)\n\n# create the message handler, which must be a simple function like this\ndef message_handler(queue_url: str, sqs_message: dict):\n    print(f\"queue_url={queue_url}, sqs_message={sqs_message}\")\n\n# create the sqsx.Queue\nqueue = RawQueue(url=queue_url, message_handler_function=message_handler, sqs_client=sqs_client)\n\n# add a new message\nqueue.add_message(\n    message_body=\"My Message\",\n    message_attributes={\"Attr1\": {\"DataType\": \"String\", \"StringValue\": \"Attr1\"}},\n)\n\n# start the consumption of messages, to stop press ctrl+c to exit gracefully\nqueue.consume_messages()\n```\n\nRunning the script:\n\n```bash\nINFO:sqsx.queue:Starting consuming tasks, queue_url=http://localhost:9324/000000000000/tests\nqueue_url=http://localhost:9324/000000000000/tests, sqs_message={'MessageId': 'fb2ed6cf-9346-4ded-8cfe-4fc297f95928', 'ReceiptHandle': 'fb2ed6cf-9346-4ded-8cfe-4fc297f95928#bd9f27a6-0a73-4d27-9c1e-0947f21d3c02', 'MD5OfBody': '069840f6917e85a02167febb964f0041', 'Body': 'My Message', 'Attributes': {'SentTimestamp': '1702573585302', 'ApproximateReceiveCount': '1', 'ApproximateFirstReceiveTimestamp': '1702573585306', 'SenderId': '127.0.0.1'}, 'MD5OfMessageAttributes': '90f34a800b9d242c1b32320e4a3ed630', 'MessageAttributes': {'Attr1': {'StringValue': 'Attr1', 'DataType': 'String'}}}\nDEBUG:sqsx.queue:Waiting some seconds because no message was received, seconds=10, queue_url=http://localhost:9324/000000000000/tests\nDEBUG:sqsx.queue:Waiting some seconds because no message was received, seconds=10, queue_url=http://localhost:9324/000000000000/tests\n^CINFO:sqsx.queue:Starting graceful shutdown process\nINFO:sqsx.queue:Stopping consuming tasks, queue_url=http://localhost:9324/000000000000/tests\n```\n\n### Working with exceptions\n\nThe default behavior is to retry the message when an exception is raised, you can change this behavior using the exceptions sqsx.exceptions.Retry and sqsx.exceptions.NoRetry.\n\nIf you want to change the backoff policy, use the sqsx.exceptions.Retry like this:\n\n```python\nfrom sqsx.exceptions import Retry\n\n# to use with sqsx.Queue and change the default backoff policy\ndef task_handler(context: dict, a: int, b: int, c: int):\n    raise Retry(min_backoff_seconds=100, max_backoff_seconds=200)\n\n# to use with sqsx.RawQueue and change the default backoff policy\ndef message_handler(queue_url: str, sqs_message: dict):\n    raise Retry(min_backoff_seconds=100, max_backoff_seconds=200)\n```\n\nIf you want to remove the task or message from the queue use the sqsx.exceptions.NoRetry like this:\n\n```python\nfrom sqsx.exceptions import NoRetry\n\n# to use with sqsx.Queue and remove the task\ndef task_handler(context: dict, a: int, b: int, c: int):\n    raise NoRetry()\n\n# to use with sqsx.RawQueue and remove the message\ndef message_handler(queue_url: str, sqs_message: dict):\n    raise NoRetry()\n```\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallisson%2Fpysqsx","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fallisson%2Fpysqsx","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fallisson%2Fpysqsx/lists"}