{"id":21258102,"url":"https://github.com/art1415926535/comlink","last_synced_at":"2025-07-24T09:32:03.996Z","repository":{"id":59685691,"uuid":"537798793","full_name":"art1415926535/comlink","owner":"art1415926535","description":"Send and receive messages by using SQS queues.","archived":false,"fork":false,"pushed_at":"2022-09-24T15:56:34.000Z","size":59,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-22T07:43:36.443Z","etag":null,"topics":["asyncio","python","python3","sqs","sqs-queue","task-queue"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/art1415926535.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2022-09-17T12:17:22.000Z","updated_at":"2024-02-12T11:26:22.000Z","dependencies_parsed_at":"2022-09-19T20:24:52.666Z","dependency_job_id":null,"html_url":"https://github.com/art1415926535/comlink","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/art1415926535/comlink","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/art1415926535%2Fcomlink","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/art1415926535%2Fcomlink/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/art1415926535%2Fcomlink/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/art1415926535%2Fcomlink/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/art1415926535","download_url":"https://codeload.github.com/art1415926535/comlink/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/art1415926535%2Fcomlink/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266663032,"owners_count":23964672,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-23T02:00:09.312Z","response_time":66,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["asyncio","python","python3","sqs","sqs-queue","task-queue"],"created_at":"2024-11-21T04:07:11.666Z","updated_at":"2025-07-24T09:32:03.384Z","avatar_url":"https://github.com/art1415926535.png","language":"Python","readme":"![Comlink logo](https://raw.githubusercontent.com/art1415926535/comlink/main/assets/logo.svg)\n\nSend and receive messages by using SQS queues.\n\n[![PyPI version](https://badge.fury.io/py/comlink.svg)](https://pypi.org/project/comlink)\n[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/comlink)](https://pypi.org/project/comlink)\n[![PyPI - License](https://img.shields.io/pypi/l/comlink)](https://github.com/art1415926535/comlink/blob/main/LICENSE)\n\n```bash\npip install comlink\n```\n\n```bash\npoetry add comlink\n```\n\n## Example\n\n```python\nfrom comlink import SqsConsumer, SqsQueue, signal_event\n\n\nasync def example(queue_url, sqs_client):\n    # Create a queue object\n    sqs_queue = SqsQueue(url=queue_url, client=sqs_client)\n\n    # Event for stopping the consumer by receiving os signal.\n    # stop_event waits for SIGINT(Ctrl+C) or SIGTERM by default\n    stop_event = signal_event()\n    # Create a consumer with a handler that just prints the message\n    consumer = SqsConsumer(queue=sqs_queue, handler=print)\n    # Start the consumer\n    consumer_task = await consumer.start(stop_event=stop_event)\n\n    # Send a message to the queue\n    await sqs_queue.put(\"Hello, world!\")\n\n    # Wait for the consumer to stop by receiving os signal\n    await consumer_task\n```\n\nMore examples can be found in the [examples](https://github.com/art1415926535/comlink/tree/main/examples) directory.\n\n## Docs\n[SqsQueue](#comlinksqsqueueurl-client-serializernone-deserializernone)\n* [coroutine put(data, **kwargs)](#coroutine-putdata-kwargs)\n* [coroutine take(max_messages, visibility_timeout, wait_time_seconds, **kwargs)](#coroutine-takemax_messages-visibility_timeout-wait_time_seconds-kwargs)\n* [remove(receipt_handle, **kwargs)](#coroutine-removereceipt_handle-kwargs)\n\n[SqsConsumer](#comlinksqsconsumerqueue-handler-batch_size1-visibility_timeout120-wait_time_seconds20)\n* [coroutine start(stop_event)](#coroutine-startstop_event)\n\n[signal_event](#comlinksignal_eventsignalsnone-log_levelloggingwarning-loopnone)\n\n\u003chr\u003e\n\n\n### `comlink.SqsQueue(url, client, serializer=None, deserializer=None)`\nIt is a wrapper around aiobotocore's SQS client. It provides a simple interface for sending and receiving messages.\n\nIn most cases, you will need to pass a instance of this class to the `comlink.SqsConsumer` and send messages to it using the `put` method.\n\n`take` and `remove` methods are used by the `SqsConsumer` class. But you can use them if you need to.\n\nParameters:\n* `url: str` - URL of the queue\n* `client: Any` - aiobotocore's SQS client\n* `serializer: Callable[[Message], str] | None` - a function that serializes the message to string\n* `deserializer: Callable[[str], Message] | None` - a function that deserializes the message from string\n\n#### Example\n```python\n\nimport json\nfrom comlink import SqsQueue\n\nasync def example(queue_url, sqs_client):\n    sqs_queue = SqsQueue(url=queue_url, client=sqs_client, serializer=json.dumps, deserializer=json.loads)\n\n    await sqs_queue.put({\"hello\": \"world\"})\n    \n    message = await sqs_queue.take(max_messages=1, visibility_timeout=10, wait_time_seconds=0)\n    print(message[0][\"Body\"])\n    \n    await sqs_queue.remove(message[0][\"ReceiptHandle\"])\n```\n\n\u003chr\u003e\n\n#### coroutine `put(data, **kwargs)`\nSends a message to the queue.  \nParameters:\n* `data: Message` - message data. If `serializer` is provided, it will be called with `data` as an argument.\n* `kwargs` - additional arguments for aiobotocore's `send_message` method. E.g. `DelaySeconds`, `MessageGroupId`, `MessageDeduplicationId` and etc.\n\nReturns result of aiobotocore's `send_message` method.\n\n\u003chr\u003e\n\n#### coroutine `take(max_messages, visibility_timeout, wait_time_seconds, **kwargs)`\nGet messages from the queue.  \nParameters:\n* `max_messages: int` - maximum number of messages to receive\n* `visibility_timeout: int` - the duration (in seconds) that the received messages are hidden from subsequent retrieve requests after being retrieved by a `take` request\n* `wait_time_seconds: int` - the duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than `wait_time_seconds`. If no messages are available and the wait time expires, the call returns successfully with an empty list of messages.\n* `kwargs` - additional arguments for aiobotocore's `receive_message` method. E.g. `AttributeNames`, `MessageAttributeNames` and etc.\n\nReturns list of messages that were received from the queue.\n\n\u003chr\u003e\n\n#### coroutine `remove(receipt_handle, **kwargs)`\nDeletes the specified message from the queue.  \nParameters:\n* `receipt_handle: str` - the receipt handle associated with the message to delete (from `take` method)\n\nReturns result of aiobotocore's `delete_message` method.\n\n\u003chr\u003e\n\n### `comlink.SqsConsumer(queue, handler, batch_size=1, visibility_timeout=120, wait_time_seconds=20)`\nIt is a consumer that receives messages from the queue and passes them to the handler.\n\nInit function only creates a consumer object. To start the consumer, you need to call the `start` method.\n\nParameters:\n* `queue: SqsQueue` - queue object\n* `handler: Callable` - a function that will be called with a message as an argument. It may be a regular function, a coroutine or an class instance with `__call__` method.\n* `batch_size: int` - maximum number of messages to receive from the queue at once\n* `visibility_timeout: int` - the duration (in seconds) that the received messages are hidden from subsequent retrieve requests after being retrieved by a `comlink.SqsQueue.take` request\n* `wait_time_seconds: int` - the duration (in seconds) for which the call waits for a message to arrive in the queue before returning. If a message is available, the call returns sooner than `wait_time_seconds`.\n\n#### Example\n```python\nfrom comlink import SqsQueue, SqsConsumer, signal_event\n\nasync def handler(message: str):\n    print(\"Received message:\", message)\n\nasync def example(sqs_queue: SqsQueue):\n    stop_event = signal_event()\n    \n    consumer = SqsConsumer(queue=sqs_queue, handler=handler)\n    consumer_task = await consumer.start(stop_event=stop_event)\n    \n    await consumer_task\n```\n\n\u003chr\u003e\n\n#### coroutine `start(stop_event)`\nStarts the consumer.\n\nParameters:\n* `stop_event: asyncio.Event` - a consumer will stop when this event is set. You can use `comlink.signal_event` function to create an event that will be set when the process receives a signal.\n\nReturns a consumer task. You can use it to wait for the consumer to stop. If the consumer is stopped by an exception, the task will raise an exception. If the consumer is stopped by the `stop_event`, the task will return `None`.\n\n\u003chr\u003e\n\n### `comlink.signal_event(signals=None, log_level=logging.WARNING, loop=None)`\nCreates an event that will be set when the process receives a signal.\n\nParameters:\n* `signals: dict[int, str] | None` - a dictionary that maps signal numbers to signal names. If `None`, the default dictionary will be used.\n* `log_level: int` - a log level that will be used to log signals. Default is `logging.WARNING`.\n* `loop: asyncio.AbstractEventLoop | None` - an event loop. If `None`, the current event loop will be used.\n\nReturns an [asyncio.Event](https://docs.python.org/3/library/asyncio-sync.html#asyncio.Event) object.\n\n#### Example\n```python\nimport logging\nimport signal\nfrom comlink import signal_event\n\n\nasync def example():\n    stop_event = signal_event(\n        signals={\n            signal.SIGUSR1: \"SIGUSR1\",\n            signal.SIGINT: \"SIGINT\",\n            signal.SIGTERM: \"SIGTERM\",\n        },\n        log_level=logging.DEBUG,\n    )\n    \n    await stop_event.wait()\n    print(\"Received a signal\")\n```\n\n\u003chr\u003e\n\n\n## Development\n\n### Setup\n\n1. Install [Poetry](https://python-poetry.org/).\n1. Install dependencies with `poetry install`.\n1. Install [Docker](https://www.docker.com/).\n1. Run `docker compose -f docker-compose.dev.yml up -d` to start \nthe development environment (localstack). Tests will fail until the environment is up and running.\n\n\n### Testing\n\nRun `poetry run pytest` to run the tests.\n\n\n### Formatting\n\nRun `poetry run black .` to format the code.\n\nRun `poetry run isort .` to sort the imports.","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fart1415926535%2Fcomlink","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fart1415926535%2Fcomlink","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fart1415926535%2Fcomlink/lists"}