{"id":19385770,"url":"https://github.com/conduitframework/conduit_sqs","last_synced_at":"2025-04-23T22:32:04.910Z","repository":{"id":48289894,"uuid":"79071643","full_name":"conduitframework/conduit_sqs","owner":"conduitframework","description":null,"archived":false,"fork":false,"pushed_at":"2021-08-03T05:22:37.000Z","size":151,"stargazers_count":11,"open_issues_count":12,"forks_count":9,"subscribers_count":1,"default_branch":"master","last_synced_at":"2025-04-17T01:26:50.349Z","etag":null,"topics":["aws","conduit","elixir","message-queue","sqs"],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/conduitframework.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-01-16T01:26:41.000Z","updated_at":"2023-09-08T17:19:28.000Z","dependencies_parsed_at":"2022-08-26T11:11:08.860Z","dependency_job_id":null,"html_url":"https://github.com/conduitframework/conduit_sqs","commit_stats":null,"previous_names":[],"tags_count":12,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conduitframework%2Fconduit_sqs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conduitframework%2Fconduit_sqs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conduitframework%2Fconduit_sqs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conduitframework%2Fconduit_sqs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/conduitframework","download_url":"https://codeload.github.com/conduitframework/conduit_sqs/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250527149,"owners_count":21445313,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["aws","conduit","elixir","message-queue","sqs"],"created_at":"2024-11-10T10:02:59.080Z","updated_at":"2025-04-23T22:32:04.903Z","avatar_url":"https://github.com/conduitframework.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ConduitSQS\n\n[![CircleCI](https://img.shields.io/circleci/project/github/conduitframework/conduit_sqs.svg?style=flat-square)](https://circleci.com/gh/conduitframework/conduit_sqs)\n[![Coveralls](https://img.shields.io/coveralls/conduitframework/conduit_sqs.svg?style=flat-square)](https://coveralls.io/github/conduitframework/conduit_sqs)\n[![Hex.pm](https://img.shields.io/hexpm/v/conduit_sqs.svg?style=flat-square)](https://hex.pm/packages/conduit_sqs)\n[![Hex.pm](https://img.shields.io/hexpm/l/conduit_sqs.svg?style=flat-square)](https://github.com/conduitframework/conduit_sqs/blob/master/LICENSE.md)\n[![Hex.pm](https://img.shields.io/hexpm/dt/conduit_sqs.svg?style=flat-square)](https://hex.pm/packages/conduit_sqs)\n[![Gratipay](https://img.shields.io/gratipay/blatyo.svg?style=flat-square)](https://gratipay.com/blatyo)\n\nA [Conduit](https://github.com/conduitframework/conduit) adapter for SQS.\n\n## Installation\n\nThe package can be installed by adding `conduit_sqs` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [{:conduit_sqs, \"~\u003e 0.3.0\"}]\nend\n```\n\n## Configuring the Adapter\n\n``` elixir\nconfig :my_app, MyApp.Broker,\n  adapter: ConduitSQS,\n  access_key_id: [{:system, \"AWS_ACCESS_KEY_ID\"}, :instance_role],\n  secret_access_key: [{:system, \"AWS_SECRET_ACCESS_KEY\"}, :instance_role]\n```\n\n### Options\n\n* `:adapter` - The adapter to use, should be `ConduitSQS`.\n* `:access_key_id` - The AWS access key ID. [See what ExAWS supports](https://hexdocs.pm/ex_aws/ExAws.html#module-getting-started).\n* `:secret_access_key` - The AWS secret access key. [See what ExAWS supports](https://hexdocs.pm/ex_aws/ExAws.html#module-getting-started).\n* `:worker_pool_size` - The default number of worker for each subscription. Defaults to `5` when not specified. Can also be overridden on each `subscribe`.\n* `:max_attempts` - How many times to retry publishing a message. Defaults to `10` when not specified. Can also be overriden on each `publish`.\n* `:base_backoff_in_ms` - The base factor used to exponentially backoff when a request fails. Defaults to `10` when not specified. Can also be overriden on `queue`, `publish`, and `subscribe`.\n* `:max_backoff_in_ms` - The maximum backoff when a request fails. Defaults to `10_000` when not specified. Can also be overriden on `queue`, `publish`, and `subscribe`.\n* `:region` - The AWS region to use by default. Defaults to `\"us-east-1\"` when not specified. Can also be overriden on `queue`, `publish`, and `subscribe`.\n\n## Configuring Queues\n\nInside the `configure` block of a broker, you can define queues that will be created at application\nstartup with the options you specify.\n\n``` elixir\ndefmodule MyApp.Broker do\n  configure do\n    queue \"my-queue\"\n    queue \"my-other-queue.fifo\", fifo_queue: true\n  end\nend\n```\n\n### Options\n\nFor creating a queue, ConduitSQS supports the same options that are specified in the SQS docs. See the\n[SQS docs for creating a queue](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_CreateQueue.html)\nto understand what each of the options mean. These options include:\n\n* `:policy`\n* `:visibility_timeout`\n* `:maximum_message_size`\n* `:message_retention_period`\n* `:approximate_number_of_messages`\n* `:approximate_number_of_messages_not_visible`\n* `:created_timestamp`\n* `:last_modified_timestamp`\n* `:queue_arn`\n* `:approximate_number_of_messages_delayed`\n* `:delay_seconds`\n* `:receive_message_wait_time_seconds`\n* `:redrive_policy`\n* `:fifo_queue`\n* `:content_based_deduplication`\n\nIn addition to the SQS options, you can also pass the following:\n\n* `:base_backoff_in_ms` - The base factor used to exponentially backoff when a request fails. Defaults to `10` when not specified. Can also be set globally.\n* `:max_backoff_in_ms` - The maximum backoff when a request fails. Defaults to `10_000` when not specified. Can also be set globally.\n* `:region` - The AWS region to use. Defaults to `\"us-east-1\"` when not specified. Can also be set globally.\n\n## Configuring a Subscriber\n\nInside an `incoming` block for a broker, you can define subscriptions to queues. Conduit will route messages on those\nqueues to your subscribers.\n\n``` elixir\ndefmodule MyApp.Broker do\n  incoming MyApp do\n    subscribe :my_subscriber, MySubscriber, from: \"my-queue\"\n    subscribe :my_other_subscriber, MyOtherSubscriber,\n      from: \"my-other-queue\",\n      max_number_of_messages: 10,\n      worker_pool_size: 1\n  end\nend\n```\n\n### Options\n\nConduitSQS requires that you specify a `from` option, which states which queue to consume from.\n\nFor subscribing to a queue, ConduitSQS supports the same options that are specified in the SQS docs. See the\n[SQS docs for receiving a message](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_ReceiveMessage.html)\nto understand what each of the options mean. These options include:\n\n* `:attribute_names`\n* `:message_attribute_names`\n* `:max_number_of_messages`\n* `:visibility_timeout`\n* `:wait_time_seconds`\n\nFor `attribute_names` and `message_attributes`, ConduitSQS will attempt to pull all attributes unless overriden. ConduitSQS defaults `max_number_of_messages` to `10`, which is the largest that SQS supports.\n\nIn addition to the SQS options, you can also pass the following:\n\n* `:worker_pool_size` - The number of workers that simultaneously consume from that queue. Defaults to 5.\n* `:max_demand` - Max number of messages in flow. Defaults to 1000.\n* `:min_demand` - Minumum threshold of messages to cause pulling of more. Defaults to 500.\n* `:base_backoff_in_ms` - The base factor used to exponentially backoff when a request fails. Defaults to `10` when not specified. Can also be set globally.\n* `:max_backoff_in_ms` - The maximum backoff when a request fails. Defaults to `10_000` when not specified. Can also be set globally.\n* `:region` - The AWS region to use. Defaults to `\"us-east-1\"` when not specified. Can also be set globally.\n\n## Configuring a Publisher\n\nInside an `outgoing` block for a broker, you can define publications to queues. Conduit will deliver messages using the\noptions specified. You can override these options, by passing different options to your broker's `publish/3`.\n\n``` elixir\ndefmodule MyApp.Broker do\n  outgoing do\n    publish :something, to: \"my-queue\"\n    publish :something_else,\n      to: \"my-other-queue\",\n      delay_seconds: 11\n  end\nend\n```\n\n### Options\n\nFor publishing to a queue, ConduitSQS supports the same options that are specified in the SQS docs. See the\n[SQS docs for publishing a message](http://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessage.html)\nto understand what each of the options mean. These options include:\n\n* `:delay_seconds`\n* `:message_deduplication_id`\n* `:message_group_id`\n\nThese options may also be specified by setting headers on the message being published. Headers on the message will\ntrump any options specified in an `outgoing` block. For example:\n\n``` elixir\nmessage =\n  %Conduit.Message{}\n  |\u003e put_header(\"message_group_id\", \"22\")\n```\n\nConduitSQS does not allow you to set the `message_attributes` option. Those will be filled from properties\non the message and from the message headers.\n\nIn addition to the SQS options, you can also pass the following:\n\n* `:to` - The destination queue for the message. If the message already has it's destination set, this option will be ignored.\n* `:max_attempts` - How many times to retry publishing a message. Defaults to `10` when not specified. Can also be set globally.\n* `:base_backoff_in_ms` - The base factor used to exponentially backoff when a request fails. Defaults to `10` when not specified. Can also be set globally.\n* `:max_backoff_in_ms` - The maximum backoff when a request fails. Defaults to `10_000` when not specified. Can also be set globally.\n* `:region` - The AWS region to use. Defaults to `\"us-east-1\"` when not specified. Can also be set globally.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconduitframework%2Fconduit_sqs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconduitframework%2Fconduit_sqs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconduitframework%2Fconduit_sqs/lists"}