{"id":26197176,"url":"https://github.com/digixglobal/conduit_nsq","last_synced_at":"2025-03-12T02:24:35.041Z","repository":{"id":57485270,"uuid":"203542295","full_name":"DigixGlobal/conduit_nsq","owner":"DigixGlobal","description":"A NSQ adapter for conduit","archived":false,"fork":false,"pushed_at":"2019-11-19T08:39:14.000Z","size":67,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-03-11T20:46:37.296Z","etag":null,"topics":["conduit","elixir","message-queue"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/DigixGlobal.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-08-21T08:37:58.000Z","updated_at":"2024-04-27T09:13:42.000Z","dependencies_parsed_at":"2022-08-26T11:11:26.537Z","dependency_job_id":null,"html_url":"https://github.com/DigixGlobal/conduit_nsq","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigixGlobal%2Fconduit_nsq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigixGlobal%2Fconduit_nsq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigixGlobal%2Fconduit_nsq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/DigixGlobal%2Fconduit_nsq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/DigixGlobal","download_url":"https://codeload.github.com/DigixGlobal/conduit_nsq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243142010,"owners_count":20242977,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conduit","elixir","message-queue"],"created_at":"2025-03-12T02:24:34.357Z","updated_at":"2025-03-12T02:24:35.027Z","avatar_url":"https://github.com/DigixGlobal.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# ConduitNSQ\n\n[![Hex version badge](https://img.shields.io/hexpm/v/conduit_nsq.svg)](https://hex.pm/packages/conduit_nsq)\n[![License badge](https://img.shields.io/hexpm/l/conduit_nsq.svg)](https://github.com/DigixGlobal/conduit_nsq/blob/master/LICENSE.md)\n[![Build status badge](https://img.shields.io/circleci/project/github/DigixGlobal/conduit_nsq/master.svg)](https://circleci.com/gh/DigixGlobal/conduit_nsq/tree/master)\n\nA [NSQ](https://nsq.io/) adapter for [conduit](https://github.com/conduitframework/conduit).\n\n**CAVEAT: This adapter library is not mature so take precaution when\nusing it for production.** This library also may not be as lightweight,\neficient, thoughtout. As an example, this library uses\n[honeydew](https://github.com/koudelka/honeydew) as a worker pool to\nthrottle sending and receiving messages instead of the builtin\n`GenStage`.\n\n## Installation\n\nThe package can be installed by adding `conduit_nsq` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [{:conduit_nsq, \"~\u003e 0.1.4\"}]\nend\n```\n\nOnce you created your own `MyApp.Broker`, remember to add it in your\napplication:\n\n```elixir\n  def start(_type, _args) do\n    children = [\n      MyApp.Broker\n    ]\n\n    opts = [strategy: :one_for_one, name: MyApp.Supervisor]\n    Supervisor.start_link(children, opts)\n  end\n```\n\n## Configuring the Adapter\n\nThis library uses [elixir_nsq](https://github.com/wistia/elixir_nsq) to\nconnect to the message queue. Aside from specifiying the `:adapter`,\nevery option in the following snippet below is passed down to `NSQ.Config`:\n\n``` elixir\nconfig :my_app, MyApp.Broker,\n  adapter: ConduitNSQ,\n  producer_nsqds: [\n    \"127.0.0.1:12150\",\n  ],\n  nsqds: [\n    \"127.0.0.1:12150\",\n    \"127.0.0.1:13150\",\n    \"127.0.0.1:14150\"\n  ],\n  nsqlookupds: [\"127.0.0.1:12161\"],\n\n  backoff_multiplier: 2_000\n```\n\nCheckout the [list of supported\noptions](https://github.com/wistia/elixir_nsq/blob/master/lib/nsq/config.ex).\n\n\nThe only different option is `:producer_nsqds` which is publisher\nspecific endpoints than consumer specific endpoints. This option is to\nsupport the idiomatic strategy of publishing to colocated `nsqds` while\nlistening in to external producers. (See [Eliminating\nSPOFs](https://nsq.io/overview/design.html#eliminating-spofs)).\n\nFor more adapter specific options:\n\n```elixir\n# Default config\nconfig :conduit_nsq,\n  publisher_workers: 3,\n  processor_workers: 10,\n  publish_timeout: 60_000,\n  process_timeout: 60_000\n```\n\n### Options\n\n    Aside from `:adapter`, the current options should be good defaults.\n\n* `:adapter` - The message queue adapter to use, should be `ConduitNSQ`.\n* `:publisher_workers` - The number of workers for publishing messages.\n  See [Honeydew.start_workers/3](https://hexdocs.pm/honeydew/Honeydew.html#start_workers/3)\n* `:processor_workers` - The number of workers for receiving messages.\n  See\n  [Honeydew.start_workers/3](https://hexdocs.pm/honeydew/Honeydew.html#start_workers/3)\n* `:publish_timeout` - Timeout in publishing messages. See [Honeydew.yield/2](https://hexdocs.pm/honeydew/Honeydew.html#yield/2)\n* `:process_timeout` - Timeout in receiving messages. See [Honeydew.yield/2](https://hexdocs.pm/honeydew/Honeydew.html#yield/2)\n\n## Configuring Topic\n\nInside the `configure` block of a broker, you can define topics via\n`queue` that will be created at application startup with the options you specify.\n\n``` elixir\ndefmodule MyApp.Broker do\n  configure do\n    queue \"my-topic\"\n    queue \"other-topic\"\n  end\nend\n```\n\nAll topics that the application will publish to must be defined here\nsince each message is routed to the corresponding\n`NSQ.Producer.Supervisor` otherwise the message might be unsent.\n\n### Ephemeral Topics or Channels\n\nYou can suffix topics and channels with `#ephemeral` based on [bounded\nmemory\nfootprint](https://nsq.io/overview/design.html#bounded-memory-footprint)).\nHowever, when publishing or subscribing to the topic, also add the\nsame suffix to the topic or channel to match\n\n## Configuring a Subscriber\n\nInside an `incoming` block for a broker, you can define subscriptions to\ntopics. Conduit will route messages on those topics to your subscribers.\n\n``` elixir\ndefmodule MyApp.Broker do\n  incoming MyApp do\n    subscribe :my_subscriber, MySubscriber,\n      topic: \"my-queue\",\n      channel: \"my-channel\"\n    subscribe :my_other_subscriber, MyOtherSubscriber,\n      topic: \"my-other-queue\",\n      from: \"my-other-channel\"\n  end\nend\n```\n\nMake sure the `:topic`s here are found in the `configure` block to\nreceive the messages.\n\n### Options\n\nThe only required options are `:topic` and `:channel` which follows\n`NSQ.Consumer.Supervisor`.\n\n- `:topic` - Topic to connect with\n- `:channel` - Topic channel to listen into\n\n## Configuring a Publisher\n\nInside an `outgoing` block for a broker, you can define publications to topics.\n\n``` elixir\ndefmodule MyApp.Broker do\n  outgoing do\n    publish :something, topic: \"my-topic\"\n    publish :something_else, topic: \"my-other-topic\",\n  end\nend\n```\n\n### Options\n\nThe only required option is `:topic` which follows\n`NSQ.Producer.Supervisor`.\n\n- `:topic` - Topic to publish to\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigixglobal%2Fconduit_nsq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fdigixglobal%2Fconduit_nsq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fdigixglobal%2Fconduit_nsq/lists"}