{"id":13508996,"url":"https://github.com/wistia/elixir_nsq","last_synced_at":"2026-02-22T11:06:02.651Z","repository":{"id":56841840,"uuid":"46298349","full_name":"wistia/elixir_nsq","owner":"wistia","description":"An NSQ client for Elixir and Erlang, written in Elixir.","archived":false,"fork":false,"pushed_at":"2024-01-01T04:52:54.000Z","size":372,"stargazers_count":90,"open_issues_count":5,"forks_count":26,"subscribers_count":48,"default_branch":"master","last_synced_at":"2025-03-07T17:48:00.140Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/wistia.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2015-11-16T19:51:25.000Z","updated_at":"2024-10-21T14:32:24.000Z","dependencies_parsed_at":"2024-01-08T19:22:18.160Z","dependency_job_id":null,"html_url":"https://github.com/wistia/elixir_nsq","commit_stats":{"total_commits":342,"total_committers":11,"mean_commits":31.09090909090909,"dds":0.08771929824561409,"last_synced_commit":"0d9f24c9444eeaac2fae04987ae6fbc68b035a42"},"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wistia%2Felixir_nsq","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wistia%2Felixir_nsq/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wistia%2Felixir_nsq/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/wistia%2Felixir_nsq/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/wistia","download_url":"https://codeload.github.com/wistia/elixir_nsq/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":246314045,"owners_count":20757455,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-08-01T02:01:01.536Z","updated_at":"2025-10-21T18:49:42.217Z","avatar_url":"https://github.com/wistia.png","language":"Elixir","funding_links":[],"categories":["Queue"],"sub_categories":[],"readme":"# elixir_nsq\n\n`elixir_nsq` is a client library for NSQ. Use it in your Elixir or Erlang\napplications to handle messages asynchronously. This library seeks to be\ncomplete, well-tested, and easy to use.\n\nThis library used [go-nsq](https://github.com/nsqio/go-nsq) and\n[pynsq](https://github.com/nsqio/pynsq) for reference, but is structured to\nbetter fit common Elixir workflows.\n\nTo use this, you will need to have [NSQ](http://nsq.io/).\n\n## Publish Messages\n\n```elixir\n{:ok, producer} = NSQ.Producer.Supervisor.start_link(\"my-topic\", %NSQ.Config{\n  nsqds: [\"127.0.0.1:4150\"]\n})\n\n# publish to the default topic \"my-topic\"\nNSQ.Producer.pub(producer, \"a message\")\nNSQ.Producer.mpub(producer, [\"one\", \"two\", \"three\"])\n\n# specify a topic\nNSQ.Producer.pub(producer, \"different-topic\", \"another message\")\nNSQ.Producer.mpub(producer, \"different-topic\", [\"four\", \"five\", \"six\"])\n```\n\n## Quick Start\n\n### Add to mix.exs\n\n```elixir\ndefp deps do\n  [{:elixir_nsq, \"~\u003e 1.2.0\"}]\nend\n\ndefp applications do\n  [:logger, :elixir_nsq]\nend\n```\n\n### Consume Messages\n\nThe handler should return `:ok` to finish normally, `:req` to requeue the\nmessage, or `{:req, delay}` to specify your own requeue delay. If your message\nhandler throws an exception, it will automatically be requeued and delayed with\na timeout based on attempts.\n\n```elixir\n{:ok, consumer} = NSQ.Consumer.Supervisor.start_link(\"my-topic\", \"my-channel\", %NSQ.Config{\n  nsqlookupds: [\"127.0.0.1:4160\", \"127.0.0.1:4161\"],\n  message_handler: fn(body, msg) -\u003e\n    IO.puts \"id: #{msg.id}\"\n    IO.puts \"attempts: #{msg.attempts}\"\n    IO.puts \"timestamp: #{msg.timestamp}\"\n    :ok\n  end\n})\n```\n\nThe message handler can also be a module that implements `handle_message/2`:\n\n```elixir\ndefmodule MyMsgHandler do\n  def handle_message(body, msg) do\n    IO.puts \"Handled in a module! #{msg.id}\"\n    :ok\n  end\nend\n\n{:ok, consumer} = NSQ.Consumer.Supervisor.start_link(\"my-topic\", \"my-channel\", %NSQ.Config{\n  nsqlookupds: [\"127.0.0.1:4160\", \"127.0.0.1:4161\"],\n  message_handler: MyMsgHandler\n})\n```\n\nIf your message is especially long-running and you know it's not dead, you can\ntouch it so that NSQ doesn't automatically fail and requeue it.\n\n```elixir\ndef MyMsgHandler do\n  def handle_message(body, msg) do\n    Task.start_link fn -\u003e\n      :timer.sleep(30_000)\n      NSQ.Message.touch(msg)\n    end\n    long_running_operation()\n    :ok\n  end\nend\n```\n\nIf you're not using nsqlookupd, you can specify nsqds directly:\n\n```elixir\n{:ok, consumer} = NSQ.Consumer.Supervisor.start_link(\"my-topic\", \"my-channel\", %NSQ.Config{\n  nsqds: [\"127.0.0.1:4150\"],\n  message_handler: fn(body, msg) -\u003e\n    :ok\n  end\n})\n```\n\n## Configuration\n\nCheck https://github.com/wistia/elixir_nsq/blob/master/lib/nsq/config.ex for\nsupported config values.\n\n## Get notified\n\nNSQ.Consumer and NSQ.Producer provide the function `event_manager/1` so that\nyou can receive events from the NSQ client. You can keep your own stats/logs\nand perform actions based on that info.\n\n```elixir\ndefmodule EventForwarder do\n    @behaviour :gen_event\n\n    def init(args) do\n      {:ok, args}\n    end\n\n    def handle_event(event, parent) do\n      send parent, event\n      {:ok, parent}\n    end\n\n    def handle_call(_event, _state) do\n      raise \"not implemented\"\n    end\nend\n\ndef setup_consumer do\n  {:ok, consumer} = NSQ.Consumer.Supervisor.start_link(\"my-topic\", \"my-channel\", %NSQ.Config{\n    nsqds: [\"127.0.0.1:4150\"],\n    message_handler: fn(body, msg) -\u003e\n      :ok\n    end\n  })\n\n  # subscribe to events from the event manager\n  NSQ.Consumer.event_manager(consumer)\n  |\u003e GenEvent.add_handler(EventForwarder, self)\nend\n```\n\nPotential event formats are:\n\n- `{:message, NSQ.Message.t}`\n- `{:message_finished, NSQ.Message.t}`\n- `{:message_requeued, NSQ.Message.t}`\n- `:resume`\n- `:continue`\n- `:backoff`\n- `:heartbeat`\n- `{:response, binary}`\n- `{:error, String.t, binary}`\n\n### Supervision Tree\n\nFor your convenience, this is the overall process structure of `elixir_nsq`.\nIn practice, the Connection.Supervisors and Task.Supervisors don't do much\nautomatic restarting because NSQ itself is built to handle that. But they are\nuseful for propagating exit commands and keeping track of all running\nprocesses.\n\n    Consumer Supervisor\n      Consumer\n        ConnInfo Agent\n        Connection.Supervisor\n          Connection\n            Message.Supervisor\n              Message\n              Message\n          Connection\n            Message.Supervisor\n              Message\n              Message\n      Connection discovery loop\n      RDY redistribution loop\n\n    Producer Supervisor\n      Producer\n        ConnInfo Agent\n        Connection.Supervisor\n          Connection\n          Connection\n\n## Running the Tests\n\nThe included tests require two nsqds and two nsqlookupds. A Procfile for use\nwith foreman is included to start these up. If you don't have\n[foreman](https://github.com/ddollar/foreman), you'll need to find a way to run\nthose commands if you want to run the tests.\n\nIf you are using nsq \u003c 1.0.0\n\n```bash\nWORKER_ID=worker-id foreman start\nmix test\n```\n\nIf you are using nsq \u003e= 1.0.0\n\n```bash\nWORKER_ID=node-id foreman start\nmix test\n```\n\nNote that some tests intentionally cause processes to exit, so you might see\nsome error logging as part of the tests. As long as they're still passing, that\nis considered normal behavior.\n\n\n## Known Issues\n\n- Snappy cannot be supported because existing NIFs cannot correctly decompress\n  the nsqd stream. I believe they need support for skipping the checksum.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwistia%2Felixir_nsq","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fwistia%2Felixir_nsq","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fwistia%2Felixir_nsq/lists"}