{"id":24569751,"url":"https://github.com/mpol1t/off_broadway_websocket","last_synced_at":"2025-10-27T11:15:31.645Z","repository":{"id":261140456,"uuid":"879965838","full_name":"mpol1t/off_broadway_websocket","owner":"mpol1t","description":"An Off-Broadway producer enabling real-time ingestion of WebSocket data.","archived":false,"fork":false,"pushed_at":"2025-04-14T07:57:14.000Z","size":157,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-04-14T08:44:38.183Z","etag":null,"topics":["broadway","elixir","websocket"],"latest_commit_sha":null,"homepage":"https://hex.pm/packages/off_broadway_websocket","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/mpol1t.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2024-10-28T21:44:14.000Z","updated_at":"2025-04-14T07:57:14.000Z","dependencies_parsed_at":"2024-12-31T11:30:34.251Z","dependency_job_id":"81574db0-ac62-4452-9bc3-041af6ef6795","html_url":"https://github.com/mpol1t/off_broadway_websocket","commit_stats":null,"previous_names":["mpol1t/off_broadway_websocket"],"tags_count":10,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpol1t%2Foff_broadway_websocket","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpol1t%2Foff_broadway_websocket/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpol1t%2Foff_broadway_websocket/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/mpol1t%2Foff_broadway_websocket/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/mpol1t","download_url":"https://codeload.github.com/mpol1t/off_broadway_websocket/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250314694,"owners_count":21410467,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["broadway","elixir","websocket"],"created_at":"2025-01-23T15:56:02.274Z","updated_at":"2025-10-27T11:15:31.639Z","avatar_url":"https://github.com/mpol1t.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![codecov](https://codecov.io/gh/mpol1t/off_broadway_websocket/graph/badge.svg?token=ZDf9PrffNJ)](https://codecov.io/gh/mpol1t/off_broadway_websocket)\n[![Hex.pm Version](https://img.shields.io/hexpm/v/off_broadway_websocket)](https://hex.pm/packages/off_broadway_websocket)\n[![License](https://img.shields.io/github/license/mpol1t/off_broadway_websocket.svg)](https://github.com/mpol1t/off_broadway_websocket/blob/main/LICENSE)\n[![Documentation](https://img.shields.io/badge/docs-hexdocs-blue.svg)](https://hexdocs.pm/off_broadway_websocket)\n[![Build Status](https://github.com/mpol1t/off_broadway_websocket/actions/workflows/elixir.yml/badge.svg)](https://github.com/mpol1t/off_broadway_websocket/actions)\n[![Elixir Version](https://img.shields.io/badge/elixir-~%3E%201.16-purple.svg)](https://elixir-lang.org/)\n\n# OffBroadwayWebSocket\n\nAn Elixir library providing a **Broadway** producer for resilient WebSocket connections using **gun**.\n\n### Features\n\n- Unified `gun_opts` for TLS, HTTP and WebSocket configuration\n- Idle-timeout detection through ping/pong and data frames\n- Demand-based dispatch compatible with Broadway back-pressure\n- Customizable retry strategies with user-defined functions\n\n---\n\n## Installation\n\n\nAdd to your `mix.exs`:\n```elixir\ndef deps do\n  [\n    {:off_broadway_websocket, \"~\u003e 1.0.2\"}\n  ]\nend\n```\nFetch \u0026 compile:\n\n\n```bash\nmix deps.get\nmix deps.compile\n```\n---\n\n## Quickstart\n\n```elixir\ndefmodule MyApp.Broadway do\n  use Broadway\n  \n  require Logger\n  \n  alias Broadway.Message\n  alias Broadway.NoopAcknowledger\n\n  def start_link(_args) do\n    Broadway.start_link(__MODULE__,\n      name: __MODULE__,\n      producer: [\n        module: {\n          OffBroadwayWebSocket.Producer,\n          # Your WebSocket endpoint:\n          url: \"wss://example.com:443\",\n          path: \"/stream/updates\",\n\n          # Idle timeout (ms) for no ping/data before reconnect:\n          ws_timeout: 15_000,\n          # How long to wait (ms) for Gun to come up:\n          await_timeout: 8_000,\n\n          # Retry configuration – must include at least :retries_left and :delay:\n          ws_retry_opts: %{\n            max_retries:     5,\n            retries_left:    5,\n            delay:           1_000,   # initial backoff (ms)\n            max_delay:       30_000,  # cap for backoff (ms)\n            backoff_factor:  2,       # exponential factor\n            jitter_fraction: 0.1      # ±10% random jitter\n          },\n          ws_retry_fun: \u0026MyApp.Backoff.exponential_backoff_with_jitter/1,\n\n          # Gun options (TCP/TLS, HTTP, WS):\n          gun_opts: %{\n            connect_timeout: 5_000,      # TCP/TLS handshake timeout\n            protocols:       [:http],     # application protocols\n            transport:       :tls,        # :tcp or :tls\n\n            tls_opts: [\n              verify:         :verify_peer,\n              cacertfile:     CAStore.file_path(),\n              depth:          10,\n              reuse_sessions: false,\n              verify_fun:     {\n                \u0026:ssl_verify_hostname.verify_fun/3,\n                [check_hostname: String.to_charlist(\"example.com\")]\n              }\n            ],\n\n            ws_opts: %{\n              keepalive:     10_000,  # send ping if silent\n              silence_pings: false\n            },\n\n            http_opts: %{\n              version:       :\"HTTP/1.1\"\n            }\n          },\n\n          # Prefix for telemetry events:\n          telemetry_id: :custom_telemetry,\n          # Optional headers\n          headers: [  \n            {\"X-ABC-APIKEY\", \"api-key\"},\n            {\"X-ABC-PAYLOAD\", %{}},\n            {\"X-ABC-SIGNATURE\", \"signature\"}\n          ],\n        },\n        transformer: {__MODULE__, :transform, []},\n        concurrency: 1\n      ],\n      processors: [\n        default: [min_demand: 0, max_demand: 100, concurrency: 8]\n      ],\n      context: []\n    )\n  end\n\n  @impl true\n  def handle_message(_stage, %Message{data: raw} = msg, _ctx) do\n    case Jason.decode(raw) do\n      {:ok, data} -\u003e\n        Logger.debug(fn -\u003e \"Data: #{inspect(data)}\" end)\n        msg\n\n      {:error, err} -\u003e\n        Logger.error(\"Decode error: #{inspect(err)}\")\n        Message.failed(msg, err)\n    end\n  end\n\n  def transform(event, _opts) do\n    %Broadway.Message{\n      data:        event,\n      acknowledger: NoopAcknowledger.init()\n    }\n  end\nend\n```\n\n---\n\n## Configuration Options\n\nWhen calling `OffBroadwayWebSocket.Producer`, you may pass:\n\n- **`:url`** (_string_, required) — WebSocket base URL.\n- **`:path`** (_string_, required) — Upgrade path and querystring.\n- **`:ws_timeout`** (_ms_, optional) — Idle timeout for no ping/data.\n- **`:await_timeout`** (_ms_, optional) — Timeout for `:gun.await_up/2`.\n- **`:headers`** (_list_, optional) — HTTP headers for WS upgrade.\n- **`:min_demand`** / **`:max_demand`** (_integer_) — Broadway backpressure.\n- **`:telemetry_id`** (_atom_) — Prefix for telemetry events.\n- **`:gun_opts`** (_map_) — All options forwarded to `:gun.open/3` and friends.\n- **`:ws_retry_opts`** (_map_) — Your initial retry state; must include:\n    - `:retries_left`, `:delay` (ms).\n    - Extra keys (e.g. `:backoff_factor`, `:jitter_fraction`) are carried through.\n- **`:ws_retry_fun`** (_function_) — A `(retry_opts() -\u003e retry_opts())` function.\n  After each failed connect, the returned map’s `:delay` is used and stored as the next call’s input. After successful\n  reconnection, `:ws_retry_opts` are reset to initial value.\n\n---\n\n## Default Configuration\n\nOut of the box, `OffBroadwayWebSocket.Producer` uses these defaults:\n\n| Option             | Default                            | Description                                 |\n|--------------------|------------------------------------|---------------------------------------------|\n| `:url`             | **—**                              | WebSocket URL (required)                    |\n| `:path`            | **—**                              | WebSocket path (required)                   |\n| `:ws_timeout`      | `nil`                              | Idle timeout (ms) for ping/data             |\n| `:await_timeout`   | `10_000`                           | `gun.await_up/2` timeout (ms)               |\n| `:headers`         | `[]`                               | Upgrade HTTP headers                        |\n| `:min_demand`      | `10`                               | Broadway `min_demand`                       |\n| `:max_demand`      | `100`                              | Broadway `max_demand`                       |\n| `:telemetry_id`    | `:websocket_producer`              | Prefix for telemetry events                 |\n| `:gun_opts`        | `%{}`                              | Direct options to `:gun.open/3`, etc.       |\n| `:ws_retry_opts`   | see _Default `ws_retry_opts`_      | Initial retry state                         |\n| `:ws_retry_fun`    | `\u0026OffBroadwayWebSocket.State.default_ws_retry_fun/1`    | Backoff function contract                   |\n\n### Default Backoff Function\n\nBy default, a constant backoff function is used with the config shown below:\n\n```elixir\n%{\n  max_retries:  5,     # total retry attempts\n  retries_left: 5,     # decremented on each failure\n  delay:        10_000 # constant delay in ms between retries\n}\n```\n---\n\n## Telemetry Events\n\nFired under `[:\u003ctelemetry_id\u003e, :connection, \u003cevent\u003e]`:\n\n| Event           | Measurements    | Metadata          | Description                       |\n|-----------------|-----------------|-------------------|-----------------------------------|\n| `:success`      | `%{count: 1}`   | `%{url: String}`  | Handshake completed               |\n| `:failure`      | `%{count: 1}`   | `%{reason: term}` | Connect or upgrade failed         |\n| `:disconnected` | `%{count: 1}`   | `%{reason: term}` | Underlying TCP connection dropped |\n| `:timeout`      | `%{count: 1}`   | `%{}`             | Idle ping/data timeout            |\n| `:status`       | `%{value: 0|1}` | `%{}`             | `0`=down, `1`=up                  |\n\nAttach as usual:\n\n```elixir\n:telemetry.attach(\n  \"log-connection-success\",\n  [:websocket_producer, :connection, :success],\n  fn event_name, measurements, metadata, _config -\u003e\n    IO.inspect({event_name, measurements, metadata}, label: \"Telemetry Event\")\n  end,\n  nil\n)\n```\n\n---\n\n## Running Tests\n\n```bash\nmix test\n```\n\n---\n\n## Dialyzer\n\n```bash\nmix dialyzer --plt\nmix dialyzer\n```\n\n---\n\n## Contributing\n\nPRs and issues welcome! Please follow Elixir conventions and include tests.\n\n---\n\n## License\n\nApache License 2.0 © 2025  \n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpol1t%2Foff_broadway_websocket","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmpol1t%2Foff_broadway_websocket","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmpol1t%2Foff_broadway_websocket/lists"}