{"id":20511306,"url":"https://github.com/intility/off_broadway_splunk","last_synced_at":"2025-09-06T17:46:02.161Z","repository":{"id":62204611,"uuid":"558790649","full_name":"intility/off_broadway_splunk","owner":"intility","description":"Broadway producer for Splunk API","archived":false,"fork":false,"pushed_at":"2024-08-20T12:34:14.000Z","size":240,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-03-27T13:01:52.574Z","etag":null,"topics":["broadway","elixir","splunk"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/intility.png","metadata":{"files":{"readme":"README.md","changelog":"Changelog.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-10-28T09:55:02.000Z","updated_at":"2024-12-16T10:56:26.000Z","dependencies_parsed_at":"2024-08-21T03:50:52.807Z","dependency_job_id":null,"html_url":"https://github.com/intility/off_broadway_splunk","commit_stats":{"total_commits":55,"total_committers":1,"mean_commits":55.0,"dds":0.0,"last_synced_commit":"027d3bc1994d5b53e19b191dee76ecfb19a0f1d9"},"previous_names":[],"tags_count":14,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intility%2Foff_broadway_splunk","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intility%2Foff_broadway_splunk/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intility%2Foff_broadway_splunk/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/intility%2Foff_broadway_splunk/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/intility","download_url":"https://codeload.github.com/intility/off_broadway_splunk/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248794565,"owners_count":21162613,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["broadway","elixir","splunk"],"created_at":"2024-11-15T20:35:27.797Z","updated_at":"2025-04-13T22:42:06.861Z","avatar_url":"https://github.com/intility.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# OffBroadway.Splunk\n\n![pipeline status](https://github.com/Intility/off_broadway_splunk/actions/workflows/elixir.yaml/badge.svg?event=push)\n\nA Splunk consumer for [Broadway](https://github.com/dashbitco/broadway).\n\nBroadway producer acts as a consumer for a given Splunk report or (triggered) alert.\nThe `OffBroadway.Splunk.Producer` process will query Splunk for available jobs for the given\nreport and keep them in a queue. Jobs wil then be processed sequentially (from earliest to latest)\nand passed through the Broadway pipeline.\n\nRead the full documentation [here](https://hexdocs.pm/off_broadway_splunk/readme.html).\n\n## Installation\n\nThis package is [available in Hex](https://hex.pm/packages/off_broadway_splunk), and can be installed\nby adding `off_broadway_splunk` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [\n    {:off_broadway_splunk, \"~\u003e 2.0\"}\n  ]\nend\n```\n\n## Usage\n\nThe `OffBroadway.Splunk.SplunkClient` tries to read the following configuration from `config.exs`.\n\n```elixir\n# config.exs\n\nconfig :off_broadway_splunk, :splunk_client,\n  base_url: System.get_env(\"SPLUNK_BASE_URL\", \"https://splunk.example.com\"),\n  api_token: System.get_env(\"SPLUNK_API_TOKEN\", \"your-api-token-here\")\n```\n\nOptions for the `OffBroadway.Splunk.SplunkClient` can be configured either in `config.exs` or passed as\noptions directly to the `OffBroadway.Splunk.Producer` module. Options are merged, with the passed options\ntaking precedence over those configured in `config.exs`.\n\n```elixir\n# my_broadway.ex\n\ndefmodule MyBroadway do\n  use Broadway\n\n  alias Broadway.Message\n\n  def start_link(_opts) do\n    Broadway.start_link(__MODULE__,\n      name: __MODULE__,\n      producer: [\n        module:\n          {OffBroadway.Splunk.Producer,\n           name: \"My fine report\",\n           config: [api_token: \"override-api-token\"]}\n      ],\n      processors: [\n        default: []\n      ],\n      batchers: [\n        default: [\n          batch_size: 500,\n          batch_timeout: 5000\n        ]\n      ]\n    )\n  end\n\n  ...callbacks...\nend\n```\n\n### Processing messages\n\nIn order to process incoming messages, we need to implement some callback functions.\n\n```elixir\ndefmodule MyBroadway do\n  use Broadway\n\n  alias Broadway.Message\n\n  ...start_link...\n\n  @impl true\n  def handle_message(_, %Message{data: data} ,_) do\n    message\n    |\u003e Message.update_data(fn -\u003e ...whatever... end)\n  end\n\n  @impl true\n  def handle_batch(_batcher, messages, _batch_info, _context) do\n    IO.puts(\"Received a batch of #{length(messages)} messages!\")\n    messages\n  end\nend\n```\n\nFor the sake of the example, we're not really doing anything here. Whenever we're receiving a batch of messages, we just prints out a\nmessage saying \"Received a batch of messages!\", and for each message we run `Message.update_data/2` passing a function that can process\nthat message ie. by doing some calculations on the data or something else.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintility%2Foff_broadway_splunk","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fintility%2Foff_broadway_splunk","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fintility%2Foff_broadway_splunk/lists"}