{"id":13491583,"url":"https://github.com/zorbash/opus","last_synced_at":"2025-04-08T12:07:46.828Z","repository":{"id":46705702,"uuid":"123845489","full_name":"zorbash/opus","owner":"zorbash","description":"A framework for pluggable business logic components","archived":false,"fork":false,"pushed_at":"2024-08-03T11:50:07.000Z","size":228,"stargazers_count":360,"open_issues_count":4,"forks_count":21,"subscribers_count":17,"default_branch":"master","last_synced_at":"2024-10-11T20:14:21.852Z","etag":null,"topics":["dsl","elixir","pipelines","railway-oriented-programming"],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/zorbash.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE.txt","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2018-03-05T01:05:52.000Z","updated_at":"2024-09-30T11:28:29.000Z","dependencies_parsed_at":"2024-08-03T12:54:53.100Z","dependency_job_id":null,"html_url":"https://github.com/zorbash/opus","commit_stats":{"total_commits":101,"total_committers":12,"mean_commits":8.416666666666666,"dds":0.2178217821782178,"last_synced_commit":"520e7362fdbb6d18aeb81bd40279440ffe3a47ef"},"previous_names":[],"tags_count":18,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zorbash%2Fopus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zorbash%2Fopus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zorbash%2Fopus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/zorbash%2Fopus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/zorbash","download_url":"https://codeload.github.com/zorbash/opus/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247838444,"owners_count":21004580,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["dsl","elixir","pipelines","railway-oriented-programming"],"created_at":"2024-07-31T19:00:58.354Z","updated_at":"2025-04-08T12:07:46.809Z","avatar_url":"https://github.com/zorbash.png","language":"Elixir","funding_links":[],"categories":["Elixir","Macros"],"sub_categories":[],"readme":"# Opus\n\n[![Build Status](https://github.com/zorbash/opus/actions/workflows/ci.yml/badge.svg)](https://github.com/zorbash/opus/actions)\n[![Package Version](https://img.shields.io/hexpm/v/opus.svg)](https://hex.pm/packages/opus)\n[![Coverage Status](https://coveralls.io/repos/github/zorbash/opus/badge.svg?branch=master)](https://coveralls.io/github/zorbash/opus?branch=master)\n\n[![Livebook badge](https://livebook.dev/badge/v1/blue.svg)](https://livebook.dev/run?url=https%3A%2F%2Fhexdocs.pm%2Fopus%2Ftutorial.livemd)\n\nA framework for pluggable business logic components.\n\n![example-image](https://i.imgur.com/WwuyojJ.png)\n\n## Installation\n\nThe package can be installed by adding `opus` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [{:opus, \"~\u003e 0.8\"}]\nend\n```\n\n## Documentation\n\n* [hexdocs](https://hexdocs.pm/opus)\n* [wiki](https://github.com/zorbash/opus/wiki)\n* [tutorial](https://hexdocs.pm/opus/tutorial.html)\n\n## Conventions\n\n* Each Opus pipeline module has a single entry point and returns tagged tuples\n    `{:ok, value} | {:error, error}`\n* A pipeline is a composition of stateless stages\n* A stage returning `{:error, _}` halts the pipeline\n* A stage may be skipped based on a condition function (`:if` and `:unless` options)\n* Exceptions are converted to `{:error, error}` tuples by default\n* An exception may be left to raise using the `:raise` option\n* Each stage of the pipeline is instrumented. Metrics are captured\n  automatically (but can be disabled).\n* Errors are meaningful and predictable\n\n## Usage\n\n```elixir\ndefmodule ArithmeticPipeline do\n  use Opus.Pipeline\n\n  step  :add_one,         with: \u0026(\u00261 + 1)\n  check :even?,           with: \u0026(rem(\u00261, 2) == 0), error_message: :expected_an_even\n  tee   :publish_number,  if: \u0026Publisher.publishable?/1, raise: [ExternalError]\n  step  :double,          if: :lucky_number?\n  step  :divide,          unless: :lucky_number?\n  step  :randomize,       with: \u0026(\u00261 * :rand.uniform)\n  link  JSONPipeline\n\n  def double(n), do: n * 2\n  def divide(n), do: n / 2\n  def lucky_number?(n) when n in 42..1337, do: true\n  def lucky_number?(_), do: false\nend\n\nArithmeticPipeline.call(41)\n# {:ok, 84.13436750126804}\n```\n\nRead this [blogpost][medium-blogpost] to get started.\n\n## Pipeline\n\nThe core aspect of this library is defining pipeline modules. As in the\nexample above you need to add `use Opus.Pipeline` to turn a module into\na pipeline. A pipeline module is a composition of stages executed in\nsequence.\n\n## Stages\n\nThere are a few different types of stages for different use-cases.\nAll stage functions, expect a single argument which is provided either\nfrom initial `call/1` of the pipeline module or the return value of the\nprevious stage.\n\nAn error value is either `:error` or `{:error, any}` and anything else\nis considered a success value.\n\n### Step\n\nThis stage processes the input value and with a success value the next\nstage is called with that value. With an error value the pipeline is\nhalted and an `{:error, any}` is returned.\n\n### Check\n\nThis stage is intended for validations.\n\nThis stage calls the stage function and unless it returns `true` it\nhalts the pipeline.\n\nExample:\n\n```elixir\ndefmodule CreateUserPipeline do\n  use Opus.Pipeline\n\n  check :valid_params?, with: \u0026match?(%{email: email} when is_bitstring(email), \u00261)\n  # other stages to actually create the user\nend\n```\n\n### Tee\n\nThis stage is intended for side effects, such as a notification or a\ncall to an external system where the return value is not meaningful.\nIt never halts the pipeline.\n\n### Link\n\nThis stage is to link with another Opus.Pipeline module. It calls\n`call/1` for the provided module. If the module is not an\n`Opus.Pipeline` it is ignored.\n\n#### Skip\n\nThe `skip` macro can be used for linked pipelines.\nA linked pipeline may act as a true bypass, based on a condition,\nexpressed as either `:if` or `:unless`. When skipped, none of the stages\nare executed and it returns the input, to be used by any next stages of\nthe caller pipeline. A very common use-case is illustrated in the following example:\n\n\n```elixir\ndefmodule RetrieveCustomerInformation do\n  use Opus.Pipeline\n\n  check :valid_query?\n  link FetchFromCache,    if: :cacheable?\n  link FetchFromDatabase, if: :db_backed?\n  step :serialize\nend\n```\n\nWith `skip` it can be written as:\n\n```elixir\ndefmodule RetrieveCustomerInformation do\n  use Opus.Pipeline\n\n  check :valid_query?\n  link FetchFromCache\n  link FetchFromDatabase\n  step :serialize\nend\n```\n\nA linked pipeline becomes:\n\n```elixir\ndefmodule FetchFromCache do\n  use Opus.Pipeline\n\n  skip :assert_suitable, if: :cacheable?\n  step :retrieve_from_cache\nend\n```\n\n### Available options\n\nThe behaviour of each stage can be configured with any of the available\noptions:\n\n* `:with`: The function to call to fulfill this stage. It can be an Atom\n  referring to a public function of the module, an anonymous function or\n  a function reference.\n* `:if`: Makes a stage conditional, it can be either an Atom referring\n  to a public function of the module, an anonymous function or a\n  function reference. For the stage to be executed, the condition *must*\n  return `true`. When the stage is skipped, the input is forwarded to\n  the next step if there's one.\n* `:unless`: The opposite of the `:if` option, executes the step only\n    when the callback function returns `false`.\n* `:raise`: A list of exceptions to not rescue. Defaults to `false`\n  which converts all exceptions to `{:error, %Opus.PipelineError{}}`\n  values halting the pipeline.\n* `:error_message`: An error message to replace the original error when a\n  stage fails. It can be a String or Atom, which will be used directly in place\n  of the original message, or an anonymous function, which receives the input\n  of the failed stage and must return the error message to be used.\n* `:retry_times`: How many times to retry a failing stage, before\n  halting the pipeline.\n* `:retry_backoff`: A backoff function to provide delay values for\n  retries. It can be an Atom referring to a public function in the\n  module, an anonymous function or a function reference. It must return\n  an `Enumerable.t` yielding at least as many numbers as the\n  `retry_times`.\n* `:instrument?`: A boolean which defaults to `true`. Set to `false` to\n  skip instrumentation for a stage.\n\n### Retries\n\n```elixir\ndefmodule ExternalApiPipeline do\n  use Opus.Pipeline\n\n  step :http_request, retry_times: 8, retry_backoff: fn -\u003e linear_backoff(10, 30) |\u003e cap(100) end\n\n  def http_request(_input) do\n    # code for the actual request\n  end\nend\n```\n\nThe above module, will retry be retried up to 8 times, each time\napplying a delay from the next value of the retry_backoff function, which returns a\nStream.\n\nAll the functions from the [:retry][hex-retry] package will be available to be used in `retry_backoff`.\n\n## Stage Filtering\n\nYou can select the stages of a pipeline to run using `call/2` with the `:except` and `:only` options.\n\nExample:\n\n```elixir\n# Runs only the stage with the :validate_params name\nCreateUserPipeline.call(params, only: [:validate_params]\n\n# Runs all the stages except the selected ones\nCreateUserPipeline.call(params, except: :send_notification)\n```\n\n## Instrumentation\n\nInstrumentation hooks which can be defined:\n\n* `:pipeline_started`: Called before a pipeline module is called\n* `:before_stage`: Called before each stage\n* `:stage_skipped`: Called when a conditional stage was skipped\n* `:stage_completed`: Called after each stage\n* `:pipeline_completed`: Called after pipeline module has returned\n\nYou can disable all instrumentation callbacks for a stage using `instrument?: false`.\n\n```elixir\ndefmodule ArithmeticPipeline do\n  use Opus.Pipeline\n\n  step :double, instrument?: false\nend\n```\n\nYou can define module specific instrumentation callbacks using:\n\n```elixir\ndefmodule ArithmeticPipeline do\n  use Opus.Pipeline\n\n  step :double, with: \u0026(\u00261 * 2)\n  step :triple, with: \u0026(\u00261 * 3)\n\n  instrument :before_stage, fn %{input: input} -\u003e\n    IO.inspect input\n  end\n\n  # Will be called only for the matching stage\n  instrument :stage_completed, %{stage: %{name: :triple}}, fn %{time: time} -\u003e\n    # send to the monitoring tool of your choice\n  end\nend\n```\n\nYou can define a default instrumentation module for all your pipelines\nby adding in your `config/*.exs`:\n\n```elixir\nconfig :opus, :instrumentation, YourModule\n\n# but you may choose to provide a list of modules\nconfig :opus, :instrumentation, [YourModuleA, YourModuleB]\n```\n\nAn instrumentation module has to export `instrument/3` functions like:\n\n```elixir\ndefmodule CustomInstrumentation do\n  def instrument(:pipeline_started, %{pipeline: ArithmeticPipeline}, %{input: input}) do\n    # publish the metrics to specific backend\n  end\n\n  def instrument(:before_stage, %{stage: %{pipeline: pipeline}}, %{input: input}) do\n    # publish the metrics to specific backend\n  end\n\n  def instrument(:stage_completed, %{stage: %{pipeline: ArithmeticPipeline}}, %{time: time}) do\n    # publish the metrics to specific backend\n  end\n\n  def instrument(:pipeline_completed, %{pipeline: ArithmeticPipeline}, %{result: result, time: total_time}) do\n    # publish the metrics to specific backend\n  end\n\n  def instrument(_, _, _), do: nil\nend\n```\n\n### Telemetry\n\nOpus includes an instrumentation module which emits events using the `:telemetry` library.  \nTo enable it, change your `config/config.exs` with:\n\n```elixir\nconfig :opus, :instrumentation, [Opus.Telemetry]\n```\n\nBrowse the available events [here][opus-telemetry].\n\nFor instructions to integrate Opus Telemetry metrics in your Phoenix\napplication, read this [post][post-opus-telemetry].\n\n## Module-Global Options\n\nYou may choose to provide some common options to all the stages of a pipeline.\n\n* `:raise`: A list of exceptions to not rescue. When set to `true`, Opus\n    does not handle any exceptions. Defaults to `false` which converts all exceptions\n    to `{:error, %Opus.PipelineError{}}` values halting the pipeline.\n* `:instrument?`: A boolean which defaults to `true`. Set to `false` to\n  skip instrumentation for a module.\n\n```elixir\ndefmodule ArithmeticPipeline do\n  use Opus.Pipeline, instrument?: false, raise: true\n  # The pipeline opts will disable instrumentation for this module\n  # and will not rescue exceptions from any of the stages\n\n  step :double, with: \u0026(\u00261 * 2)\n  step :triple, with: \u0026(\u00261 * 3)\nend\n```\n\n## Graph\n\nYou may visualise your pipelines using `Opus.Graph`:\n\n```elixir\nOpus.Graph.generate(:your_app)\n# =\u003e {:ok, \"Graph file has been written to your_app_opus_graph.png\"}\n```\n\n:exclamation: This feature requires the [`opus_graph`][opus_graph] package to be installed, add it in your\nmix.exs.\n\n```elixir\ndefp deps do\n  {:opus_graph, \"~\u003e 0.1\", only: [:dev]}\nend\n```\n\n### Setup\n\nFirst make sure to add `graphvix` to your dependencies:\n\n```elixir\n# in mix.exs\n\ndefp deps do\n  [\n    {:opus, \"~\u003e 0.5\"},\n    {:graphvix, \"~\u003e 0.5\", only: [:dev]}\n  ]\nend\n\n```\n\nThis feature uses [graphviz][graphviz], so make sure to have it\ninstalled. To install it:\n\n```shell\n# MacOS\n\nbrew install graphviz\n```\n\n```shell\n# Debian / Ubuntu\n\napt-get install graphviz\n```\n\n`Opus.Graph` is in fact a pipeline and its visualisation is:\n\n![graph-png](https://i.imgur.com/41kHjZL.png)\n\nYou can customise the visualisation:\n\n```elixir\nOpus.Graph.generate(:your_app, %{filetype: :svg})\n# =\u003e {:ok, \"Graph file has been written to your_app_opus_graph.svg\"}\n```\n\nRead the available visualisation options [here][hexdocs-graph].\n\n## Influences\n\n* [dry.rb - transaction][dryrb-transaction]\n* [trailblazer - operation][trailblazer-operation]\n\n## Press\n\n* [Quiqup Engineering - How to Create Beautiful Pipelines with Opus](https://medium.com/quiqup-engineering/how-to-create-beautiful-pipelines-on-elixir-with-opus-f0b688de8994)\n* [Pagerduty - How I Centralized our Scattered Business Logic Into One Clear Pipeline for our Elixir Webhook Service](https://www.pagerduty.com/eng/elixir-webhook-service/)\n* [A Slack bookmarking application in Elixir with Opus](https://zorbash.com/post/slack-bookmarks-collaboration-elixir/)\n* [Opus Telemetry](https://zorbash.com/post/phoenix-telemetry/)\n\nUsing Opus in your company / project?  \nLet us know by submitting an issue describing how you use it.\n\n## License\n\nCopyright (c) 2018 Dimitris Zorbas, MIT License.\nSee [LICENSE.txt](https://github.com/zorbash/opus/blob/master/LICENSE.txt) for further details.\n\n[hex-retry]: https://github.com/safwank/ElixirRetry/blob/master/lib/retry/delay_streams.ex\n[hexdocs-graph]: https://hexdocs.pm/opus/Opus.Graph.html\n[graphviz]: https://www.graphviz.org/\n[dryrb-transaction]: https://dry-rb.org/gems/dry-transaction/\n[trailblazer-operation]: http://trailblazer.to/gems/operation/2.0/\n[medium-blogpost]: https://medium.com/quiqup-engineering/how-to-create-beautiful-pipelines-on-elixir-with-opus-f0b688de8994\n[opus_graph]: https://github.com/zorbash/opus_graph\n[opus-telemetry]: https://hexdocs.pm/opus/Opus.Telemetry.html\n[post-opus-telemetry]: https://zorbash.com/post/phoenix-telemetry/\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzorbash%2Fopus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fzorbash%2Fopus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fzorbash%2Fopus/lists"}