{"id":32172222,"url":"https://github.com/midas/productive","last_synced_at":"2026-02-20T16:01:31.266Z","repository":{"id":57536613,"uuid":"61334384","full_name":"midas/productive","owner":"midas","description":null,"archived":false,"fork":false,"pushed_at":"2017-10-18T23:57:33.000Z","size":16,"stargazers_count":8,"open_issues_count":0,"forks_count":2,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-10-21T18:41:32.198Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/midas.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2016-06-17T00:24:16.000Z","updated_at":"2023-09-01T12:11:06.000Z","dependencies_parsed_at":"2022-08-28T23:32:01.976Z","dependency_job_id":null,"html_url":"https://github.com/midas/productive","commit_stats":null,"previous_names":[],"tags_count":4,"template":false,"template_full_name":null,"purl":"pkg:github/midas/productive","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/midas%2Fproductive","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/midas%2Fproductive/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/midas%2Fproductive/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/midas%2Fproductive/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/midas","download_url":"https://codeload.github.com/midas/productive/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/midas%2Fproductive/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29656589,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-20T09:27:29.698Z","status":"ssl_error","status_checked_at":"2026-02-20T09:26:12.373Z","response_time":59,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-10-21T18:31:49.950Z","updated_at":"2026-02-20T16:01:31.260Z","avatar_url":"https://github.com/midas.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Productive\n\nAn assembly line like pipeline that supports binary or greater logic branching using pattern \nmatching in a way that is easy to reason about and extend.  Think [plug](https://github.com/elixir-lang/plug) \nwith the ability to build any _product_ instead of a _connection_.\n\n\n## Why?\n\nWe are very fortunate as Elixir developers as we are already provided the `case` special form, the\n`|\u003e` operator and now the `with` special form. Using these tools one can already express a procedure \nas a pipeline.  However, when a pipeline becomes more complex due to quantity of steps of branching\nbewteen the steps, these low level language contructs begin to break down.\n\nLet's consider the use case of reading a BEAM file's `:abstract_code` chunks.\n\n  1. Use case initially implemented with `case`\n\n      ```elixir\n      case File.read(path) do\n        {:ok, binary} -\u003e\n          case :beam_lib.chunks(binary, :abstract_code) do\n            {:ok, data} -\u003e\n              {:ok, wrap(data)}\n            error -\u003e\n              error\n          end\n        error -\u003e\n          error\n      end\n      ```\n\n  1. Use case reafactored to use `|\u003e` and functions\n\n      ```elixir\n      path\n      |\u003e File.read()\n      |\u003e read_chunks()\n      |\u003e wrap()\n\n      defp read_chunks({:ok, binary}) do\n        {:ok, :beam_lib.chunks(binary, :abstract_code)\n      end\n      defp read_chunks(error), do: error\n\n      defp wrap({:ok, data}) do\n        {:ok, wrap(data)}\n      end\n      defp wrap(error), do: error\n      ```\n\n  1. Use case reafactored to use `with`\n\n      ```elixir\n      with {:ok, binary} \u003c- File.read(path),\n           {:ok, data} \u003c- :beam_lib.chunks(binary, :abstract_code),\n           do: {:ok, wrap(data)}\n      ```\n\nWhile `case`, `|\u003e` and `with` are very useful, they each have limitations. \n\n  1. `case` tends to hide the high level steps in the pipeline due to branching and nesting.  Additionally, it can become very \n     ugly fast when there are many steps and/or many branching conditions.  Notice in the `case` implementation above, with\n     only two steps it is already a little hard to quickly pick out the steps.\n\n  1. `with` is useful only when each step in the flow has a binary branch, ie. `{:ok, something}` or `{:error, error}`.\n\n  1. `|\u003e` can also become very ugly fast when there are many steps and/or many branching conditions.  While it is not as hard \n     to reason about as the `case` implementation, it can still become unruly.  Additionally, it can be hard to develop a \n     pattern for dealing with reusing steps in other pipelines.\n\n### Terminology\n\nBefore we implement this use case as a _productive_ pipeline, let's clarify some terminology.\n\n  1. A **product** is the data structure that accumulates a pipelines state.  A product is passed into and returned from every \n     step in a pipeline. As the name infers, a product is the ultimate thing we are building in the pipeline. The product of\n     a [plug](https://github.com/elixir-lang/plug) is the connection. While not a requirement, it is recommended to implement \n     the product as a struct.\n\n  1. A **pipeline** is the enumerated steps necessary to complete a task.\n\n  1. A **step** is a single unit of work. The step defines one or functions to determine the state of the product \n     and is how branching is accomplished.  A step also defines one or more functions to perform work based on the determined\n     state of the product.  Both the state determination and work performing are multi-clause functions employing pattern matching\n     to select the correct clause.  While a step can do as much work as one desires, it is recommended for a step to do a single thing\n     well, as it will more easily compose with other steps and promote code resuse.\n\n\n### A Simple Example\n\nUsing the same use case from above let's implement this pipeline using _productive_.\n\n  1. Define a product\n\n      ```elixir\n      defmodule AbstractCodeChunks do\n        defstruct code_chunks: nil,\n                  errors: [],\n                  file_contents: nil,\n                  filepath: nil,\n                  halted: false\n\n        def init, do: %AbstractCodeChunks{}\n\n        def init(args) is_list(args) do\n          %AbstractCodeChunks{\n            filepath: Keyword.get(args, :filepath)\n          }\n        end\n      end\n      ```\n\n  1. Define a pipeline\n\n      ```elixir\n      defmodule ReadBeamFileAbstractCodeChunks do\n        use Productive.Pipeline\n\n        # notice how wasy it is to reason about the high level steps and \n        # order of steps  to complete a product\n        step ReadFile\n        step ExtractAbstractCodeChunks\n        step WrapChunks\n      end\n      ```\n\n  1. Define the steps\n\n      ```elixir\n      defmodule ReadFile do\n        use Productive.Step\n\n        @read   \"read\"\n        @unread \"unread\"\n\n        # Block invalid start state\n        def prepare(%{filepath: nil}, _opts), do: raise \"filepath cannot be nil\"\n\n        # Valid start states\n        def prepare(%{file_contents: nil, filepath: _}, _opts), do: @unread\n        def prepare(_product, _opts), do: @read\n\n        def work(@read, product, _opts), do: product\n\n        def work(@unread, %{filepath: filepath} = product, _opts) do\n          %{product | file_contents: File.read(filepath)}\n        end\n      end\n\n      defmodule ExtractAbstractCodeChunks do\n        use Productive.Step\n\n        @read \"read\"\n\n        # Block invalid start state\n        def prepare(%{file_contents: nil}, _opts), do: raise \"file_contents cannot be nil\"\n\n        # Valid start states\n        def prepare(_product, _opts), do: @read\n\n        def work(@read, product, _opts) do\n          :beam_lib.chunks(binary, :abstract_code)\n          |\u003e process_chunks( product )\n        end\n\n        defp process_chunks({:ok, data}, product), do: %{product | code_chunks: {:ok, data}}\n\n        defp process_chunks(error, product) do\n          product\n          |\u003e add_errors_and_halt!( error )\n        end\n      end\n\n      defmodule WrapChunks do\n        use Productive.Step\n\n        @chunks_extracted \"chunks_extracted\"\n\n        # Block invalid start state\n        def prepare(%{code_chunks: nil}, _opts), do: raise \"code_chunks cannot be nil\"\n\n        # Valid start states\n        def prepare(%{code_chunks: _}, _opts), do: @chunks_extracted\n\n        def work(@chunks_extracted, %{code_chunks: code_chunks} = product, _opts) do\n          %{product | code_chunks: wrap(code_chunks)}\n        end\n\n        defp wrap(data) do\n          # does something ...\n        end\n      end\n      ```\n  1. Use the pipeline\n\n    ```elixir\n    product = AbstractCodeChunks.init(filepath: \"/some/file/path\")\n\n    case ReadBeamFileAbstractCodeChunks.call(product) do\n      {:ok, product} -\u003e\n        # Do something with the finialized product\n      {:error, errors} -\u003e\n        raise Enum.join(errors, \" ; \")\n    end\n\n    # or expressed with pipelines and functions ##########\n\n    AbstractCodeChunks.init(filepath: \"/some/file/path\")\n    |\u003e ReadBeamFileAbstractCodeChunks.call\n    |\u003e process_results\n    \n    defp process_results({:ok, product}) do\n      # Do something with the finialized product\n    end\n    \n    defp process_results({:error, errors})\n      raise Enum.join(errors, \" ; \")\n    end\n    ```\n\nIt should be obvious that implementing a pipeline as simple as this using _productive_ is overkill. I \nwould favor the implementation using `with` for this use case.  However, this use case did provide a \nsimple example to show a 1-to-1 comparison.\n\nHowever, you can see that each step of the pipeline is portable. The only API requirements are in the\ndata structure of the product (more specifically only the part of the product the step operates on). Thus, \ncode resuse becomes easy and encouraged. \n\nAdditionally, all knowledge of state calculation and work performance is captured in a single module which \nmakes reasoning about the step and the greater pipeline much easier. While examining the implementation of\na step, all implementation of other steps in the pipeline are located in a different module and not obscuring \nthis step's logic. Everything the step needs to know is carried with the product. Also, the step does not care what \nstep occurred before or after it. The step only needs to be able to handle the current calculated state of the\nproduct.\n\nDebugging the pipeline becomes easier as the state is easily examinable by interrogating the product. Not having\nstate spread out in random places makes reasoning about the pipeline much easier.\n\nFinally, each step of the pipeline is now easily unit testable.\n\nWhile this use case only has binary branching between steps, a _productive_ pipeline can have infinite \nbranching between steps.\n\n### A More Complex Example\n\nNext, let's look at a more complex use case so we can see the real power of _productive_.\n\nTODO\n\nOur use case is TODO\n\nImplemented as a `case` statment we can see the deficiencies of this factoring of the code.\n\nTODO\n\nRefactored using `|\u003e` and functions you can see the deficiencies of this factoring of the code.\n\nTODO\n\n## Installation\n\nIf [available in Hex](https://hex.pm/docs/publish), the package can be installed as:\n\n  1. Add productive to your list of dependencies in `mix.exs`:\n\n        def deps do\n          [{:productive, \"~\u003e 0.2.0\"}]\n        end\n\n  2. Ensure productive is started before your application:\n\n        def application do\n          [applications: [:productive]]\n        end\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmidas%2Fproductive","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmidas%2Fproductive","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmidas%2Fproductive/lists"}