{"id":13561030,"url":"https://github.com/antonmi/ALF","last_synced_at":"2025-04-03T16:31:53.505Z","repository":{"id":42523078,"uuid":"425955021","full_name":"antonmi/ALF","owner":"antonmi","description":"Flow-based Application Layer Framework","archived":false,"fork":false,"pushed_at":"2024-05-25T10:39:48.000Z","size":5341,"stargazers_count":205,"open_issues_count":0,"forks_count":6,"subscribers_count":7,"default_branch":"main","last_synced_at":"2025-03-29T04:06:55.713Z","etag":null,"topics":["elixir","flow-based-programming","framework","pipeline"],"latest_commit_sha":null,"homepage":"","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/antonmi.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-11-08T18:45:19.000Z","updated_at":"2025-03-24T04:12:47.000Z","dependencies_parsed_at":"2023-01-24T21:45:42.614Z","dependency_job_id":"e9865d68-bda9-4d3d-8e71-1c20a31361a6","html_url":"https://github.com/antonmi/ALF","commit_stats":{"total_commits":211,"total_committers":3,"mean_commits":70.33333333333333,"dds":"0.34123222748815163","last_synced_commit":"3aed0a69a031543f307ba445f9d00e211e5f63af"},"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2FALF","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2FALF/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2FALF/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/antonmi%2FALF/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/antonmi","download_url":"https://codeload.github.com/antonmi/ALF/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247037062,"owners_count":20873089,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir","flow-based-programming","framework","pipeline"],"created_at":"2024-08-01T13:00:51.816Z","updated_at":"2025-04-03T16:31:48.488Z","avatar_url":"https://github.com/antonmi.png","language":"Elixir","readme":"# ALF\n\n[![Hex.pm](https://img.shields.io/hexpm/v/alf.svg?style=flat-square)](https://hex.pm/packages/alf)\n\n## Flow-based Application Layer Framework\n\n#### ALF is a set of flow-control abstractions built on top Elixir GenStage which allows writing programs following the [Flow-Based Programming (FBP)](https://en.wikipedia.org/wiki/Flow-based_programming) approach.\n\n#### ALF is a framework for your application layer, it provides a simple and expressive way of presenting the logic as sequential processing of \"information packets\" (IPs) (or, simply, messages or events), and thus brings high design-, coding-, and run-time observability.\n\n#### ALF is NOT a general-purpose \"language\", so implementing a complex domain logic with it might be questionable (although it is possible).\n\n#### ALF is a successor of the [Flowex](https://github.com/antonmi/flowex) project. Check it's [README](https://github.com/antonmi/flowex#readme) to get the general idea. ALF adds conditional branching, packet cloning, goto statement and other functionalities. Therefore, one can create application trees (graphs) of arbitrary complexity.\n\n### Something to read and watch\n\n[Flow-Based Programming with Elixir and ALF](https://www.youtube.com/watch?v=2XrYd1W5GLo) - Code BEAM 2022\n\n[ALF — Flow-based Application Layer Framework](https://anton-mishchuk.medium.com/alf-flow-based-application-layer-framework-8072ae9b0b6b) - post\n\n[Where is Application Layer](https://medium.com/p/6c65a459543a) - post\n\n[ALF - Flow-based Application Layer Framework](https://www.youtube.com/watch?v=nh4TzOavknM) - Sydney Elixir Meetup\n\n### Broadway, Flow?\n\nWhat's the difference between ALF and [Broadway](https://github.com/dashbitco/broadway) or [Flow](https://github.com/dashbitco/flow)?\n\nThe short answer is: Broadway and Flow are tools for processing streams of data while ALF is a framework for writing general business logic.\n\nThe three libraries are build on top of the GenStage library, so they are similar in a sense that there are GenStages in which you can put your own code. But the focus is completely different.\n\nFlow focuses on \"computations on collections, similar to the Enum and Stream modules\", it's a quite low-level tool for processing large collections of data.\n\nBroadway is about \"data ingestion and data processing pipelines\". The main abstraction are \"data processors\", there are lots of adapters to different data sources and so on.\n\nALF is NOT about data-processing (although you can easily do it with ALF). It's about a FBP-way to build your application layer logic.\n\n## Installation\n\nJust add `:alf` as dependency to your `mix.exs` file.\n\n```elixir\n  defp deps do\n    [\n      {:alf, \"~\u003e 0.11\"}\n    ]\n  end\n```\n\nALF starts its own supervisor (`ALF.DynamicSupervisor`). All the pipelines and managers are started under the supervisor\n\n## Important notice!\n\nIn the version \"0.8\" the interface was changed significantly.\n\nAnd now all the interactions go through the pipeline module itself.\n\n```elixir\n# start and stop\nMyPipeline.start() # instead of ALF.Manager.start(MyPipeline)\nMyPipeline.stop()  # instead of ALF.Manager.stop(MyPipeline) \n\n# send events\nMyPipeline.call(event, opts \\\\ []) # new\nMyPipeline.stream(enumerable, opts \\\\ []) # instead of ALF.Manager.stream_to(enumerable, MyPipeline)\nMyPipeline.cast(event, opts \\\\ []) # new\n```\n\n## Quick start\n\nRead a couple of sections of [Flowex README](https://github.com/antonmi/flowex#readme) to get the basic idea of how your code is put to GenStages.\n\n### Define your pipeline\n\nA pipeline is a list of components defined in the `@components` module variable.\n\n```elixir\ndefmodule ThePipeline do\n  use ALF.DSL\n\n  @components [\n    stage(:add_one),\n    stage(:mult_by_two),\n    stage(:minus_three)\n  ]\n\n  def add_one(event, _opts), do: event + 1\n  def mult_by_two(event, _opts), do: event * 2\n  def minus_three(event, _opts), do: event - 3\nend\n```\n\n### Start the pipeline\n\n```elixir\n:ok = ThePipeline.start()\n```\n\nThis starts a manager (GenServer) with the `ThePipeline` name. The manager starts all the components and puts them under another supervision tree.\n\n![alt text](images/add_mult_minus_pipeline.png \"Your first simple pipeline\")\n\n### Use the pipeline\n\nThere are several ways you can run your pipeline.\nThere are `call/2`, `cast/2` and `stream/2` functions.\n\n`call/2` calls the pipeline and blocks the caller process until the result is returned.\n\n```elixir\nThePipeline.call(1) # returns 1\nThePipeline.call(2, debug: true) # it returns %ALP.IP{} struct\n```\n\n`cast/2` sends event to the pipeline and returns the IP reference immediately.\n\n```elixir\nThePipeline.cast(1) # returns reference like #Reference\u003c0.3669068923.1709703170.126245\u003e\n```\n\nOne can actually receive the result of the `cast/2` back with `send_result: true` option.\n\n```elixir\nref = ThePipeline.cast(1, send_result: true)\nreceive do\n  {^ref, %ALF.IP{event: event}} -\u003e \n    event\nend\n```\n\n`stream/2` is a bit different.\nIt receives a stream or `Enumerable.t` and returns another stream where results will be streamed.\n\n```elixir\ninputs = [1,2,3]\noutput_stream = ThePipeline.stream(inputs)\nEnum.to_list(output_stream) # it returns [1, 3, 5]\n```\nThe `debug: true` option also works for streams\n\n### Parallel processing of several streams\n\nThe ALF pipeline can handle arbitrary amount of events streams in parallel.\nFor example:\n\n```elixir\n stream1 = ThePipeline.stream(0..9)\n stream2 = ThePipeline.stream(10..19)\n stream3 = ThePipeline.stream(20..29)\n\n [result1, result2, result3] =\n   [stream1, stream2, stream3]\n   |\u003e Enum.map(\u0026Task.async(fn -\u003e Enum.to_list(\u00261) end))\n   |\u003e Task.await_many()\n```\n\nCheck [test/examples](https://github.com/antonmi/ALF/tree/main/test/examples) folder for more examples\n\n### Synchronous evaluation\nThere are cases when you don't need the underlying gen_stage infrastructure (a separate process for each component).\nE.g. in tests, or if you debug a wierd error.\nThere is a possibility to run a pipeline synchronously, when everything is run in one process.\nJust pass `sync: true` option to the `start` function.\n\n```elixir\n:ok = ThePipeline.start(sync: true)\n```\n\nThere are only `call/2` and `stream/2` functions available in this mode, no `cast/2`.\n\n### The main idea behind ALF DSL\n\nUser's code that is evaluated inside components must be defined either as a 2-arity function or as a module with the `call/2` function.\nThe name of the function/module goes as a first argument in DSL. And the name also become the component's name.\n\n```elixir\n  stage(:my_fun)\n  # or\n  stage(MyComponent)\n```\n\nwhere `my_fun` is\n```elixir\ndef my_fun(event, opts) do\n  #logic is here\n  new_event\nend\n```\n\nand `MyComponent` is\n\n```elixir\ndefmodule MyComponent do\n  # optional\n  def init(opts), do: %{opts | foo: :bar}\n\n  def call(event, opts) do\n    # logic is here\n    new_event\n  end\nend\n```\n\nMost of the components accept the `opts` argument, the options will be passed as a second argument to the corresponding function.\n\n```elixir\n  stage(MyComponent, opts: [foo: :bar])\n```\n\nCheck `@dsl_options` in [lib/components](https://github.com/antonmi/ALF/tree/main/lib/components) for available options.\n\n## Components overview\n\n\u003cimg src=\"images/components.png\" alt=\"ALF components\" width=\"500\"/\u003e\n\n### Stage\n\nStage is the stateless component where one puts a piece of application logic. It might be a simple 2-arity function or a module with `call/2` function:\n\n```elixir\n  stage(:my_fun, opts: %{foo: bar})\n  # or\n  stage(MyComponent, opts: %{})\n```\n\nwhere `MyComponent` is\n\n```elixir\ndefmodule MyComponent do\n  # optional\n  def init(opts), do: %{opts | foo: :bar}\n\n  def call(event, opts) do\n    # logic is here\n    new_datum\n  end\nend\n```\nThere is the `:count` option that allows running several copies of a stage.\n\n```elixir\n  stage(:my_fun, count: 5)\n```\nUse it for controlling parallelism.\n\n### Composer\n\nComposer is a stateful component, the state of the composer (\"memo\") can be updated on each event.\n\nThink about `Enum.reduce/3`, where one can set the initial value of accumulator and then return a new value after each iteration.\n\n`composer(:sum, memo: 0)`\n\nThe implementation function is a 3-arity function with the \"memo\" as the second argument.\n\nThe function must return a `{list(event), new_memo}` tuple.\n\n```elixir\ndef sum(event, memo, _opts) do\n  new_memo = memo + event\n  {[event], new_memo} \nend\n```\n\nThe first element in the returned value is a list of events, meaning that a composer can produce many events or no events at all.\n\nBoth, having memory and the ability to return several events give the composer component a huge power.\n\nSee, for example, the [telegram_test.exs](https://github.com/antonmi/ALF/tree/main/test/examples/telegram_test.exs) example which solves the famous \"Telegram Problem\".\n\n#### Composer vs Stage\n\nOne may have noticed that the stage component is actually a special case of the composer - no memory, only one event is returned.\n\nHowever, I would keep them as separate components in order to explicitly separate stateless and stateful transformation in a pipeline.\n\n### Switch\n\nSwitch allows to forward IP (information packets) to different branches:\n\n```elixir\nswitch(:my_switch_function,\n        branches: %{\n          part1: [stage(:foo)],\n          part2: [stage(:bar)]\n        },\n        opts: [foo: :bar]\n      )\n# or with module\nswitch(MySwitchModule, ...)\n```\n\nThe `my_switch_function` function is 2-arity function that must return the key of the branch:\n\n```elixir\ndef my_switch_function(event, opts) do\n  if event == opts[:foo], do: :part1, else: :part2\nend\n\n# or\n\ndefmodule MySwitchModule do\n  # optional\n  def init(opts), do: %{opts | baz: :qux}\n\n  def call(event, opts) do\n    if event == opts[:foo], do: :part1, else: :part2\n  end\nend\n```\n\n### Goto\n\nSend packet to a given `goto_point`\n\n```elixir\ngoto(:my_goto_function, to: :my_goto_point, opts: [foo: :bar])\n# or\ngoto(MyGotoModule, to: :my_goto_point, opts: [foo: :bar])\n```\n\nThe `function` function is 2-arity function that must return `true` of `false`\n\n```elixir\ndef my_goto_function(event, opts) do\n  event == opts[:foo]\nend\n```\n\n### GotoPoint\n\nThe `Goto` component companion\n\n```elixir\ngoto_point(:goto_point)\n```\n\n### Done\n\nIf the `condition_fun` returns truthy value, the event will go directly to the consumer.\nIt allows to exit early if the work is already done or when the controlled error occurred in execution flow.\nThe same behavior can be also implemented using \"switch\" or \"goto\", however the \"done\" component is much simpler.\n\n```elixir\ndone(:condition_fun)\n```\n\n### DeadEnd\n\nEvent won't propagate further.\n\n```elixir\ndead_end(:dead_end)\n```\n\n## Implicit components\n\n\u003cimg src=\"images/implicit_components.png\" alt=\"Implicit components\" width=\"5000\"/\u003e\n\n### Producer and Consumer\n\nNothing special to know, these are internal components that put at the beginning and at the end of your pipeline.\n\n### Plug and Unplug\n\nPlug and Unplug are used for transforming events before and after reusable parts of a pipeline.\nThe components can not be used directly and are generated automatically when one use `plug_with` macro. See below.\n\n\n## Components / Pipeline reusing\n\n### `from` macro\n\nOne can easily include components from another pipeline:\n\n```elixir\ndefmodule ReusablePipeline do\n  use ALF.DSL\n  @components [\n    stage(:foo),\n    stage(:bar)\n  ]\nend\n\ndefmodule ThePipeline do\n  use ALF.DSL\n  @components from(ReusablePipeline) ++ [stage(:baz)]\nend\n```\n\n### `plug_with` macro\n\nUse the macro if you include other components that expect different type/format/structure of input events.\n\n```elixir\ndefmodule ThePipeline do\n  use ALF.DSL\n\n  @components [\n                plug_with(AdapterModuleBaz, do: [stage(:foo), stage(:bar)])\n              ] ++\n                plug_with(AdapterModuleForReusablePipeline) do\n                  from(ReusablePipeline)\n                end\n\nend\n```\n\n`plug_with` adds `Plug` component before the components in the block and `Unplug` at the end.\nThe first argument is an \"adapter\" module which must implement the `plug/2` and `unplug/3` functions\n\n```elixir\ndef AdapterModuleBaz do\n  def init(opts), do: opts # optional\n\n  def plug(event, _opts) do\n    # The function is called inside the `Plug` component.\n    # `event` will be put on the \"AdapterModuleBaz\" until IP has reached the \"unplug\" component.\n    # The function must return `new_event` with the structure expected by the following component\n    new_event\n  end\n\n  def unplug(event, prev_event, _opts) do\n    # here one can access previous \"event\" in `prev_event`\n    # transform the event back for the following components.\n    new_event\n  end\nend\n```\n\n## Diagrams\n\nThe amazing thing with the FBP approach is that one can easily visualize the application logic.\nBelow there are several ALF-diagrams for the examples in [test/examples](https://github.com/antonmi/ALF/tree/main/test/examples).\n\n### Bubble sort\n\n![Bubble sort](images/bubble_sort.png \"Bubble sort\")\n\n### Bubble sort with Switch\n![Bubble sort with Switch](images/bubble_sort_with_switch.png \"Bubble sort with Switch\")\n\n### Tic-tac-toe\nSee [tictactoe repo](https://github.com/antonmi/tictactoe)\n![Tic-Tac-Toe](images/tic-tac-toe.png)\n\n","funding_links":[],"categories":["Elixir","Actors"],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonmi%2FALF","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fantonmi%2FALF","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fantonmi%2FALF/lists"}