{"id":26586352,"url":"https://github.com/salemove/ex_freddy","last_synced_at":"2025-03-23T11:17:40.883Z","repository":{"id":45078956,"uuid":"77840313","full_name":"salemove/ex_freddy","owner":"salemove","description":"Elixir OTP behaviours for creating AMQP publishers and consumers","archived":false,"fork":false,"pushed_at":"2025-03-04T09:45:18.000Z","size":319,"stargazers_count":14,"open_issues_count":1,"forks_count":3,"subscribers_count":41,"default_branch":"master","last_synced_at":"2025-03-22T15:18:07.713Z","etag":null,"topics":["elixir"],"latest_commit_sha":null,"homepage":"https://hexdocs.pm/freddy/","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/salemove.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2017-01-02T14:48:16.000Z","updated_at":"2025-03-04T09:45:21.000Z","dependencies_parsed_at":"2023-12-12T13:42:39.443Z","dependency_job_id":null,"html_url":"https://github.com/salemove/ex_freddy","commit_stats":{"total_commits":127,"total_committers":9,"mean_commits":14.11111111111111,"dds":0.2913385826771654,"last_synced_commit":"03998f5667508f37a012c4e5897e92923cde6bf4"},"previous_names":[],"tags_count":13,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salemove%2Fex_freddy","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salemove%2Fex_freddy/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salemove%2Fex_freddy/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/salemove%2Fex_freddy/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/salemove","download_url":"https://codeload.github.com/salemove/ex_freddy/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":245090875,"owners_count":20559298,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elixir"],"created_at":"2025-03-23T11:17:40.262Z","updated_at":"2025-03-23T11:17:40.873Z","avatar_url":"https://github.com/salemove.png","language":"Elixir","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Freddy\n\n[![Build Status](https://travis-ci.org/salemove/ex_freddy.svg?branch=master)](https://travis-ci.org/salemove/ex_freddy)\n\nOTP behaviours for creating AMQP publishers and consumers.\n\n**The project is in active development stage, expect breaking changes between minor versions up to 1.0.**\n\n## Installation\n\nAdd `freddy` to your list of dependencies in `mix.exs`:\n```elixir\ndef deps do\n  [{:freddy, \"~\u003e 0.15.0\"}]\nend\n```\n\n## Stable connection\n\nNeither [official RabbitMQ client](https://github.com/rabbitmq/rabbitmq-erlang-client),\nnor its Elixir wrapper [amqp](https://github.com/pma/amqp) provide an out-of-box way\nto create a stable monitored connection to RabbitMQ server, which will be gracefully\nreestablished after server restart or intermittent network failures.\n\nFreddy attempts to provide such abstraction, which is called `Freddy.Connection`. It is\na standard OTP-compliant process that can be easily integrated into OTP supervision tree\nusing standard capabilities.\n\nAll Freddy behaviors (publishers and consumers) require `Freddy.Connection`.\n\nThe connection process can be started like this:\n\n```elixir\n{:ok, conn} = Freddy.Connection.start_link(config)\n```\n\nCheck out [`Freddy.Connection.start_link/2`](https://hexdocs.pm/freddy/Freddy.Connection.html#start_link/2) \nfor available options.\n\nAdd this process to an OTP application supervision tree:\n\n```elixir\ndefmodule MyApp do\n  use Application\n\n  def start(_type, _args) do\n    import Supervisor.Spec\n\n    children = [\n      worker(Freddy.Connection, [[], [name: Freddy.Connection]])\n    ]\n\n    opts = [strategy: :one_for_one, name: MyApp.Supervisor]\n    Supervisor.start_link(children, opts)\n  end\nend\n```\n\nWe recommend to start `Freddy.Connection` and all your publishers and consumers in the OTP\nsupervision tree. Ideally connection and dependent processes should be grouped in one supervisor\nwith restart strategy `:rest_for_one`.\n\n## Connection to multiple hosts\n\nIt is possible to leverage H/A RabbitMQ setup by providing multiple connection options when\nstarting a `Freddy.Connection` process. Don't forget to specify `connection_timeout`, or your\nprocess may stuck in infinite wait loop.\n\n```elixir\nhost1 = [host: \"10.0.100.1\", connection_timeout: 1000]\nhost2 = [host: \"10.0.100.2\", connection_timeout: 1000]\nhost3 = [host: \"10.0.100.3\", connection_timeout: 1000]\n{:ok, conn} = Freddy.Connection.start_link([host1, host2, host3])\n```\n\n`Freddy.Connection` will establish connection to one of the specified hosts, prioritizing them\nby order of appearing in the list. If it can't establish connection to the first host, it will\nimmediately attempt to establish connection to second, and so on. If none of the hosts are responding,\n`Freddy.Connection` will wait a second and attempt to connect to all hosts again.\n\n## Publishers\n\nFreddy provides a behaviour module [`Freddy.Publisher`](https://hexdocs.pm/freddy/Freddy.Publisher.html)\nto implement your own stateful publishers.\n\nCheck out [the behaviour documentation](https://hexdocs.pm/freddy/Freddy.Publisher.html) for information\nabout all available callbacks.\n\nBy default publisher processes encode message payload to JSON before sending the message to RabbitMQ server,\nit is responsibility of consumers to decode message back. This behaviour can be changed by redefining\nthe default implementation of `Freddy.Publisher.encode_message/4` callback.\n\n### Example\n\nBelow is an example of how to implement a publishing process that queues up messages when RabbitMQ\nconnection is disrupted (instead of silently dropping them):\n\n```elixir\ndefmodule ReliableBroadcaster do\n  use Freddy.Publisher\n\n  @exchange %Freddy.Core.Exchange{name: \"notifications\", type: :fanout, opts: [durable: true]}\n  @config [exchange: @exchange]\n\n  def start_link(connection, opts \\\\ []) do\n    Freddy.Publisher.start_link(__MODULE__, connection, @config, nil, opts)\n  end\n\n  @impl true\n  def init(_) do\n    state = %{connected: false, queue: :queue.new()}\n    {:ok, state}\n  end\n\n  @impl true\n  # This function is called after an exchange has been declared\n  def handle_connected(meta, %{queue: queue} = state) do\n    new_state = %{state | connected: true, queue: drain_queue(queue, meta)}\n    {:noreply, new_state}\n  end\n\n  @impl true\n  # This function is called right after disconnect\n  def handle_disconnected(_reason, state) do\n    {:noreply, %{state | connected: false}}\n  end\n\n  @impl true\n  # Catch messages before publication and queue them up if connection is not available\n  def before_publication(\n    payload, routing_key, opts, %{connected: connected?, queue: queue} = state\n  ) do\n    if not connected? do\n      message = {payload, routing_key, opts}\n      {:ignore, %{state | queue: :queue.in(message, queue)}}\n    else\n      {:ok, state}\n    end\n  end\n\n  defp drain_queue(queue, meta) do\n    case :queue.out(queue) do\n      {{:value, {payload, routing_key, opts}}, new_queue} -\u003e\n        Freddy.Publisher.publish(meta, payload, routing_key, opts)\n        drain_queue(new_queue, meta)\n\n      {:empty, empty_queue} -\u003e\n        empty_queue\n    end\n  end\nend\n```\n\n## Consumers\n\nStateful consumer processes are implemented with [`Freddy.Consumer`](https://hexdocs.pm/freddy/Freddy.Consumer.html)\nbehaviour module.\n\nCheck out [the behaviour documentation](https://hexdocs.pm/freddy/Freddy.Consumer.html) for\ninformation about all available callbacks.\n\nConsumer process typically works as follows:\n\n1. After initialization consumer opens an AMQP channel\n2. An exchange and a queue are declared using the opened channel\n3. The declared queue is bound to the exchange (see\n   [RabbitMQ routing tutorial](https://www.rabbitmq.com/tutorials/tutorial-four-elixir.html)),\n4. The consumer process starts consumption from the queue\n5. Broker confirms that consumer process is registered on the server\n6. Messages from the queue are delivered to the consumer process\n7. When consumer has successfully processed a message, it acknowledges the message on server,\n   and the server removes the message from the queue.\n\n### Message format\n\nBy default consumer processes assume that incoming messages payload are encoded into JSON and decode\nthem before starting processing. This behaviour can be changed by redefining the default implementation\nof `Freddy.Consumer.decode_message/3` callback.\n\n### Example\n\nThis is an example of a process that creates an exclusive queue with server-generated name,\nbinds this queue to fanout exchange \"notifications\" and processes each message in a separate\nasynchronous task.\n\nPlease note that as we process messages asynchronously and we don't consume messages with `:no_ack`\noption, we must explicitly acknowledge or reject processed messages using `Freddy.Consumer.ack/2` or\n`Freddy.Consumer.reject/2`.\n\nThe chosen approach is very naive and we do not recommend to use this code in production, it\nis here only for educational purposes.\n\n```elixir\ndefmodule NotificationsProcessor do\n  use Freddy.Consumer\n\n  @config [\n    exchange: [name: \"notifications\", type: :fanout],\n    queue: [opts: [exclusive: true, auto_delete: true]],\n    qos: [prefetch_count: 10],\n    routing_keys: [\"#\"]\n  ]\n\n  def start_link(conn, handler_mfa) do\n    Freddy.Consumer.start_link(__MODULE__, conn, @config, handler_mfa)\n  end\n\n  @impl true\n  def init(handler) do\n    {:ok, handler}\n  end\n\n  @impl true\n  def handle_message(payload, %{routing_key: key} = meta, {m, f, a} = handler) do\n    Task.start_link(fn -\u003e\n      try do\n        apply(m, f, [payload, key | a])\n\n        Freddy.Consumer.ack(meta)\n      rescue _error -\u003e\n        # we might want to log error here too\n        Freddy.Consumer.reject(meta, requeue: true)\n      end\n    end)\n\n    {:noreply, state}\n  end\nend\n```\n\n## Remote Procedure Call (RPC)\n\n### Client\n\n[RPC Client](https://www.rabbitmq.com/tutorials/tutorial-six-elixir.html) in a nutshell is a combination of\nconsumer and publisher. A process publishes RPC requests into default or any other exchange and expects a\nserver to publish response message to a special anonymous queue from which an RPC client process consumes.\n\nEach request contains a name of reply queue and a correlation ID - an identifier which allows RPC\nclient to understand for which request the response has arrived. When server sends a reply, it publishes a\nmessage into default exchange with a routing key equal to the name of the client reply queue and copies\ncorrelation ID from the request to the response message.\n\nA diagram below illustrates an RPC request-response lifecycle:\n\n```\n  +------------+             +----------------+\n  |   Client   |------------\u003e|  Pub exchange  |\n  +------------+             +----------------+\n         ^                            |\n         |                            |\n         |                            v\n  +-------------+            +----------------+\n  | Reply queue |            |  Server queue  |\n  +-------------+            +----------------+\n         ^                            |\n         |                            |\n         |                            v\n+----------------+           +----------------+\n|Default exchange|\u003c----------|   RPC Server   |\n+----------------+           +----------------+\n```\n\n[`Freddy.RPC.Client`](https://hexdocs.pm/freddy/Freddy.RPC.Client.html) is also implemented as behaviour\nmodule, leaving you an opportunity to customize your application logic through set of callback functions.\n\nCheck out [`Freddy.RPC.Client`](https://hexdocs.pm/freddy/Freddy.RPC.Client.html) documentation for\ninformation about available callbacks.\n\n#### Example\n\nThis is an example of RPC client that publishes requests to the default exchange, logs unsuccessful requests\nand emits response time to a StatsD server.\n\n```elixir\ndefmodule RPC.Client do\n  use Freddy.RPC.Client\n\n  require Logger\n\n  alias Freddy.RPC.Request\n\n  @server_queue \"RemoteService\"\n\n  def start_link(conn, opts \\\\ []) do\n    Freddy.RPC.Client.start_link(__MODULE__, conn, [], nil, opts)\n  end\n\n  def request(client, payload) do\n    Freddy.RPC.Client.request(client, @server_queue, payload)\n  end\n\n  @impl true\n  def on_timeout(request, state) do\n    Logger.warn(\"Request to server #{request.routing_key} timed out after #{Request.duration(request)} ms\")\n    {:reply, {:error, :timeout}, state}\n  end\n\n  @impl true\n  def on_return(request, state) do\n    Logger.warn(\"Request to server #{request.routing_key} couldn't be routed\")\n    {:reply, {:error, :no_route}, state}\n  end\n\n  @impl true\n  def on_response(response, request, state) do\n    send_metrics(request)\n    {:reply, response, state}\n  end\n\n  defp send_metrics(request) do\n    MyApp.Statix.histogram(\"rpc.request\", Request.duration(request), tags: [\"server:#{request.routing_key}\"])\n  end\nend\n```\n\n#### Testing\n\nWe recommend you to structure your code in such way that your test environment will not use real RPC client.\n\nFor example, you can create a client behavior and use [Mox](https://github.com/plataformatec/mox) to mock all\ncalls to a client. This way you can test how your code communicates with RPC client. Please refer to\n[Mox documentation](https://hexdocs.pm/mox/Mox.html) for more information about mocks and explicit contracts.\n\nIf you need to test RPC client itself, you can hook into request lifecycle and use `before_request/2` callback\nto return required responses. It is recommended to use fake connection in test environment:\n\n```elixir\ndefmodule MockClient do\n  use Freddy.RPC.Client\n\n  def start_link(conn) do\n    Freddy.RPC.Client.start_link(__MODULE__, conn, [], [])\n  end\n\n  def flush(client) do\n    Freddy.RPC.Client.call(client, :flush)\n  end\n\n  def before_request(request, sink) do\n    {:reply, :ok, [request | sink]}\n  end\n\n  def handle_call(:flush, sink) do\n    {:reply, Enum.reverse(sink), []}\n  end\nend\n```\n\nAnd use it in tests:\n```elixir\ntest \"sends an RPC request\" do\n  {:ok, conn} = Freddy.Connection.start_link(adapter: :sandbox)\n  {:ok, client} = MockClient.start_link(conn)\n  MyLib.call(client: client)\n\n  assert [%{routing_key: \"server\"}] = MockClient.flush(client)\nend\n```\n\n### Server\n\nSimilarly to RPC client, RPC server is also a combination of consumer and publisher, but on the server side,\nthe consumer is used to accept RPC request messages, whilst the publisher is used to send back response messages.\n\nCheck out [`Freddy.RPC.Server`](https://hexdocs.pm/freddy/Freddy.RPC.Server.html) documentation for\ninformation about available callbacks.\n\n#### Message format\n\nBy default server processes assume that incoming messages payload are encoded into JSON and decode\nthem before processing. This behaviour can be changed by redefining the default implementation\nof `Freddy.RPC.Server.decode_request/3` callback.\n\n### Acknowledgement mode\n\nBy default RPC server starts in automatic acknowledgement mode. It means that all\nincoming requests will be acknowledged automatically by RabbitMQ server once delivered\nto a client (RPC server process).\n\nIf your logic requires manual acknowledgements, you should start server with configuration\noption `[consumer: [no_ack: false]]` and acknowledge messages manually using\n`Freddy.RPC.Server.ack/2` function.\n\n#### Example\n\nBelow is an example of simple synchronous echo RPC server, which can process only one request at a time:\n\n```elixir\ndefmodule RPC.Server do\n  use Freddy.RPC.Server\n\n  def start_link(conn) do\n    config = [\n      queue: [name: \"EchoServer\"]\n    ]\n\n    Freddy.RPC.Server.start_link(__MODULE__, conn, config, [])\n  end\n\n  def handle_request(payload, _meta, state) do\n    {:reply, payload, state}\n  end\nend\n```\n\nA slightly more complicated example of asynchronous RPC server, which processes every request in a separate\nprocess with manual acknowledgement of processed requests:\n\n```elixir\ndefmodule RPC.Server do\n  use Freddy.RPC.Server\n\n  import Freddy.RPC.Server, only: [ack: 1, reply: 2]\n\n  def start_link(conn, handler) when is_function(handler, 1) do\n    config = [\n      queue: [name: \"AsyncServer\"],\n      qos: [prefetch_count: 100], # this is protection from DoS\n      consumer: [no_ack: false] # this enables manual acknowledgements\n    ]\n\n    Freddy.RPC.Server.start_link(__MODULE__, conn, config, handler)\n  end\n\n  @impl true\n  def init(handler) do\n    {:ok, task_sup} = Task.Supervisor.start_link()\n\n    {:ok, {task_sup, handler}}\n  end\n\n  @impl true\n  def handle_request(request, meta, {task_sup, handler}} = state) do\n    Task.Supervisor.start_child(task_sup, fn -\u003e\n      result = handler.(request)\n      ack(meta)\n      reply(meta, result)\n    end)\n\n    {:noreply, state}\n  end\nend\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalemove%2Fex_freddy","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsalemove%2Fex_freddy","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsalemove%2Fex_freddy/lists"}