{"id":14968799,"url":"https://github.com/supabase/libcluster_postgres","last_synced_at":"2025-05-15T10:07:00.372Z","repository":{"id":210064182,"uuid":"723099820","full_name":"supabase/libcluster_postgres","owner":"supabase","description":"Postgres strategy for libcluster","archived":false,"fork":false,"pushed_at":"2025-04-17T09:46:22.000Z","size":826,"stargazers_count":218,"open_issues_count":4,"forks_count":12,"subscribers_count":20,"default_branch":"main","last_synced_at":"2025-05-12T17:22:58.518Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"https://supabase.com/blog/elixir-clustering-using-postgres","language":"Elixir","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/supabase.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null},"funding":{"github":["supabase"],"patreon":null,"open_collective":null,"ko_fi":null,"tidelift":null,"community_bridge":null,"liberapay":null,"issuehunt":null,"otechie":null,"custom":null}},"created_at":"2023-11-24T17:25:05.000Z","updated_at":"2025-05-09T06:06:24.000Z","dependencies_parsed_at":"2023-11-30T16:29:16.754Z","dependency_job_id":"4e48f853-9141-4907-8916-c64085b625a2","html_url":"https://github.com/supabase/libcluster_postgres","commit_stats":{"total_commits":17,"total_committers":6,"mean_commits":"2.8333333333333335","dds":0.7058823529411764,"last_synced_commit":"3d6511d8247471a2fb307df23f942ed09a5fce7c"},"previous_names":["supabase/libcluster_postgres"],"tags_count":4,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supabase%2Flibcluster_postgres","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supabase%2Flibcluster_postgres/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supabase%2Flibcluster_postgres/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/supabase%2Flibcluster_postgres/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/supabase","download_url":"https://codeload.github.com/supabase/libcluster_postgres/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254301989,"owners_count":22048003,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-09-24T13:40:35.379Z","updated_at":"2025-05-15T10:06:55.354Z","avatar_url":"https://github.com/supabase.png","language":"Elixir","funding_links":["https://github.com/sponsors/supabase"],"categories":[],"sub_categories":[],"readme":"# libcluster Postgres Strategy\n\n[![Hex version badge](https://img.shields.io/hexpm/v/libcluster_postgres.svg)](https://hex.pm/packages/libcluster_postgres)\n[![License badge](https://img.shields.io/hexpm/l/libcluster_postgres.svg)](https://github.com/supabase/libcluster_postgres/blob/main/LICENSE)\n[![Elixir CI](https://github.com/supabase/libcluster_postgres/actions/workflows/elixir.yaml/badge.svg)](https://github.com/supabase/libcluster_postgres/actions/workflows/elixir.yaml)\n[![ElixirForum](https://img.shields.io/badge/Elixir_Forum-grey)](https://elixirforum.com/t/libcluster-postgres-clustering-strategy-for-libcluster/60053)\n\nPostgres Strategy for [libcluster](https://hexdocs.pm/libcluster/) which is used by Supabase on the [realtime](https://github.com/supabase/realtime), [supavisor](https://github.com/supabase/supavisor) and [logflare](https://github.com/logflare/logflare) projects.\n\nYou can test it out by running `docker compose up`\n\n![example.png](https://github.com/supabase/libcluster_postgres/blob/main/example.png?raw=true)\n\n## Installation\n\nThe package can be installed\nby adding `libcluster_postgres` to your list of dependencies in `mix.exs`:\n\n```elixir\ndef deps do\n  [{:libcluster_postgres, \"~\u003e 0.1\"}]\nend\n```\n\n## How to use it\n\nTo use it, set your configuration file with the informations for your database:\n\n```elixir\nconfig :libcluster,\n  topologies: [\n    example: [\n      strategy: LibclusterPostgres.Strategy,\n      config: [\n          hostname: \"localhost\",\n          username: \"postgres\",\n          password: \"postgres\",\n          database: \"postgres\",\n          port: 5432,\n          parameters: [],\n          # optional, defaults to false\n          ssl: true,\n          # optional, please refer to the Postgrex docs\n          ssl_opts: nil,\n          # optional, defaults to node cookie\n          channel_name: \"cluster\"\n      ],\n    ]\n  ]\n```\n\nThen add it to your supervision tree:\n\n```elixir\ndefmodule MyApp do\n  use Application\n\n  def start(_type, _args) do\n    topologies = Application.get_env(:libcluster, :topologies)\n    children = [\n      # ...\n      {Cluster.Supervisor, [topologies]}\n      # ...\n    ]\n    Supervisor.start_link(children, strategy: :one_for_one)\n  end\nend\n```\n### Why do we need a distributed Erlang Cluster?\n\nAt Supabase, we use clustering in all of our Elixir projects which include [Logflare](https://github.com/Logflare/logflare), [Supavisor](https://github.com/supabase/supavisor) and [Realtime](https://github.com/supabase/realtime). With multiple servers connected we can load shed, create globally distributed services and provide the best service to our customers so we’re closer to them geographically and to their instances, reducing overall latency.\n\n![Example of Realtime architecture where a customer from CA will connect to the server closest to them and their Supabase instance](https://github.com/supabase/libcluster_postgres/blob/main/realtime_example.png?raw=true)\n\nExample of Realtime architecture where a customer from CA will connect to the server closest to them and their Supabase instance\nTo achieve a connected cluster, we wanted to be as cloud-agnostic as possible. This makes our self-hosting options more accessible. We don’t want to introduce extra services to solve this single issue - Postgres is the logical way to achieve it.\n\nThe other piece of the puzzle was already built by the Erlang community being the defacto library to facilitate the creation of connected Elixir servers: [libcluster](https://github.com/bitwalker/libcluster).\n\n### What is libcluster?\n\n[libcluster](https://github.com/bitwalker/libcluster) is the go-to package for connecting multiple BEAM instances and setting up healing strategies. libcluster provides out-of-the-box strategies and it allows users to define their own strategies by implementing a simple behavior that defines cluster formation and healing according to the supporting service you want to use.\n\n### How did we use Postgres?\n\nPostgres provides an event system using two commands: [NOTIFY](https://www.postgresql.org/docs/current/sql-notify.html) and [LISTEN](https://www.postgresql.org/docs/current/sql-listen.html) so we can use them to propagate events within our Postgres instance.\n\nTo use this features, you can use psql itself or any other Postgres client. Start by listening on a specific channel, and then notify to receive a payload.\n\n```markdown\npostgres=# LISTEN channel;\nLISTEN\npostgres=# NOTIFY channel, 'payload';\nNOTIFY\nAsynchronous notification \"channel\" with payload \"payload\" received from server process with PID 326.\n```\n\nNow we can replicate the same behavior in Elixir and [Postgrex](https://hex.pm/packages/postgrex) within IEx (Elixir's interactive shell).\n\n```elixir\nMix.install([{:postgrex, \"~\u003e 0.17.3\"}])\nconfig = [\n  hostname: \"localhost\",\n  username: \"postgres\",\n  password: \"postgres\",\n  database: \"postgres\",\n  port: 5432\n]\n{:ok, db_notification_pid} = Postgrex.Notifications.start_link(config)\nPostgrex.Notifications.listen!(db_notification_pid, \"channel\")\n{:ok, db_conn_pid} = Postgrex.start_link(config)\nPostgrex.query!(db_conn_pid, \"NOTIFY channel, 'payload'\", [])\n\nreceive do msg -\u003e IO.inspect(msg) end\n# Mailbox will have a message with the following content:\n# {:notification, #PID\u003c0.223.0\u003e, #Reference\u003c0.57446457.3896770561.212335\u003e, \"channel\", \"test\"}\n```\n\n### Building the strategy\n\nUsing the libcluster `Strategy` behavior, inspired by [this GitHub repository](https://github.com/kevbuchanan/libcluster_postgres) and knowing how `NOTIFY/LISTEN` works, implementing a strategy becomes straightforward:\n\n1. We send a `NOTIFY` to a channel with our `node()` address to a configured channel\n\n```elixir\n# lib/cluster/strategy/postgres.ex:52\ndef handle_continue(:connect, state) do\n    with {:ok, conn} \u003c- Postgrex.start_link(state.meta.opts.()),\n         {:ok, conn_notif} \u003c- Postgrex.Notifications.start_link(state.meta.opts.()),\n         {_, _} \u003c- Postgrex.Notifications.listen(conn_notif, state.config[:channel_name]) do\n      Logger.info(state.topology, \"Connected to Postgres database\")\n\n      meta = %{\n        state.meta\n        | conn: conn,\n          conn_notif: conn_notif,\n          heartbeat_ref: heartbeat(0)\n      }\n\n      {:noreply, put_in(state.meta, meta)}\n    else\n      reason -\u003e\n        Logger.error(state.topology, \"Failed to connect to Postgres: #{inspect(reason)}\")\n        {:noreply, state}\n    end\n  end\n```\n\n2. We actively listen for new `{:notification, pid, reference, channel, payload}` messages and connect to the node received in the payload\n\n```elixir\n# lib/cluster/strategy/postgres.ex:80\ndef handle_info({:notification, _, _, _, node}, state) do\n    node = String.to_atom(node)\n\n    if node != node() do\n      topology = state.topology\n      Logger.debug(topology, \"Trying to connect to node: #{node}\")\n\n      case Strategy.connect_nodes(topology, state.connect, state.list_nodes, [node]) do\n        :ok -\u003e Logger.debug(topology, \"Connected to node: #{node}\")\n        {:error, _} -\u003e Logger.error(topology, \"Failed to connect to node: #{node}\")\n      end\n    end\n\n    {:noreply, state}\n  end\n```\n\n3. Finally, we configure a heartbeat that is similar to the first message sent for cluster formation so libcluster is capable of heal if need be\n\n```elixir\n# lib/cluster/strategy/postgres.ex:73\ndef handle_info(:heartbeat, state) do\n    Process.cancel_timer(state.meta.heartbeat_ref)\n    Postgrex.query(state.meta.conn, \"NOTIFY #{state.config[:channel_name]}, '#{node()}'\", [])\n    ref = heartbeat(state.config[:heartbeat_interval])\n    {:noreply, put_in(state.meta.heartbeat_ref, ref)}\nend\n```\n\nThese three simple steps allow us to connect as many nodes as needed, regardless of the cloud provider, by utilising something that most projects already have: a Postgres connection.\n\n## Acknowledgements\n\nA special thank you to [@gotbones](https://twitter.com/gotbones) for creating libcluster and [@kevinbuch\\_](https://twitter.com/kevinbuch_) for the original inspiration for this strategy.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsupabase%2Flibcluster_postgres","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsupabase%2Flibcluster_postgres","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsupabase%2Flibcluster_postgres/lists"}