{"id":20826724,"url":"https://github.com/cedrickchee/postgresql-consul-demo","last_synced_at":"2026-05-19T03:32:47.809Z","repository":{"id":138118500,"uuid":"419625382","full_name":"cedrickchee/postgresql-consul-demo","owner":"cedrickchee","description":"A minimal demo app showing PostgreSQL HA cluster managed by Patroni and Consul in Docker","archived":false,"fork":false,"pushed_at":"2021-10-21T07:34:43.000Z","size":25,"stargazers_count":2,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-03-12T07:27:30.402Z","etag":null,"topics":["consul","docker-compose","educational-project","patroni","postgresql-ha-cluster","postgresql-replication"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cedrickchee.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2021-10-21T07:34:06.000Z","updated_at":"2025-02-21T15:54:19.000Z","dependencies_parsed_at":"2024-03-26T12:01:19.486Z","dependency_job_id":null,"html_url":"https://github.com/cedrickchee/postgresql-consul-demo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cedrickchee/postgresql-consul-demo","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fpostgresql-consul-demo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fpostgresql-consul-demo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fpostgresql-consul-demo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fpostgresql-consul-demo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cedrickchee","download_url":"https://codeload.github.com/cedrickchee/postgresql-consul-demo/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cedrickchee%2Fpostgresql-consul-demo/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":264713754,"owners_count":23652892,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["consul","docker-compose","educational-project","patroni","postgresql-ha-cluster","postgresql-replication"],"created_at":"2024-11-17T23:09:51.088Z","updated_at":"2026-05-19T03:32:42.786Z","avatar_url":"https://github.com/cedrickchee.png","language":"Python","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# Postgresql HA cluster in docker using Patroni and Consul.\n\nThis project accompanies a blog located here: https://digitalis.io/blog/postgresql/using-consul-for-postgresql-ha-with-no-load-balancers-needed/\n\nThis is a minimalistic demo application showing HA Postgresql cluster managed by Patroni and Consul with no extra load balancers.\nThe entire setup runs in `docker-compose` and requires `make` utility for helper commands.\n\nFor Postgresql and application containers the vanilla `ubuntu:20.04` image is used and for Consul servers the official `consul:1.10.3` image.\n\nPatroni nodes run 2 services under supervisor: patroni and consul in agent mode.\n\nConsul nodes run consul service in server mode.\n\nApplication nodes run 3 services under supervisor: consul in agent mode, dnsmasq and application itself.\n\nApplication connects to a current Postgresql master node and reports to the log its IP address.\n\n\n```\n                 ┌───dns:8600──┐                    ┌─────────────┐                            \n                 │             │                    │             │                            \n                 │             │                    │             │                            \n       ┌──────────────────┐    │          ┌──────────────────┐    │                            \n       │                  │    │          │                  │    │                            \n       │  application01   ├─┐  │  ┌───────│  application02   ├─┐  │                            \n       │                  │ │  │  │       │                  │ │  │                            \n       └───┬──────────────┘ │◀─┘  │       └───┬──────────────┘ │◀─┘                            \n       │   │    consul-agent│     │           │    consul-agent│                               \n       │   └────────────────┘     │           └────────────────┘                               \n       │            │             │                    │                                       \n       │            │             │       ┌────────────┘                                       \n       │            │             │       │                                                    \n       │            │             │       ▼                                                    \n       │            │             │.─────────────.                                             \n       │            │           ,─'               '─.                                          \n postgres:5432      └─────────▶(      consul01       )                                         \n       │                        `─┬.             _.─'──.                                       \n       │                          │ `───────────'       '─.                                    \n       │                          │  (      consul02       )◀───────────rpc:8300──────────────┐\n       │                          │   `──.             _.─'──.                                │\n       │                          │       `───────────'       '─.                             │\n       │                          │ ┌─────▶(      consul03       )◀─┐                         │\n       │                          │ │       `──.             _.─'   │                         │\n       │                          │ │           `───────────'       │                         │\n       │                          │ │                               │                         │\n       ▼                          │ │                               │                         │\n       ╔═══════════════════╗      │ │    ╔═══════════════════╗      │   ╔═══════════════════╗ │\n       ║ PG                ║      │ │    ║ PG                ║      │   ║ PG                ║ │\n       ║                   ║      │ │    ║                   ║      │   ║                   ║ │\n    ┌──║     patroni01     ║◀─────┘ │ ┌──║     patroni02     ║      │┌──║     patroni03     ║ │\n    │  ║                   ╠─┐      │ │  ║                   ╠─┐    ││  ║                   ╠─┐\n    │  ║         [leader]  ║ │      │ │  ║         [follower]║ │    ││  ║         [follower]║ │\n    │  ╚════╦══════════════╝ │──────┘ │  ╚════╦══════════════╝ │────┘│  ╚════╦══════════════╝ │\n    │       │    consul-agent│        │       │    consul-agent│     │       │    consul-agent│\n    │       └────────────────┘        │       └────────────────┘     │       └────────────────┘\n    │                ▲                │                ▲             │                ▲        \n    │   http:8500    │                │                │             │                │        \n    └────────────────┘                └────────────────┘             └────────────────┘        \n```\n\n## Usage\n\n### `make build`\n\n```sh\nPull and build container images. This make take some time when running for first time.\nconsul01 uses an image, skipping\nconsul02 uses an image, skipping\nconsul03 uses an image, skipping\nBuilding patroni01\nSending build context to Docker daemon  12.29kB\nStep 1/14 : FROM ubuntu:20.04\n20.04: Pulling from library/ubuntu\n7b1a6ab2e44d: Pull complete \nDigest: sha256:626ffe58f6e7566e00254b638eb7e0f3b11d4da9675088f4781a50ae288f3322\nStatus: Downloaded newer image for ubuntu:20.04\n ---\u003e ba6acccedd29\nStep 2/14 : ARG DEBIAN_FRONTEND=noninteractive\n ---\u003e Running in 0200e7404741\nRemoving intermediate container 0200e7404741\n ---\u003e caa769cf7322\nStep 3/14 : RUN apt update\n ---\u003e Running in 7233f5befb28\n\nWARNING: apt does not have a stable CLI interface. Use with caution in scripts.\n\nGet:1 http://security.ubuntu.com/ubuntu focal-security InRelease [114 kB]\n...\nRemoving intermediate container 4310e7ecc28f\n ---\u003e ee0f2dec7130\nStep 5/14 : RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -\n ---\u003e Running in 230b7dea74c7\nWarning: apt-key output should not be parsed (stdout is not a terminal)\nOK\nRemoving intermediate container 230b7dea74c7\n ---\u003e 990a41aafb51\nStep 6/14 : RUN echo \"deb http://apt.postgresql.org/pub/repos/apt/ `lsb_release -cs`-pgdg main\" \u003e  /etc/apt/sources.list.d/pgdg.list\n ---\u003e Running in 3248bd7cad00\nRemoving intermediate container 3248bd7cad00\n ---\u003e 55526e9d7da3\nStep 7/14 : RUN apt update \u0026\u0026 apt -y install postgresql-13 postgresql-client-13 patroni python3-consul\n ---\u003e Running in 12fed402b068\n...\nSuccess. You can now start the database server using:\n\n    pg_ctlcluster 13 main start\n...\nProcessing triggers for dbus (1.12.16-2ubuntu2.1) ...\nRemoving intermediate container 6fdf6e8ceebd\n ---\u003e 23bf8193a16b\nStep 5/10 : RUN pip3 install psycopg2\n ---\u003e Running in 88b951792136\nCollecting psycopg2\n  Downloading psycopg2-2.9.1.tar.gz (379 kB)\nBuilding wheels for collected packages: psycopg2\n  Building wheel for psycopg2 (setup.py): started\n...\nSuccessfully built psycopg2\nInstalling collected packages: psycopg2\nSuccessfully installed psycopg2-2.9.1\nRemoving intermediate container 88b951792136\n ---\u003e 72358ff035c6\nStep 6/10 : RUN wget --quiet https://releases.hashicorp.com/consul/1.10.3/consul_1.10.3_linux_amd64.zip\n ---\u003e Running in c89270de26f4\nRemoving intermediate container c89270de26f4\n ---\u003e 71d535a8fcf4\nStep 7/10 : RUN unzip consul_1.10.3_linux_amd64.zip \u0026\u0026 mv consul /usr/local/bin/ \u0026\u0026 rm consul_1.10.3_linux_amd64.zip\n ---\u003e Running in a7e3f1ecdf96\nArchive:  consul_1.10.3_linux_amd64.zip\n  inflating: consul                  \nRemoving intermediate container a7e3f1ecdf96\n ---\u003e f8661a59d1fc\nStep 8/10 : RUN mkdir -p /etc/consul/certs \u0026\u0026 mkdir -p /consul/data\n ---\u003e Running in c0d4b8434c6e\nRemoving intermediate container c0d4b8434c6e\n ---\u003e 8be64b493b83\nStep 9/10 : COPY ./entrypoint.sh /usr/bin/entrypoint.sh\n ---\u003e e9dbb8bbc570\nStep 10/10 : CMD [\"/usr/bin/entrypoint.sh\"]\n ---\u003e Running in f0dc27b15cd0\nRemoving intermediate container f0dc27b15cd0\n ---\u003e d4d3a9c9b3b1\nSuccessfully built d4d3a9c9b3b1\nSuccessfully tagged postgresql-consul-demo_application01:latest\nBuilding application02\nSending build context to Docker daemon  10.24kB\nStep 1/10 : FROM ubuntu:20.04\n ---\u003e ba6acccedd29\nStep 2/10 : ARG DEBIAN_FRONTEND=noninteractive\n ---\u003e Using cache\n ---\u003e caa769cf7322\nStep 3/10 : RUN apt update\n ---\u003e Using cache\n ---\u003e 647290579e00\nStep 4/10 : RUN apt -y install vim python3-pip libpq-dev supervisor wget unzip gettext-base dnsmasq iputils-ping dnsutils\n ---\u003e Using cache\n ---\u003e 23bf8193a16b\nStep 5/10 : RUN pip3 install psycopg2\n ---\u003e Using cache\n ---\u003e 72358ff035c6\nStep 6/10 : RUN wget --quiet https://releases.hashicorp.com/consul/1.10.3/consul_1.10.3_linux_amd64.zip\n ---\u003e Using cache\n ---\u003e 71d535a8fcf4\nStep 7/10 : RUN unzip consul_1.10.3_linux_amd64.zip \u0026\u0026 mv consul /usr/local/bin/ \u0026\u0026 rm consul_1.10.3_linux_amd64.zip\n ---\u003e Using cache\n ---\u003e f8661a59d1fc\nStep 8/10 : RUN mkdir -p /etc/consul/certs \u0026\u0026 mkdir -p /consul/data\n ---\u003e Using cache\n ---\u003e 8be64b493b83\nStep 9/10 : COPY ./entrypoint.sh /usr/bin/entrypoint.sh\n ---\u003e Using cache\n ---\u003e e9dbb8bbc570\nStep 10/10 : CMD [\"/usr/bin/entrypoint.sh\"]\n ---\u003e Using cache\n ---\u003e d4d3a9c9b3b1\nSuccessfully built d4d3a9c9b3b1\nSuccessfully tagged postgresql-consul-demo_application02:latest\n```\n\n```sh\n$ docker image ls\nREPOSITORY                                  TAG            IMAGE ID       CREATED         SIZE\npostgresql-consul-demo_application02        latest         d4d3a9c9b3b1   2 minutes ago   646MB\npostgresql-consul-demo_application01        latest         d4d3a9c9b3b1   2 minutes ago   646MB\npostgresql-consul-demo_patroni01            latest         aa547a52f3f4   3 minutes ago   900MB\npostgresql-consul-demo_patroni02            latest         aa547a52f3f4   3 minutes ago   900MB\npostgresql-consul-demo_patroni03            latest         aa547a52f3f4   3 minutes ago   900MB\nubuntu                                      20.04          ba6acccedd29   5 days ago      72.8MB\n...\n```\n\n### `make start`\n\n```sh\nStart cluster\nCreating network \"postgresql-consul-demo_pglab\" with driver \"bridge\"\nPulling consul01 (consul:1.10.3)...\n1.10.3: Pulling from library/consul\n4e9f2cdf4387: Already exists\n...\nb694e38e70f2: Pull complete\nDigest: sha256:7b0effaf3faf9ce134be8648e3cf5f82263a52cd80f5395452a9ca69700f0357\nStatus: Downloaded newer image for consul:1.10.3\nCreating patroni03     ... done\nCreating application01 ... done\nCreating patroni01     ... done\nCreating patroni02     ... done\nCreating consul01      ... done\nCreating application02 ... done\nCreating consul03      ... done\nCreating consul02      ... done\n```\n\n\n### `make status`\n\n```sh\n============================== Docker status ==============================\n    Name                   Command               State                                                         Ports\n--------------------------------------------------------------------------------------------------------------------------------------------------------------------------\napplication01   /usr/bin/entrypoint.sh           Up\napplication02   /usr/bin/entrypoint.sh           Up\nconsul01        docker-entrypoint.sh agent ...   Up      8300/tcp, 8301/tcp, 8301/udp, 8302/tcp, 8302/udp, 0.0.0.0:8500-\u003e8500/tcp, 0.0.0.0:8600-\u003e8600/tcp,\n                                                         0.0.0.0:8600-\u003e8600/udp\nconsul02        docker-entrypoint.sh agent ...   Up      8300/tcp, 8301/tcp, 8301/udp, 8302/tcp, 8302/udp, 8500/tcp, 8600/tcp, 8600/udp\nconsul03        docker-entrypoint.sh agent ...   Up      8300/tcp, 8301/tcp, 8301/udp, 8302/tcp, 8302/udp, 8500/tcp, 8600/tcp, 8600/udp\npatroni01       /usr/bin/entrypoint.sh           Up\npatroni02       /usr/bin/entrypoint.sh           Up\npatroni03       /usr/bin/entrypoint.sh           Up\n\n============================== Consul status ==============================\nNode           Address          Status  Type    Build   Protocol  DC   Segment\nconsul01       172.19.0.7:8301  alive   server  1.10.3  2         dc1  \u003call\u003e\nconsul02       172.19.0.8:8301  alive   server  1.10.3  2         dc1  \u003call\u003e\nconsul03       172.19.0.6:8301  alive   server  1.10.3  2         dc1  \u003call\u003e\napplication01  172.19.0.3:8301  alive   client  1.10.3  2         dc1  \u003cdefault\u003e\napplication02  172.19.0.9:8301  alive   client  1.10.3  2         dc1  \u003cdefault\u003e\npatroni01      172.19.0.4:8301  alive   client  1.10.3  2         dc1  \u003cdefault\u003e\npatroni02      172.19.0.5:8301  alive   client  1.10.3  2         dc1  \u003cdefault\u003e\npatroni03      172.19.0.2:8301  alive   client  1.10.3  2         dc1  \u003cdefault\u003e\n\n============================== Patroni status =============================\n+ Cluster: pglab (7021393365364887600) -----+----+-----------+\n| Member    | Host      | Role    | State   | TL | Lag in MB |\n+-----------+-----------+---------+---------+----+-----------+\n| patroni01 | patroni01 | Replica | running |  1 |         0 |\n| patroni02 | patroni02 | Replica | running |  1 |         0 |\n| patroni03 | patroni03 | Leader  | running |  1 |           |\n+-----------+-----------+---------+---------+----+-----------+\n```\n\n\u003e Note: if patroni nodes are experiencing error: \"patroni private key file must\n\u003e be owned by the database user or root\", ensure the permissions are correct for\n\u003e the PostgreSQL server certificate and key file. The files should be owned by\n\u003e the database user (default: `postgres`), and its permissions should be `0400`.\n\u003e\n\u003e To fix, run these commands on the container host:\n\u003e ```sh\n\u003e $ chown -R 102:102 patroni/certs\n\u003e $ chmod -R 0600 patroni/certs\n\u003e ```\n\n\u003e Note: if application nodes are experiencing error: \"WARNING: password file\n\u003e \"/root/.pgpass\" has group or world access; permissions should be u=rw (0600)\n\u003e or less ... ERROR fe_sendauth: no password supplied\".\n\u003e\n\u003e Fix it by running these commands on the container host:\n\u003e ```sh\n\u003e $ chmod 0600 application/pgpass\n\u003e ```\n\n### `make stop`\n\n```\nStop cluster\nStopping application01 ... done\nStopping application02 ... done\nStopping patroni01     ... done\nStopping patroni03     ... done\nStopping patroni02     ... done\nStopping consul01      ... done\nStopping consul03      ... done\nStopping consul02      ... done\n```\n\n### `make destroy`\n\n```\nForce destroy cluster\nStopping application01 ... done\nStopping application02 ... done\nStopping patroni01     ... done\nStopping patroni03     ... done\nStopping patroni02     ... done\nStopping consul03      ... done\nStopping consul02      ... done\nStopping consul01      ... done\nRemoving application01 ... done\nRemoving application02 ... done\nRemoving patroni01     ... done\nRemoving patroni03     ... done\nRemoving patroni02     ... done\nRemoving consul03      ... done\nRemoving consul02      ... done\nRemoving consul01      ... done\nRemoving network postgresql-consul-demo_pglab\n```\n\n### `make recreate`\n\n```\nDestroy and recreate cluster\nStopping application02 ... done\nStopping application01 ... done\nStopping patroni01     ... done\nStopping patroni03     ... done\nStopping consul03      ... done\nStopping patroni02     ... done\nStopping consul02      ... done\nStopping consul01      ... done\nGoing to remove application02, application01, patroni01, patroni03, consul03, patroni02, consul02, consul01\nRemoving application02 ... done\nRemoving application01 ... done\nRemoving patroni01     ... done\nRemoving patroni03     ... done\nRemoving consul03      ... done\nRemoving patroni02     ... done\nRemoving consul02      ... done\nRemoving consul01      ... done\nCreating consul02      ... done\nCreating consul01      ... done\nCreating patroni02     ... done\nCreating application01 ... done\nCreating consul03      ... done\nCreating application02 ... done\nCreating patroni03     ... done\nCreating patroni01     ... done\n```\n\n### `make switchover`\n\n```\nForce patroni switchover to another node\nCurrent cluster topology\n+ Cluster: pglab (7021401786240585771) -----+----+-----------+\n| Member    | Host      | Role    | State   | TL | Lag in MB |\n+-----------+-----------+---------+---------+----+-----------+\n| patroni01 | patroni01 | Replica | running |  1 |         0 |\n| patroni02 | patroni02 | Replica | running |  1 |         0 |\n| patroni03 | patroni03 | Leader  | running |  1 |           |\n+-----------+-----------+---------+---------+----+-----------+\n2021-10-21 06:32:54.72729 Successfully switched over to \"patroni02\"\n+ Cluster: pglab (7021401786240585771) -----+----+-----------+\n| Member    | Host      | Role    | State   | TL | Lag in MB |\n+-----------+-----------+---------+---------+----+-----------+\n| patroni01 | patroni01 | Replica | running |  1 |         0 |\n| patroni02 | patroni02 | Leader  | running |  1 |           |\n| patroni03 | patroni03 | Replica | stopped |    |   unknown |\n+-----------+-----------+---------+---------+----+-----------+\n```\n\n### `make applogs`\n\n```\nAttaching to application02, application01\napplication01    | 2021-04-19 16:42:32,564 CRIT Supervisor is running as root.  Privileges were not dropped because no user is specified in the config file.  If you intend to run as root, you can set user=root in the config file to avoid this message.\napplication01    | 2021-04-19 16:42:32,564 INFO Included extra file \"/etc/supervisor/conf.d/application.conf\" during parsing\napplication01    | 2021-04-19 16:42:32,564 INFO Included extra file \"/etc/supervisor/conf.d/consul.conf\" during parsing\napplication01    | 2021-04-19 16:42:32,572 INFO RPC interface 'supervisor' initialized\napplication01    | 2021-04-19 16:42:32,573 CRIT Server 'unix_http_server' running without any HTTP authentication checking\napplication01    | 2021-04-19 16:42:32,573 INFO supervisord started with pid 8\napplication01    | 2021-04-19 16:42:33,578 INFO spawned: 'application' with pid 10\napplication01    | 2021-04-19 16:42:33,583 INFO spawned: 'consul' with pid 11\napplication01    | 2021-04-19 16:42:33,587 INFO spawned: 'dnsmasq' with pid 12\napplication01    | 2021-04-19 16:42:34,114 INFO Starting.\napplication01    | 2021-04-19 16:42:34,131 INFO Connecting.\napplication01    | 2021-04-19 16:42:34,333 ERROR could not translate host name \"master.pglab.service.consul\" to address: Name or service not known\napplication01    |\napplication01    | 2021-04-19 16:42:35,336 INFO success: application entered RUNNING state, process has stayed up for \u003e than 1 seconds (startsecs)\napplication01    | 2021-04-19 16:42:35,336 INFO success: consul entered RUNNING state, process has stayed up for \u003e than 1 seconds (startsecs)\napplication01    | 2021-04-19 16:42:35,336 INFO success: dnsmasq entered RUNNING state, process has stayed up for \u003e than 1 seconds (startsecs)\napplication01    | 2021-04-19 16:42:36,337 INFO Connecting.\napplication01    | 2021-04-19 16:42:38,768 ERROR could not translate host name \"master.pglab.service.consul\" to address: Name or service not known\napplication01    |\napplication01    | 2021-04-19 16:42:40,771 INFO Connecting.\napplication01    | 2021-04-19 16:42:40,775 ERROR could not translate host name \"master.pglab.service.consul\" to address: Name or service not known\napplication01    |\napplication01    | 2021-04-19 16:42:42,778 INFO Connecting.\napplication01    | 2021-04-19 16:42:42,801 INFO I'm connected to 172.24.0.5\napplication01    | 2021-04-19 16:42:43,804 INFO I'm connected to 172.24.0.5\napplication01    | 2021-04-19 16:42:44,805 INFO I'm connected to 172.24.0.5\n...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedrickchee%2Fpostgresql-consul-demo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcedrickchee%2Fpostgresql-consul-demo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcedrickchee%2Fpostgresql-consul-demo/lists"}