{"id":21401102,"url":"https://github.com/massdriver-cloud/webinar-postgresql-logical-replication","last_synced_at":"2026-01-03T00:40:39.123Z","repository":{"id":186780584,"uuid":"675750621","full_name":"massdriver-cloud/webinar-postgresql-logical-replication","owner":"massdriver-cloud","description":"A webinar on how to upgrade or migrate PostgreSQL with minimal downtime using logical replication.","archived":false,"fork":false,"pushed_at":"2023-08-09T15:56:14.000Z","size":30,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":4,"default_branch":"main","last_synced_at":"2025-01-23T03:13:53.131Z","etag":null,"topics":["cloud","database-management","postgres"],"latest_commit_sha":null,"homepage":"https://massdriver.cloud","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/massdriver-cloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2023-08-07T16:26:20.000Z","updated_at":"2024-04-22T12:50:26.000Z","dependencies_parsed_at":null,"dependency_job_id":"a682f9e8-c5ed-4e21-8f69-61e0ed284afe","html_url":"https://github.com/massdriver-cloud/webinar-postgresql-logical-replication","commit_stats":{"total_commits":17,"total_committers":1,"mean_commits":17.0,"dds":0.0,"last_synced_commit":"2fc2a33373fb34ac3f30c0a32340bb181640722a"},"previous_names":["massdriver-cloud/webinar-postgresql-logical-replication"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/massdriver-cloud%2Fwebinar-postgresql-logical-replication","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/massdriver-cloud%2Fwebinar-postgresql-logical-replication/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/massdriver-cloud%2Fwebinar-postgresql-logical-replication/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/massdriver-cloud%2Fwebinar-postgresql-logical-replication/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/massdriver-cloud","download_url":"https://codeload.github.com/massdriver-cloud/webinar-postgresql-logical-replication/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243893893,"owners_count":20364919,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cloud","database-management","postgres"],"created_at":"2024-11-22T15:26:08.338Z","updated_at":"2026-01-03T00:40:39.084Z","avatar_url":"https://github.com/massdriver-cloud.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Upgrading or Migrating PostgreSQL with Minimal Downtime\n\n## Setup\n\nBefore we get started migrating, we'll use [docker compose (or the container tool of your choosing)](https://docs.docker.com/desktop/) to get an example e-commerce application (Spree) up and running to migrate.\n\nRun the following command:\n\n```shell\ndocker compose up -d\n```\n\nIf you want to see the logs of any of the services you can run `docker compose logs -f`.\n\nNext we'll seed the database.\n\n```shell\ndocker compose run --entrypoint bash spree -c \"bundle exec rails db:create db:migrate \u0026\u0026 bundle exec rake db:seed spree_sample:load\"\n```\n\nIf prompted enter a username and password:\n\nUsername: `test@example.com`\n\nPassword: `password!`\n\nThis may take a few minutes. It will create a PostgreSQL 12 instance, a Spree E-Commerce instance, populate the PG 12 database with products, and an idle PostgreSQL 15 instance.\n\n**Note**: This webinar/tutorial connects to the source and destination multiple times and includes the docker command each time for doing so. I recommend opening two additional shells and keeping PG 12 and 15 side by side to simplify following along.\n\nThe Spree application may restart a few times while the data is getting populated.\n\nOnce the application is up visit [http://localhost:4000/admin](http://localhost:4000/admin) and log in with:\n\nUsername: `test@example.com`\n\nPassword: `password!`\n\nThe data in this storefront is what we will be migrating to PostgreSQL 15.\n\n## Upgrading / Migrating Postgres with Logical Replication\n\nPostgreSQL logical replication is a data synchronization method that enables the replication of individual database changes (such as inserts, updates, and deletes) in a fine-grained manner between two PostgreSQL databases. It uses a publish-subscribe model, allowing changes from a source database's specified tables to be captured, transformed into a logical representation, and then applied to target databases.\n\nWith logical replication there are two options for getting your existing data into the destination database:\n\n* Using `copy_data = true` with the replication subscription\n* Using `copy_data = false` and performing pg_dump/pg_restore\n\n\n**Using `copy_data = true` with the replication subscription:**\n\nPros:\n* Ease of Use: Setting up a subscription with copy_data set to true is straightforward. PostgreSQL handles data replication automatically.\n* Minimal Lag: Since data changes are replicated as they occur, the data lag between the source and target databases is minimized.\n\nCons:\n\n* Network Overhead: Continuous replication generates network traffic, which might lead to increased network utilization and potential latency when replicating large datasets.\n* Resource Consumption: Real-time replication can consume significant server resources on both databases, affecting their performance.\n\n**Using `copy_data = false` and performing pg_dump/pg_restore:**\n\nPros:\n* Controlled Replication: You have control over when the replication process occurs. This can help you manage the impact on server resources.\n* Reduced Network Traffic: As replication is not continuous, there's less ongoing network traffic compared to real-time replication.\n\nCons:\n* Complexity: Manual dump and restore processes involve more steps and potential for errors, especially with large datasets.\n* Data Lag: Data replication is not immediate, resulting in data lag between the source and target databases.\n\n\nTo perform logical replication we'll need to make sure the WAL level (`wal_level`) is set to `logical`.\n\nThe Write-Ahead Log (WAL) in PostgreSQL is a transaction log that records changes to the database in a sequential manner. It serves as a reliable mechanism to ensure data durability, high availability, and crash recovery by allowing the replay of logged changes to reconstruct the database to a consistent state in the event of system failures.\n\nLet's set the WAL level and restart the postgres instances:\n\n```shell\ndocker cp postgresql.conf postgres12:/var/lib/postgresql/data/postgresql.conf\ndocker compose restart postgres12\n\ndocker cp postgresql.conf postgres15:/var/lib/postgresql/data/postgresql.conf\ndocker compose restart postgres15\n```\n\n**Note:** This webinar keeps the config pretty simple. You may want also want to create a custom role for replication.\n\n### Logical replication with `copy_data = true`\n\nThis is a much simpler approach and is recommended for small datasets, migrations on the same subnet, or when resource consumption is not a concern.\n\nWe will:\n1. Dump and restore the schema to the destination database\n2. Create a replication publication\n3. Create a replication subscription\n\nLogical replication only works on DML (data manipulation language). It will not replication DDL (schema changes), so we must get the schema onto the destination database first.\n\n```shell\n# Connect to PG12\ndocker compose exec postgres12 bash\n\n# Dump the 'store' database to tar format (-F t)\npg_dump -U pg12_user --schema-only -F t store \u003e store-dump.tar\nexit\n```\n\nCopy the tar from PG 12 to PG 15:\n```shell\n# Copy dump to local filesystem\ndocker cp postgres12:/store-dump.tar .\n\n# Copy dump to PG15\ndocker cp ./store-dump.tar postgres15:/store-dump.tar\n```\n\n```shell\n# Connect to PG15\ndocker compose exec postgres15 bash\n\n# Restore the Databaseshow rep\npg_restore -d pg15_db /store-dump.tar --no-owner --role=pg15_user -C -U pg15_user\nexit\n```\n\n**Note:** `pg_dump` and `pg_restore` have a lot of options. We advise reading up on the user manual when preparing your dump/restore strategy.\n\nLet's break down the `pg_dump` and `pg_restore` commands.\n\n`pg_dump`:\n\n* `-U` user\n* `-F` file format: tar\n* `--schema-only` only copy schema\n* `\"store\"` final argument is the database schema to dumb\n\n`pg_restore`:\n\n* `-d` database - here we picked \"pg15_db\" as we knew that database existed. You must pick an existing database. The dump has the name of the schema we actually plan to restore.\n* `-C` create schema - this will cause `pg_restore` to create the schema referenced in the dump before restoring\n* `-U` user\n* `--no-owner` - remove the `pg12_user` as the owner of the database objects\n* `--role` - set the new owner of the objects created\n\n#### Setup Replication\n\nIn the publish-subscribe model of logical replication in PostgreSQL, a database creates and publishes a stream of changes as a publication. Other databases, acting as subscribers, can subscribe to these publications and receive a copy of the changes to keep their data synchronized, providing a flexible and selective means of replicating specific data changes between databases.\n\n##### Create the publication on PG 12\n\n```shell\ndocker compose exec postgres12 bash\npsql -U pg12_user -d store\n```\n\n```sql\nCREATE PUBLICATION pub_pg1215_migration FOR ALL TABLES;\n```\n\n##### Create the subscription on PG 15\n\n```shell\ndocker compose exec postgres15 bash\npsql -U pg15_user -d store\n```\n\n```sql\nCREATE SUBSCRIPTION sub_pg1215_migration \n  CONNECTION 'dbname=store host=postgres12 user=pg12_user password=pg12_password' \n  PUBLICATION pub_pg1215_migration;\n```\n#### Confirm Data is Replicating\n\n```shell\ndocker compose exec postgres12 bash\npsql -U pg12_user -d store\n```\n\n```sql\nselect count(*) from spree_products;\n-- ~\u003e 116 rows\n```\n\n```shell\ndocker compose exec postgres15 bash\npsql -U pg15_user -d store\n```\n\n```sql\nselect count(*) from spree_products;\n-- ~\u003e # of rows from above\n```\n\nSo that feels good. Add some products in the UI and rerun the PG15 query above to see the records getting replicated.\n\nNow let's see what Postgres thinks.\n\nConnect to PG 12 source database:\n\n```shell\ndocker compose exec postgres12 bash\n```\n\nCheck the current WAL LSN:\n\n```sql\nselect pg_current_wal_lsn();\n```\n\nCheck the values of the log sequence numbers (LSN)\n```\npg_current_wal_lsn\n--------------------\n0/243FA00\n```\n\nAdditionally you can look at the replication status for the specific subscription:\n\n```sql\nselect * from  pg_stat_replication;\n```\n\n```\n-[ RECORD 1 ]----+------------------------------\npid              | 59\nusesysid         | 10\nusename          | pg12_user\napplication_name | sub_pg1215_migration\nclient_addr      | 172.25.0.4\nclient_hostname  |\nclient_port      | 45238\nbackend_start    | 2023-08-08 16:13:09.264021+00\nbackend_xmin     |\nstate            | streaming\nsent_lsn         | 0/243FA00 \u003c-- These are what you are looking for\nwrite_lsn        | 0/243FA00\nflush_lsn        | 0/243FA00\nreplay_lsn       | 0/243FA00\nwrite_lag        |\nflush_lag        |\nreplay_lag       |\nsync_priority    | 0\nsync_state       | async\nreply_time       | 2023-08-08 16:39:05.514689+00\n```\n\nLet's check the LSN received on the PG 15 instance.\n\n```shell\ndocker compose exec postgres15 bash\n```\n\n```sql\nselect * from  pg_stat_subscription;\n```\n\n```\n-[ RECORD 1 ]---------+------------------------------\nsubid                 | 17694\nsubname               | sub_pg1215_migration\npid                   | 51\nrelid                 |\nreceived_lsn          | 0/243FA00 \u003c-- Last LSN received\nlast_msg_send_time    | 2023-08-08 16:47:17.351882+00\nlast_msg_receipt_time | 2023-08-08 16:47:17.352277+00\nlatest_end_lsn        | 0/243FA00 \u003c-- Last LSN reported back to publisher\nlatest_end_time       | 2023-08-08 16:47:17.351882+00\n```\n\nLooks like they are in sync! The difference between `last_msg_send_time` and `last_msg_receipt_time` can give you an idea of lag given both instances time are in sync.\n\nIf you are migrating using `copy_data = true` its time to [promote](#database-promotion) your database.\n\n### Logical replication with `copy_data = false` + pg_dump/pg_restore\n\n**Note:** If you performed the previous logical replication step, you'll need to reset PG 15 and remove the publisher on PG 12. Click [here](#resetting-the-databases) for instructions.\n\nThis is the recommended approach for large data sets especially migrating or upgrading different networks. It can be time and resource consuming to replicate 100GB between something like Heroku and AWS RDS. This requires additional steps and downtime to prepare.\n\nThis process is about 8 steps to get the databases in sync:\n\n1. Stop the Spree API, this will be the beginning of the first **downtime** \n2. Take a dump of PG 12\n3. Create publication \u0026 replication slot on PG 12\n4. Restart the app\n5. Restore snapshot to pg 15\n6. Create paused subscription on PG 15\n7. Optionally add records to PG 12, check diff between dbs\n8. Enable the subscription to start receiving updates\n\nTo get started we'll stop the Spree API. This will be the first and probably longest of two downtimes. We stop the app because we don't want records written between the time the database dump is taken and the replication slot is created.\n\n```shell\ndocker compose stop spree\n```\n\nConnect to PG12 and run pg_dump:\n\n```shell\ndocker compose exec postgres12 bash\npg_dump -U pg12_user -F t store \u003e store-dump.tar\n\npsql -U pg12_user -d store\n```\n\nCreate a publication on PG 12:\n\n```sql\nCREATE PUBLICATION pub_pg1215_migration FOR ALL TABLES;\nSELECT pg_create_logical_replication_slot('sub_pg1215_migration', 'pgoutput');\n```\n\nRestart the Spree API:\n\n```shell\ndocker compose start spree\n```\n\nCopy dump to PG 15 container:\n\n```shell\n# Copy schema dump to local filesystem\ndocker cp postgres12:/store-dump.tar .\n\n# Copy schema dump to PG15\ndocker cp ./store-dump.tar postgres15:/store-dump.tar\n```\n\nConnect to PG 15 and restore:\n\n```shell\ndocker compose exec postgres15 bash\npg_restore -d pg15_db /store-dump.tar --no-owner --role=pg15_user -C -U pg15_user;\n\npsql -U pg15_user -d store;\n```\n\nCreate a subscription on PG 15:\n\n```sql\nCREATE SUBSCRIPTION sub_pg1215_migration \n  CONNECTION 'dbname=store host=postgres12 user=pg12_user password=pg12_password' \n  PUBLICATION pub_pg1215_migration WITH (copy_data = false, create_slot=false, enabled=false, slot_name=sub_pg1215_migration);\n```\n\nAt this point the snapshot is loaded on PG 15, but replication has not started. \n\n**Add and edit some records in the Spree admin dashboard to simulate users and we'll see replication in a moment.** In the meantime, you can see the record lag by running `SELECT COUNT(*) from spree_products;` in both databases as you add records in the UI.\n\nOn the PG 15 instance enable the subscription:\n\n```sql\nALTER SUBSCRIPTION sub_pg1215_migration ENABLE;\n```\n\nYou should now be able to run queries on PG 12 and PG 15 and get the same results, for example: `SELECT COUNT(*) from spree_products;`\n\nNow let's see what Postgres thinks.\n\nConnect to PG 12 source database:\n\n```shell\ndocker compose exec postgres12 bash\n```\n\nCheck the current WAL LSN:\n\n```sql\nselect pg_current_wal_lsn();\n```\n\nCheck the values of the log sequence numbers (LSN)\n```\npg_current_wal_lsn\n--------------------\n0/25A97D0\n```\n\nLet's check the LSN received on the PG 15 instance.\n\n```shell\ndocker compose exec postgres15 bash\n```\n\n```sql\nselect * from  pg_stat_subscription;\n```\n\n```\n-[ RECORD 1 ]---------+------------------------------\nsubid                 | 26874\nsubname               | sub_pg1215_migration\npid                   | 1781\nrelid                 |\nreceived_lsn          | 0/25A97D0 \u003c-- Last LSN received\nlast_msg_send_time    | 2023-08-09 04:10:25.602431+00\nlast_msg_receipt_time | 2023-08-09 04:10:25.602842+00\nlatest_end_lsn        | 0/25A97D0 \u003c-- Last LSN reported back to publisher\nlatest_end_time       | 2023-08-09 04:10:25.602431+00\n```\n\nLooks like they're in sync! Its time to \"promote\" to PG 15.\n\n## Database Promotion\n\nThe promotion process is pretty straightforward:\n\n1. Put any applications that write to your source database in maintenance mode. This will be your second, potentially brief, **downtime**.\n2. Check the replication and subscription status and drop the subscription\n3. Copy sequence data from the source to destination database\n4. Change the applications database connection info to the new database\n5. Bring your application back up\n\nFirst, let's put the application in maintenance mode (**DOWNTIME BEGINS**)\n\n```shell\ndocker compose stop spree\n```\n\nCheck the current WAL LSN on **PG 12**:\n\n```sql\nselect pg_current_wal_lsn();\n-- -[ RECORD 1 ]--+----------\n-- pg_current_wal_lsn | 0/25A97D0\n```\n\nCheck that the last LSN acknowledged matches on **PG 15**:\n\n```sql\nselect latest_end_lsn from pg_stat_subscription where subname = 'sub_pg1215_migration';\n-- -[ RECORD 1 ]--+----------\n-- latest_end_lsn | 0/25A97D0\n```\n\n**Note:** Another great tool for monitoring replication is [PG Metrics](https://pgmetrics.io/docs/index.html).\n\nOnce the two values match you can drop the SUBSCRIPTION on **PG 15**:\n\n```sql\nDROP SUBSCRIPTION sub_pg1215_migration;\n```\n\n**Important!** Now you'll need to sync the sequence data between PG 12 and PG 15. Sequences in PostgreSQL are database objects that provide a way to generate unique numeric values, often used for auto-incrementing primary keys in tables. Sequences **are not** replicated with logical replication. Skipping this step will result potentially conflicting primary key values for auto incrementing keys.\n\n\nRun the following command on both databases to see the differnces in sequence data:\n\n```sql\nSELECT sequencename, last_value\nFROM pg_sequences\nORDER BY last_value DESC NULLS LAST, sequencename;\n```\n\nRun the following command from th PG 15's shell or an instance that can access both databases:\n\n```shell\npsql -h postgres12 -U pg12_user -XAtqc 'SELECT $$select setval($$ || quote_literal(sequencename) || $$, $$ || last_value || $$); $$ AS sql FROM pg_sequences' store \u003e sequences.sql\n\ncat sequences.sql | psql -h postgres15 -U pg15_user store\n```\n\nIf your application has materialized views, those will need to be manually refreshed as they are not automatically refreshed\n\nIts time to point the application at the upgraded database.\n\nFind the following line in `docker-compose.yml` and comment it out:\n\n`DATABASE_URL: postgres://pg12_user:pg12_password@postgres12:5432/store`\n\nUncomment\n\n`DATABASE_URL: postgres://pg15_user:pg15_password@postgres15:5432/store`\n\n```shell\ndocker compose up spree -d\ndocker compose stop postgres12\n```\n\nYou should be on Postgres 15!\n\nTear it all down:\n\n```shell\ndocker compose down --remove-orphans --volumes\n```\n\n## Appendix\n\n### Resetting the Databases\n\nThe following steps can be used to remove publications, subscriptions, and the PG 15 data created in the steps above.\n\n**Reset PG 15:**\n\n```shell\ndocker compose exec postgres15 bash\npsql -U pg15_user -d store\n```\n```sql\nDROP SUBSCRIPTION sub_pg1215_migration;\n\\c pg15_db;\nDROP DATABASE store;\n```\n\n**Drop the publication on PG 12:**\n\n```shell\ndocker compose exec postgres12 bash\npsql -U pg12_user -d store\n```\n\n```sql\nDROP PUBLICATION pub_pg1215_migration;\nSELECT pg_drop_replication_slot('sub_pg1215_migration'); -- this may fail if the subscriber already dropped the slot\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmassdriver-cloud%2Fwebinar-postgresql-logical-replication","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmassdriver-cloud%2Fwebinar-postgresql-logical-replication","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmassdriver-cloud%2Fwebinar-postgresql-logical-replication/lists"}