{"id":21300923,"url":"https://github.com/kibae/pg-logical-replication","last_synced_at":"2025-05-16T02:07:21.922Z","repository":{"id":21011792,"uuid":"89267029","full_name":"kibae/pg-logical-replication","owner":"kibae","description":"PostgreSQL Logical Replication client for node.js","archived":false,"fork":false,"pushed_at":"2025-05-11T18:27:51.000Z","size":575,"stargazers_count":106,"open_issues_count":9,"forks_count":23,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-05-11T18:35:50.603Z","etag":null,"topics":["cdc","decoderbufs","logical-decoding","logical-replication","nodejs","pgoutput","postgres","postgresql","typescript","wal2json"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/pg-logical-replication","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kibae.png","metadata":{"files":{"readme":"README-1.x.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.md","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-04-24T17:14:04.000Z","updated_at":"2025-05-11T18:20:23.000Z","dependencies_parsed_at":"2024-06-18T20:07:39.682Z","dependency_job_id":"6046c060-71f2-447b-8b0a-60d6833e1419","html_url":"https://github.com/kibae/pg-logical-replication","commit_stats":{"total_commits":48,"total_committers":8,"mean_commits":6.0,"dds":"0.41666666666666663","last_synced_commit":"9b191c5c4e0e9e953537c933fc331ae711e77587"},"previous_names":[],"tags_count":9,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kibae%2Fpg-logical-replication","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kibae%2Fpg-logical-replication/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kibae%2Fpg-logical-replication/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kibae%2Fpg-logical-replication/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kibae","download_url":"https://codeload.github.com/kibae/pg-logical-replication/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254453652,"owners_count":22073617,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["cdc","decoderbufs","logical-decoding","logical-replication","nodejs","pgoutput","postgres","postgresql","typescript","wal2json"],"created_at":"2024-11-21T15:42:18.118Z","updated_at":"2025-05-16T02:07:21.889Z","avatar_url":"https://github.com/kibae.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pg-logical-replication\n\n- PostgreSQL Logical Replication client for node.js\n\n## 1. Install\n- **pg-logical-replication** depends on [pq (node-postgres)](https://github.com/brianc/node-postgres) \u003e= 6.2.2\n\n```sh\n$ npm install pg-logical-replication\n```\n\n## 2. LogicalReplication\n```javascript\nnew LogicalReplication( object config ) : Stream\n```\n- Creates a new, unconnected instance of a logical replication stream configured via supplied configuration object.\n- https://github.com/brianc/node-postgres/wiki/Client#parameters\n\n```javascript\nvar LogicalReplication = require('pg-logical-replication');\nvar stream = new LogicalReplication({/*config*/});\n```\n\n## 3. Stream\n### 3-1. Method - Stream.getChanges\n- Start WAL streaming of changes.\n```javascript\nstream.getChanges( /*string*/ slotName, /*string*/ uptoLsn, /*object*/ option, /*function(err)*/ initialErrorCallback );\n```\n- ```uptoLsn``` can be null, the minimum value is \"0/00000000\".\n- ```option``` can contain any of the following optional properties\n    - ```standbyMessageTimeout``` : maximum seconds between keepalive messages (default: 10)\n    - ```includeXids``` : bool (default: false)\n    - ```includeTimestamp``` : bool (default: false)\n    - ```queryOptions``` : object containing decoder specific options (optional)\n        - ```'include-types': false```\n        - ```'filter-tables': 'foo.bar'```\n\n### 3-2. Method - Stream.stop\n- Stop WAL streaming.\n```javascript\nstream.stop();\n```\n\n### 3-3. Event - Stream.on('data')\n- Raised when new data streamed from PostgreSQL server.\n```javascript\nstream.on('data', (/*object*/ msg)=\u003e{/*...*/});\n```\n- ```msg``` contains ```lsn (string)```, ```log (buffer)```\n\n### 3-4. Event - Stream.on('error')\n- Raised when error or disconnected.\n```javascript\nstream.on('error', (/*object*/ err)=\u003e{/*...*/});\n```\n\n## 4. Plugin\n### 4-1. test_decoding output\n- If you are using ```test_decoding```, this plugin will be useful.\n```javascript\nvar PluginTestDecoding = LogicalReplication.LoadPlugin('output/test_decoding');\nPluginTestDecoding.parse(msg.log.toString('utf8'));\n```\n\n\n----\n\n## Example\n```javascript\n/*\n * Test progress\n * 1. Create logical replication slot with test_decoding output plugin : SELECT * FROM pg_create_logical_replication_slot('test_slot', 'test_decoding');\n * 2. Launch nodejs with this file : node test.js\n * 3. Modify data of database\n */\nvar LogicalReplication = require('pg-logical-replication');\nvar PluginTestDecoding = LogicalReplication.LoadPlugin('output/test_decoding');\n\n//Connection parameter : https://github.com/brianc/node-postgres/wiki/Client#parameters\nvar connInfo = {};\n\n//Initialize with last LSN value\nvar lastLsn = null;\n\nvar stream = (new LogicalReplication(connInfo))\n  .on('data', function(msg) {\n    lastLsn = msg.lsn || lastLsn;\n\n    var log = (msg.log || '').toString('utf8');\n    try {\n      console.log(PluginTestDecoding.parse(log));\n      //TODO: DO SOMETHING. eg) replicate to other dbms(pgsql, mysql, ...)\n    } catch (e) {\n      console.trace(log, e);\n    }\n  }).on('error', function(err) {\n    console.trace('Error #2', err);\n    setTimeout(proc, 1000);\n  });\n\n(function proc() {\n  stream.getChanges('test_slot', lastLsn, {\n    includeXids: false, //default: false\n    includeTimestamp: false, //default: false\n  }, function(err) {\n    if (err) {\n      console.trace('Logical replication initialize error', err);\n      setTimeout(proc, 1000);\n    }\n  });\n})();\n```\n\n## PostgreSQL side\n- postgresql.conf\n```\nwal_level = logical\nmax_wal_senders = [bigger than 1]\nmax_replication_slots = [bigger than 1]\n```\n- Create logical replication slot\n```sql\nSELECT * FROM pg_create_logical_replication_slot('test_slot', 'test_decoding');\n```\n- Delete logical replication slot\n```sql\nSELECT pg_drop_replication_slot('test_slot');\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkibae%2Fpg-logical-replication","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkibae%2Fpg-logical-replication","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkibae%2Fpg-logical-replication/lists"}