{"id":20058822,"url":"https://github.com/ably-labs/ably-postgres-connector","last_synced_at":"2025-05-05T15:30:23.283Z","repository":{"id":47467369,"uuid":"351145543","full_name":"ably-labs/ably-postgres-connector","owner":"ably-labs","description":"A library to trigger realtime messages to millions of clients on any changes to your PostgresDB ","archived":false,"fork":false,"pushed_at":"2021-09-14T15:30:34.000Z","size":422,"stargazers_count":12,"open_issues_count":0,"forks_count":0,"subscribers_count":3,"default_branch":"main","last_synced_at":"2024-08-09T14:58:58.017Z","etag":null,"topics":["postgres","postgresql","realtime-database"],"latest_commit_sha":null,"homepage":"https://www.npmjs.com/package/ably-postgres-connector","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/ably-labs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2021-03-24T16:14:29.000Z","updated_at":"2024-07-18T12:55:01.000Z","dependencies_parsed_at":"2022-08-24T13:30:47.307Z","dependency_job_id":null,"html_url":"https://github.com/ably-labs/ably-postgres-connector","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably-labs%2Fably-postgres-connector","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably-labs%2Fably-postgres-connector/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably-labs%2Fably-postgres-connector/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/ably-labs%2Fably-postgres-connector/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/ably-labs","download_url":"https://codeload.github.com/ably-labs/ably-postgres-connector/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":224452842,"owners_count":17313668,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["postgres","postgresql","realtime-database"],"created_at":"2024-11-13T13:04:26.630Z","updated_at":"2024-11-13T13:04:27.291Z","avatar_url":"https://github.com/ably-labs.png","language":"TypeScript","funding_links":[],"categories":[],"sub_categories":[],"readme":"## Streaming PostgresDB changes to millions of clients in realtime\r\n\r\nThe Ably-Postgres connector publishes a message on a given Ably channel whenever any operations (insert/update/delete) are executed on the tables of your PostgreSQL database.\r\n\r\nYou can setup the connector with the configuration details of your database, as well as the Ably app, including your API Key, channel names for various types of updates, etc.\r\n\r\nCheck out the [example config](config/default.json) for more info.\r\n\r\n### Prerequisites\r\n\r\n- [PostgreSQL](https://www.postgresql.org/) (this project was tested on version 13)\r\n- [An Ably account](https://ably.com/)\r\n\r\n### Installation\r\n\r\n```sh\r\n    npm install ably-postgres-connector --save\r\n```\r\n\r\n### Setup config\r\n\r\n- The first step is to add in your configuration. You can do this via env file or a JSON file.\r\n\r\n#### Option 1 - Adding config via a JSON file\r\n\r\n- Create a JSON config file within your application, let's say `config/default.json` for example. (refer to the [example JSON config](config/default.json)).\r\n- Add your database and Ably account credentials as needed.\r\n\r\n##### Example usage\r\n\r\n```javascript\r\nconst { Connector } = require(\"ably-postgres-connector\");\r\nconst useWithJSONConfig = () =\u003e {\r\n  const ablyconnector = new Connector(\"config/default.json\");\r\n  ablyconnector.start();\r\n};\r\nuseWithJSONConfig();\r\n```\r\n\r\n##### Running\r\n\r\n```sh\r\n    cd examples\r\n    npm i\r\n    node with-json-config.js\r\n```\r\n\r\n#### Option 2 - Adding config via a env file\r\n\r\n- Create a env config file within your application, let's say `config/.env` for example. (refer to the [example env config](config/.env)).\r\n- Add your database and Ably account credentials as needed.\r\n\r\n##### Example usage\r\n\r\n```javascript\r\nconst { Connector } = require(\"ably-postgres-connector\");\r\nconst useWithEnvConfig = () =\u003e {\r\n  const ablyconnector = new Connector(\"config/.env\");\r\n  ablyconnector.start();\r\n};\r\nuseWithEnvConfig();\r\n```\r\n\r\n##### Running (Using the example file)\r\n\r\n```sh\r\n    cd examples\r\n    npm i\r\n    node with-env-config.js\r\n```\r\n\r\n#### Option 3 - Adding config via a env file through docker-compose\r\n\r\n- Create a env config file within your application, let's say `config/.env` for example. (refer to the [example env config](config/.env)).\r\n- Add your database and Ably account credentials as needed.\r\n- Add path of `.env` file to your `docker-compose` file (refer to the [example docker-compose](docker-compose.yml)).\r\n\r\n##### Example usage\r\n\r\n```javascript\r\nconst { Connector } = require(\"ably-postgres-connector\");\r\nconst useWithEnvDockerCompose = () =\u003e {\r\n  const ablyconnector = new Connector();\r\n  ablyconnector.start();\r\n};\r\nuseWithEnvDockerCompose();\r\n```\r\n\r\n```yaml\r\n# connector-block\r\nconnector:\r\n  build:\r\n    context: .\r\n  env_file: ./config/.env\r\n  depends_on:\r\n    - db\r\n  ports:\r\n    - \"3000:3000\"\r\n```\r\n\r\n##### Running (Using the example docker-compose file)\r\n\r\n- Uses the `Docker` folder to setup the postgresql image with a dummy DB \u0026 users table.\r\n- Uses the `Dockerfile` to create the container with node, build the connector \u0026 add the config file.\r\n\r\n```sh\r\n    docker-compose run connector\r\n```\r\n\r\n### Connector in Action!\r\n\r\nVisit your Ably dev console and connect to the channel `ably-users-added` (or whichever channel you specified in your config). Try performing various operations (insert, update, delete) on your table. For every change, you should see a new message in the specific channel(s).\r\n\r\n## How does the connector work?\r\n\r\n\u003cimg width=\"1252\" alt=\"ably-to-db-postgres@2x (3)\" src=\"https://user-images.githubusercontent.com/5900152/131161607-cf4ff6d9-f6d6-45c9-9a3e-caa9d26a8b51.png\"\u003e\r\n\r\n- The config file contains the details related to the tables you want to listen for data changes on and your Ably API key.\r\n- Using that config file, the connector creates an Ably config table `ablycontroltable` to maintain the table to Ably channel mapping in the DB.\r\n- The connector then creates a DB procedure/function which performs the [`pg_notify`](https://www.postgresql.org/docs/current/sql-notify.html) function that publishes data changes on a data channel.\r\n- The connector then creates triggers for the table-operation combination specified in the config. The job of the trigger is to execute the procedure created above.\r\n- The connector is listening for changes on that particular data channel using the [`LISTEN`](https://www.postgresql.org/docs/current/sql-listen.html) feature. When it gets a notification it publishes the data on the appropriate Ably channel.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fably-labs%2Fably-postgres-connector","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fably-labs%2Fably-postgres-connector","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fably-labs%2Fably-postgres-connector/lists"}