{"id":13880557,"url":"https://github.com/subzerocloud/pg-amqp-bridge","last_synced_at":"2025-07-16T17:30:38.776Z","repository":{"id":46986394,"uuid":"92020566","full_name":"subzerocloud/pg-amqp-bridge","owner":"subzerocloud","description":"Send messages to RabbitMQ from PostgreSQL","archived":true,"fork":false,"pushed_at":"2021-09-02T09:37:27.000Z","size":1713,"stargazers_count":371,"open_issues_count":1,"forks_count":39,"subscribers_count":13,"default_branch":"master","last_synced_at":"2024-08-07T08:14:43.870Z","etag":null,"topics":["amqp","bridge","message-broker","postgresql","postgrest","rabbitmq","subzero"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/subzerocloud.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-05-22T06:46:40.000Z","updated_at":"2024-03-31T19:19:27.000Z","dependencies_parsed_at":"2022-09-07T17:22:37.580Z","dependency_job_id":null,"html_url":"https://github.com/subzerocloud/pg-amqp-bridge","commit_stats":null,"previous_names":[],"tags_count":8,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subzerocloud%2Fpg-amqp-bridge","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subzerocloud%2Fpg-amqp-bridge/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subzerocloud%2Fpg-amqp-bridge/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/subzerocloud%2Fpg-amqp-bridge/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/subzerocloud","download_url":"https://codeload.github.com/subzerocloud/pg-amqp-bridge/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":226148889,"owners_count":17581048,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["amqp","bridge","message-broker","postgresql","postgrest","rabbitmq","subzero"],"created_at":"2024-08-06T08:03:11.705Z","updated_at":"2024-11-24T09:31:55.633Z","avatar_url":"https://github.com/subzerocloud.png","language":"Rust","funding_links":[],"categories":["Rust"],"sub_categories":[],"readme":"# PostgreSQL to AMQP bridge \n\n#### Send messages to RabbitMQ from PostgreSQL\n\n\n![pg-amqp-bridge](/media/pg-amqp-bridge.gif?raw=true \"pg-amqp-bridge\")\n\n## But Why? ![why](/media/but-why.gif?raw=true \"why?\")\n\nThis tool enables a decoupled architecture, think sending emails when a user signs up. Instead of having explicit code in your `signup` function that does the work (and slows down your response), you just have to worry about inserting the row into the database. After this, a database trigger (see below) will generate an event which gets sent to RabbitMQ. From there, you can have multiple consumers reacting to that event (send signup email, send sms notification). Those consumers tend to be very short, self contained scripts.\nIf you pair **pg-amqp-bridge**  and the [Web STOMP](https://www.rabbitmq.com/web-stomp.html) plugin for RabbitMQ , you can enable real time updates with almost zero code.\n\nThe larger goal is to enable the development of backends around [PostgREST](https://postgrest.com)/[subZero](https://subzero.cloud/) philosophy. Check out the [PostgREST Starter Kit](https://github.com/subzerocloud/postgrest-starter-kit) to see how `pg-amqp-bridge` fits in a larger project.\n\n## Alternative upstreams, SSL and WAL events support (commercial)\nFor upstreams other then RabbitMQ (and additional features) check out [pg-event-proxy](https://github.com/subzerocloud/pg-event-proxy-example)\n\nCurrently the supported upstreams\n- amqp 0.9 (RabbitMQ)\n- mqtt (Apache ActiveMQ, Cassandana, HiveMQ, Mosquitto, RabbitMQ, AWS IoT, Amazon MQ, ...)\n- redis pubsub (Redis)\n- SNS (Amazon Simple Notification Service)\n- SQS (Amazon Simple Queue Service)\n- Lambda (AWS Lambda)\n\n\n\n## Configuration\n\nConfiguration is done through environment variables:\n\n- **POSTGRESQL_URI**: e.g. `postgresql://username:password@domain.tld:port/database`\n- **AMQP_URI**: e.g. `amqp://rabbitmq//`\n- **BRIDGE_CHANNELS**: e.g. `pgchannel1:task_queue,pgchannel2:direct_exchange,pgchannel3:topic_exchange`\n- **DELIVERY_MODE**: can be `PERSISTENT` or `NON-PERSISTENT`, default is `NON-PERSISTENT`\n\n**Note:** It's recommended to always use the same name for postgresql channel and exchange/queue in `BRIDGE_CHANNELS`, for example\n`app_events:app_events,table_changes:tables_changes`\n\n## Running in console \n#### Install\n```shell\nVERSION=0.0.1 \\\nPLATFORM=x86_64-unknown-linux-gnu \\\ncurl -SLO https://github.com/subzerocloud/pg-amqp-bridge/releases/download/${VERSION}/pg-amqp-bridge-${VERSION}-${PLATFORM}.tar.gz \u0026\u0026 \\\ntar zxf pg-amqp-bridge-${VERSION}-${PLATFORM}.tar.gz \u0026\u0026 \\\nmv pg-amqp-bridge /usr/local/bin\n```\n#### Run\n```shell\nPOSTGRESQL_URI=\"postgres://postgres@localhost\" \\\nAMQP_URI=\"amqp://localhost//\" \\\nBRIDGE_CHANNELS=\"pgchannel1:task_queue,pgchannel2:direct_exchange,pgchannel3:topic_exchange\" \\\npg-amqp-bridge\n```\n\n## Running as docker container\n\n```shell\ndocker run --rm -it --net=host \\\n-e POSTGRESQL_URI=\"postgres://postgres@localhost\" \\\n-e AMQP_URI=\"amqp://localhost//\" \\\n-e BRIDGE_CHANNELS=\"pgchannel1:task_queue,pgchannel2:direct_exchange,pgchannel3:topic_exchange\" \\\nsubzerocloud/pg-amqp-bridge\n```\n\nYou can enable logging of the forwarded messages with the ```RUST_LOG=info``` environment variable.\n\n## Sending messages\n**Note**: the bridge doesn't declare exchanges or queues, if they aren't previoulsy declared it will exit with an error.\n\n\n#### Sending messages to a queue\n\n```sql\nNOTIFY pgchannel1, 'Task message';\n```\n\nSince ```pgchannel1``` is bound to ```task_queue``` in BRIDGE_CHANNELS ```'Task message'``` will be sent to ```task_queue```.\n\n#### Sending messages to a direct exchange\n\nYou can specify a routing key with the format ```routing_key|message```:\n\n```sql\nNOTIFY pgchannel2, 'direct_key|Direct message';\n```\n\nSince there is a ```pgchannel2:direct_exchange``` declared in BRIDGE_CHANNELS ```'Direct message'``` will be sent to ```direct_exchange``` with a routing key of ```direct_key```.\n\n#### Sending messages to a topic exchange\n\nYou can specify the routing key with the usual syntax used for topic exchanges.\n\n```sql\nNOTIFY pgchannel3, '*.orange|Topic message';\nNOTIFY pgchannel3, 'quick.brown.fox|Topic message';\nNOTIFY pgchannel3, 'lazy.#|Topic message';\nNOTIFY pgchannel3, 'key|X-First-Header: value1, value2; X-Second-Header: value3|message'\n```\n\n## Helper Functions\n\nTo make sending messages a bit easier you can setup the following functions in your database\n\n```sql\ncreate schema rabbitmq;\n\ncreate or replace function rabbitmq.send_message(channel text, routing_key text, message text) returns void as $$\n\tselect\tpg_notify(channel, routing_key || '|' || message);\n$$ stable language sql;\n\ncreate or replace function rabbitmq.on_row_change() returns trigger as $$\n  declare\n    routing_key text;\n    row record;\n  begin\n    routing_key := 'row_change'\n                   '.table-'::text || TG_TABLE_NAME::text || \n                   '.event-'::text || TG_OP::text;\n    if (TG_OP = 'DELETE') then\n        row := old;\n    elsif (TG_OP = 'UPDATE') then\n        row := new;\n    elsif (TG_OP = 'INSERT') then\n        row := new;\n    end if;\n    -- change 'events' to the desired channel/exchange name\n    perform rabbitmq.send_message('events', routing_key, row_to_json(row)::text);\n    return null;\n  end;\n$$ stable language plpgsql;\n```\n\nAfter this, you can send events from your stored procedures like this\n\n```sql\nrabbitmq.send_message('exchange-name', 'routing-key', 'Hi!');\n```\n\nYou can stream row changes by attaching a trigger to tables\n\n```sql\ncreate trigger send_change_event\nafter insert or update or delete on tablename\nfor each row execute procedure rabbitmq.on_row_change();\n```\n\n## Running from source\n\n#### Install Rust\n\n```shell\ncurl https://sh.rustup.rs -sSf | sh\n```\n\n#### Run\n\n```shell\nPOSTGRESQL_URI=\"postgres://postgres@localhost\" \\\nAMQP_URI=\"amqp://localhost//\" \\\nBRIDGE_CHANNELS=\"pgchannel1:task_queue,pgchannel2:direct_exchange,pgchannel3:topic_exchange\" \\\ncargo run\n```\n\n#### Test\n\n**Note**: RabbitMQ and PostgreSQL need to be running on your localhost\n\n```shell\ncargo test\n```\n\n## Contributing\n\nAnyone and everyone is welcome to contribute.\n\n## Support\n\n* [Slack](https://slack.subzero.cloud/) — Watch announcements, share ideas and feedback\n* [GitHub Issues](https://github.com/subzerocloud/pg-amqp-bridge/issues) — Check open issues, send feature requests\n\n## Author\n\n[@steve-chavez](https://github.com/steve-chavez)\n\n## Credits\n\nInspired by [@FGRibreau](https://github.com/FGRibreau/postgresql-to-amqp)'s work\n\n## License\n\nCopyright © 2017-present subZero Cloud, LLC.\u003cbr /\u003e\nThis source code is licensed under [MIT](https://github.com/subzerocloud/pg-amqp-bridge/blob/master/LICENSE.txt) license\u003cbr /\u003e\nThe documentation to the project is licensed under the [CC BY-SA 4.0](http://creativecommons.org/licenses/by-sa/4.0/) license.\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubzerocloud%2Fpg-amqp-bridge","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsubzerocloud%2Fpg-amqp-bridge","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsubzerocloud%2Fpg-amqp-bridge/lists"}