{"id":21707755,"url":"https://github.com/flightaware/pg-deltaflood","last_synced_at":"2025-07-24T02:33:21.982Z","repository":{"id":66872741,"uuid":"96554252","full_name":"flightaware/pg-deltaflood","owner":"flightaware","description":"stream changes to a PostgreSQL database in an easily parsed format for mirroring to a third party application or database","archived":false,"fork":false,"pushed_at":"2021-07-08T18:17:43.000Z","size":77,"stargazers_count":9,"open_issues_count":0,"forks_count":2,"subscribers_count":7,"default_branch":"master","last_synced_at":"2025-04-12T16:17:18.451Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flightaware.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-07-07T15:56:57.000Z","updated_at":"2024-05-30T21:23:45.000Z","dependencies_parsed_at":"2023-02-23T10:46:09.575Z","dependency_job_id":null,"html_url":"https://github.com/flightaware/pg-deltaflood","commit_stats":null,"previous_names":[],"tags_count":3,"template":false,"template_full_name":null,"purl":"pkg:github/flightaware/pg-deltaflood","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Fpg-deltaflood","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Fpg-deltaflood/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Fpg-deltaflood/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Fpg-deltaflood/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flightaware","download_url":"https://codeload.github.com/flightaware/pg-deltaflood/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flightaware%2Fpg-deltaflood/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":266785505,"owners_count":23983825,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-24T02:00:09.469Z","response_time":99,"last_error":null,"robots_txt_status":null,"robots_txt_updated_at":null,"robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-25T22:19:08.408Z","updated_at":"2025-07-24T02:33:21.961Z","avatar_url":"https://github.com/flightaware.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"DELTAFLOOD\n==========\n\nThis is a PostgreSQL output plugin that provides a way to stream deltas to a database in an easily _and rapidly_ parsed format for\nmirroring those changes to an external application. It is based on the \"test_decoding\" plugin shipped with the PostgreSQL\ndistribution.\n\nThe output format consists of a stream of tab-separated lines, each line containing the changes to the database, along with\nsome extra pseudo-columns:\n\n* _table table_name\n* _xid (optional)\n* _oid (optional)\n* _lsn (optional)\n* _action [insert|replace|update|delete]\n\nThe \"delete\" actions are only output for tables with a primary key that can be used to uniquely identify deleted rows.\n\nUpdates will be preceded by a replace line with the old primary key when appropriate. The reader can either\ntreat \"update\" as \"insert\" and \"replace\" as \"delete\", or generate an update operation with an appropriate \"where\" clause if it's preceded by a change.\n\nYou really need a primary key or OIDs (oh, hey, that's a primary key) if you want to usefully use this for replication. :)\n\nSETUP\n-----\n\ndeltaflood now uses PGXN so you only need to follow the following instructions for PostgreSQL \u003c= 8.1 and why would you do that?\n\nThis depends on a full copy of the postgres build tree being available. The easiest way to do this is to \"git clone\" the\nPostgres repo and this repo under the same common parent directory. If the postgres build tree is somewhere other than\n../postgres you will need to modify \"top_builddir\" in Makefile to point to it.\n\n```\ngit clone git@github.com:flightaware/pg-deltaflood.git\ngit clone git@github.com:postgres/postgres.git\n```\n\nOr\n\n```\ngit clone https://github.com/flightaware/pg-deltaflood.git\ngit clone https://github.com/postgres/postgres.git\n```\n\n\nBUILDING\n--------\n\nWith current versions of Postgres, if you have it properly set up so `pg_config --pgxs` returns something like `/usr/local/lib/postgresql/pgxs/src/makefiles/pgxs.mk`, just \"gmake; gmake install\"\n\nWith PostgreSQL \u003c= 8.1:\n\nAfter setting up the build trees for postgres and deltaflood next to each other:\n\n```\ncd ../postgres\nautoreconf; ./configure $suitable-options\ngmake\ncd ../pg-deltaflood\ngmake NO_PGXS=1; sudo gmake NO_PGXS=1 install\n```\n\nUSING\n-----\n\n```\npg_recvlogical --create-slot --slot streamer --plugin=deltaflood\npg_recvlogical --start --slot streamer --file=- -o opt=value...\n```\n\nOptions:\n\n* include-xids=bool default true\n\n    include _xid (Transaction ID) pseudo-column\n    \n* include-oids=bool default true\n\n    include _oid (object ID) pseudo_column\n    \n* include-lsn=bool default false\n\n    include _lsn (Logical Serial Number) pseudo_column\n    \n* full-name=bool default false\n\n    use fully qualified table name with schema\n    \n* skip-nulls=bool default true\n\n    Don't even output columns with null values\n    \n* escape_chars=bool default true\n\n    escape tab as \\t, newline as \\n, other control characters as \\NNN in octal\n    \n* convert-bool=bool default true\n\n    convert boolean values to \"1\" or \"0\" regardless of original representation\n\n* null=null_string default NULL\n\n    if skip-nulls=false, then use this as the null string\n    \n* separator=tab default \\t\n\n    separate columns with this string\n    \n* tables=table,table,table default all tables\n\n    Only output rows for listed tables\n\nOutput example\n--------------\n\n```\n$ pg_recvlogical --start --slot streamer --file=- -o tables=bar,baz -o include-oids\n_table\tbar\t_xid\t579\t_action\tinsert\ta\tcat\tb\tmeow\n_table\tbar\t_xid\t581\t_action\tinsert\ta\tcat\tb\tmeow\n_table\tbaz\t_xid\t582\t_action\tinsert\t_oid\t16405\ta\trocket\tb\tscience\n_table\tbaz\t_xid\t584\t_action\tinsert\t_oid\t16406\ta\trocket\tb\tscience\n^C\n$ pg_recvlogical --start --slot streamer --file=- -o include-lsn \n_table\tzzz\t_xid\t7619\t_lsn\t0/177B148\t_action\tinsert\ta\table\tb\ta\n_table\tzzz\t_xid\t7620\t_lsn\t0/177B148\t_action\tdelete\ta\table\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflightaware%2Fpg-deltaflood","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflightaware%2Fpg-deltaflood","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflightaware%2Fpg-deltaflood/lists"}