{"id":19458506,"url":"https://github.com/postgrespro/pg_dtm","last_synced_at":"2025-04-25T06:30:28.148Z","repository":{"id":69542820,"uuid":"45294940","full_name":"postgrespro/pg_dtm","owner":"postgrespro","description":"Distributed transaction manager","archived":false,"fork":false,"pushed_at":"2015-11-18T13:11:05.000Z","size":116,"stargazers_count":26,"open_issues_count":0,"forks_count":5,"subscribers_count":34,"default_branch":"master","last_synced_at":"2025-04-24T10:48:42.393Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"C","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/postgrespro.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2015-10-31T08:43:54.000Z","updated_at":"2024-11-15T23:00:22.000Z","dependencies_parsed_at":"2023-03-11T06:09:47.504Z","dependency_job_id":null,"html_url":"https://github.com/postgrespro/pg_dtm","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgrespro%2Fpg_dtm","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgrespro%2Fpg_dtm/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgrespro%2Fpg_dtm/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgrespro%2Fpg_dtm/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/postgrespro","download_url":"https://codeload.github.com/postgrespro/pg_dtm/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":250766957,"owners_count":21483894,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-10T17:27:23.401Z","updated_at":"2025-04-25T06:30:27.640Z","avatar_url":"https://github.com/postgrespro.png","language":"C","funding_links":[],"categories":[],"sub_categories":[],"readme":"# pg_dtm\n\n### Design\n\nThis repo implements distributed transaction manager using Snapshot Sharing mechanism. General concepts and alternative approaches described in postgres wiki https://wiki.postgresql.org/wiki/DTM.\n\nBackend-DTM protocol description can be found in [dtmd/README](dtmd/README).\n\n### Installation\n\n* Patch postgres using xtm.patch. After that build and install postgres in usual way.\n```bash\ncd ~/code/postgres\npatch -p1 \u003c ~/code/pg_dtm/xtm.patch\n```\n* Install pg_dtm extension.\n```bash\nexport PATH=/path/to/pgsql/bin/:$PATH\ncd ~/code/pg_dtm\nmake \u0026\u0026 make install\n```\n* Run dtmd.\n```bash\ncd ~/code/pg_dtm/dtmd\nmake\nmkdir /tmp/clog\n./bin/dtmd \u0026\n```\n* To run something meaningful you need at leat two postgres instances. Also pg_dtm requires presense in ```shared_preload_libraries```.\n```bash\ninitdb -D ./install/data1\ninitdb -D ./install/data2\necho \"port = 5433\" \u003e\u003e ./install/data2/postgresql.conf\necho \"shared_preload_libraries = 'pg_dtm'\" \u003e\u003e ./install/data1/postgresql.conf\necho \"shared_preload_libraries = 'pg_dtm'\" \u003e\u003e ./install/data2/postgresql.conf\npg_ctl -D ./install/data1 -l ./install/data1/log start\npg_ctl -D ./install/data2 -l ./install/data2/log start\n```\n\n#### Automatic provisioning\n\nFor a cluster-wide deploy we use ansible, more details in tests/deploy_layouts. (Ansible instructions will be later)\n\n### Usage\n\nNow cluster is running and you can use global tx between two nodes. Let's connect to postgres instances at different ports:\n\n```sql\ncreate extension pg_dtm; -- node1\ncreate table accounts(user_id int, amount int); -- node1\ninsert into accounts (select 2*generate_series(1,100)-1, 0); -- node1, odd user_id's\n    create extension pg_dtm; -- node2\n    create table accounts(user_id int, amount int); -- node2\n    insert into accounts (select 2*generate_series(1,100), 0); -- node2, even user_id's\nselect dtm_begin_transaction(); -- node1, returns global xid, e.g. 42\n\tselect dtm_join_transaction(42); -- node2, join global tx\nbegin; -- node1\n\tbegin; -- node2\nupdate accounts set amount=amount-100 where user_id=1; -- node1, transfer money from user#1\n\tupdate accounts set amount=amount+100 where user_id=2; -- node2, to user#2\ncommit; -- node1, blocks until second commit happend\n\tcommit; -- node2\n```\n\n### Consistency testing\n\nTo ensure consistency we use simple bank test: perform a lot of simultaneous transfers between accounts on different servers, while constantly checking total amount of money on all accounts. This test can be found in tests/perf.\n\n```bash\n\u003e go run ./tests/perf/*\n  -C value\n    \tConnection string (repeat for multiple connections)\n  -a int\n    \tThe number of bank accounts (default 100000)\n  -b string\n    \tBackend to use. Possible optinos: transfers, fdw, pgshard, readers. (default \"transfers\")\n  -g\tUse DTM to keep global consistency\n  -i\tInit database\n  -l\tUse 'repeatable read' isolation level instead of 'read committed'\n  -n int\n    \tThe number updates each writer (reader in case of Reades backend) performs (default 10000)\n  -p\tUse parallel execs\n  -r int\n    \tThe number of readers (default 1)\n  -s int\n    \tStartID. Script will update rows starting from this value\n  -v\tShow progress and other stuff for mortals\n  -w int\n    \tThe number of writers (default 8)\n```\n\nSo previous installation can be initialized with:\n```\ngo run ./tests/perf/*.go  \\\n-C \"dbname=postgres port=5432\" \\\n-C \"dbname=postgres port=5433\" \\\n-g -i\n```\nand tested with:\n```\ngo run ./tests/perf/*.go  \\\n-C \"dbname=postgres port=5432\" \\\n-C \"dbname=postgres port=5433\" \\\n-g\n```\n\n### Using with postres_fdw.\n\nWe also provide a patch, that enables support of global transactions with postres_fdw. After patching and installing postres_fdw it is possible to run same test via fdw usig key ```-b fdw```.\n\n### Using with pg_shard\n\nCitus Data have branch in their pg_shard repo, that interacts with transaction manager. https://github.com/citusdata/pg_shard/tree/transaction_manager_integration\nTo use this feature one should have following line in postgresql.conf (or set it via GUC)\n```\npg_shard.use_dtm_transactions = 1\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpostgrespro%2Fpg_dtm","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fpostgrespro%2Fpg_dtm","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fpostgrespro%2Fpg_dtm/lists"}