{"id":16678232,"url":"https://github.com/mindflavor/pgdelaytest","last_synced_at":"2026-05-18T00:31:20.949Z","repository":{"id":93918958,"uuid":"567758275","full_name":"MindFlavor/pgdelaytest","owner":"MindFlavor","description":"A simple tool to evaluate replication latency on PostgreSQL","archived":false,"fork":false,"pushed_at":"2022-12-12T14:00:50.000Z","size":44,"stargazers_count":2,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-07-17T01:44:09.002Z","etag":null,"topics":["gcp-pubsub","latency-monitor","postgresql","rust"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/MindFlavor.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-18T14:05:58.000Z","updated_at":"2024-07-09T10:31:05.000Z","dependencies_parsed_at":"2023-03-13T17:09:25.022Z","dependency_job_id":null,"html_url":"https://github.com/MindFlavor/pgdelaytest","commit_stats":null,"previous_names":[],"tags_count":1,"template":false,"template_full_name":null,"purl":"pkg:github/MindFlavor/pgdelaytest","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MindFlavor%2Fpgdelaytest","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MindFlavor%2Fpgdelaytest/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MindFlavor%2Fpgdelaytest/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MindFlavor%2Fpgdelaytest/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/MindFlavor","download_url":"https://codeload.github.com/MindFlavor/pgdelaytest/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/MindFlavor%2Fpgdelaytest/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":33160452,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-05-17T22:39:12.733Z","status":"ssl_error","status_checked_at":"2026-05-17T22:39:10.741Z","response_time":107,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["gcp-pubsub","latency-monitor","postgresql","rust"],"created_at":"2024-10-12T13:28:49.887Z","updated_at":"2026-05-18T00:31:20.933Z","avatar_url":"https://github.com/MindFlavor.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PostgreSQL latency test\n\n## What is it?\n\nThis small utility shows the replication latency between two PG instances. It basically increments a counter on the primary and measures the time it takes for that change to be propagated to the secondary. It can optionally publish every data point to a Google Cloud pub/sub topic of your choosing. This way, with the automatic BigQuery subscription it's possibile to store and analyze tons of datapoints with ease.\n\n## Usage\n\nCompile the tool using Rust (ie `cargo install --path .`) and launch it from the command line. As for PostgreSQL, create a table and insert a single row with this simple script: \n\n```sql\nCREATE TABLE tbl(id INT, value INT);                                                              \n\nINSERT INTO tbl(id, value) VALUES(1, 900);\n```\n\nRequired parameters for the tool are the two connection strings (in the format specificed here: [https://docs.rs/tokio-postgres/latest/tokio_postgres/config/struct.Config.html](https://docs.rs/tokio-postgres/latest/tokio_postgres/config/struct.Config.html)). You can also optionally change the sleep time between tries and the pub/sub topic to write to.\n\nFor example this command:\n\n```bash\npgdelaytest --primary-connection-string \"host=primary user=test password=password\" --secondary-connection-string \"host=secondary user=test password=password\" publish --pub-sub-topic pglatency\n```\n\nTests the latency between `primary` and `secondary`, publishing the results both to stdout and to the GCP topic `pglatency` for streaming to BigQuery.\n\nNote that, in order to publish to pub/sub, a valid GCP identity must be available and proper permissions must be granted.\n\n## Usage (Docker)\n\nYou can either build the container with `docker build . -t pgdelaytest:latest` or pull it from Docker.io. Then execute it passing env variables. For example:\n\n```bash\ndocker run -e PRIMARY_CONNECTION_STRING=\"host=host user=test password=password\" -e SECONDARY_CONNECTION_STRING=\"host=secondary user=test password=password\" -e PUB_SUB_TOPIC=topic -e GOOGLE_APPLICATION_CREDENTIALS=/service_account_pvk.json -v /service_account_pvk.json:/service_account_pvk.json pgdelaytest:latest\n```\n\n*Note*: this example uses a service account key file, it is not necessary if you don't want to publish to pub/sub or you have default credentials at hand.\n\n## Methodology\n\nThe tool updates a row on the primary and right away tries to get the same row from the secondary. If the value matches, the reported latency is zero. If not, the tool keeps querying the same row until the value matches and then reports the time taken as *replication latency*. \n\nThis means two things:\n\n1. Zero latency does not mean zero microseconds: it means the latency is so low that the tool is unable to determine it.\n2. The latency measured, if bigger than zero, incorporates an error the depends on how fast the tool can query the secondary instance. It might be interpreted as an upper bound.\n\nThe tool is able to calculate milliseconds (or even microseconds) but given the constraints above I think it's best to give rough estimates in seconds. If you don't agree, please open an issue and I'll add the option.\n\n## Pub/sub\n\nThe tool optionally publishes the event message to a pub/sub topic in GCP. This can be used to have the automatic BigQuery subscription stream data to BigQuery. You can find the schema of the message in the [schema.proto](extra/schema.proto) file. The BigQuery table definition is in the [create_table.sql](extra/create_table.sql) file.\n\nFor example, this query shows the latest entries:\n\n```sql\nSELECT TIMESTAMP_MILLIS(timestamp) AS event_time, latency_ms \nFROM `mind-lab-wash.someds.frompubsublatency` \nORDER BY timestamp DESC\nLIMIT 1000;\n```\n\nThis can be useful to view the data graphically via Looker Studio or Pro.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindflavor%2Fpgdelaytest","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fmindflavor%2Fpgdelaytest","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fmindflavor%2Fpgdelaytest/lists"}