{"id":42482627,"url":"https://github.com/cloudspannerecosystem/spanner-change-streams-tail","last_synced_at":"2026-01-28T11:15:47.982Z","repository":{"id":64211327,"uuid":"567055588","full_name":"cloudspannerecosystem/spanner-change-streams-tail","owner":"cloudspannerecosystem","description":"CLI to tail Cloud Spanner change streams","archived":false,"fork":false,"pushed_at":"2024-06-07T01:53:39.000Z","size":419,"stargazers_count":18,"open_issues_count":6,"forks_count":2,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-06-19T06:26:59.625Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudspannerecosystem.png","metadata":{"files":{"readme":"README.md","changelog":"changestreams/dialect.go","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-11-17T01:09:17.000Z","updated_at":"2024-04-07T04:13:06.000Z","dependencies_parsed_at":"2024-04-19T15:26:03.390Z","dependency_job_id":"9ea1bd31-ea9b-498c-9d5f-67350fcb956e","html_url":"https://github.com/cloudspannerecosystem/spanner-change-streams-tail","commit_stats":{"total_commits":18,"total_committers":1,"mean_commits":18.0,"dds":0.0,"last_synced_commit":"a8ae95417e88316d2028a04279c927500c1d6543"},"previous_names":[],"tags_count":5,"template":false,"template_full_name":null,"purl":"pkg:github/cloudspannerecosystem/spanner-change-streams-tail","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-change-streams-tail","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-change-streams-tail/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-change-streams-tail/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-change-streams-tail/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudspannerecosystem","download_url":"https://codeload.github.com/cloudspannerecosystem/spanner-change-streams-tail/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-change-streams-tail/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28844862,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T10:53:21.605Z","status":"ssl_error","status_checked_at":"2026-01-28T10:53:20.789Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-28T11:15:47.215Z","updated_at":"2026-01-28T11:15:47.976Z","avatar_url":"https://github.com/cloudspannerecosystem.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# spanner-change-streams-tail\n\n[![run-tests](https://github.com/cloudspannerecosystem/spanner-change-streams-tail/actions/workflows/run-tests.yaml/badge.svg)](https://github.com/cloudspannerecosystem/spanner-change-streams-tail/actions/workflows/run-tests.yaml) [![godoc](https://pkg.go.dev/badge/github.com/cloudspannerecosystem/spanner-change-streams-tail/changestreams)](https://pkg.go.dev/github.com/cloudspannerecosystem/spanner-change-streams-tail/changestreams)\n\nA handy tool to \"tail -f\" [Cloud Spanner Change Streams](https://cloud.google.com/spanner/docs/change-streams) on the local machine.\n\nBoth GoogleSQL and PostgreSQL database dialects are supported.\n\n## Install\n\n```\ngo install github.com/cloudspannerecosystem/spanner-change-streams-tail@latest\n```\n\n## Usage\n\n```\nUsage:\n  spanner-change-streams-tail [OPTIONS]\n\nOptions:\n  -p, --project=  (required)   GCP Project ID\n  -i, --instance= (required)   Cloud Spanner Instance ID\n  -d, --database= (required)   Cloud Spanner Database ID\n  -s, --stream=   (required)   Cloud Spanner Change Stream ID\n  -f, --format=                Output format [text|json] (default: text)\n      --start=                 Start timestamp with RFC3339 format (default: current timestamp)\n      --end=                   End timestamp with RFC3339 format (default: none)\n      --visualize-partitions   Visualize the change stream partitions in Graphviz DOT\n\nHelp Options:\n  -h, -help                    Show this help message\n```\n\n## Example\n\n### Simple example\n\nBy default, this tool shows the Data Change record in plan text.\n\n```\n$ spanner-change-streams-tail -p myproject -i myinstance -d mydb -s mystream\nReading the stream...\n2022-05-19 06:49:15.093823 +0000 UTC | INSERT | Players | [{\"keys\":{\"PlayerId\":\"29\"},\"new_values\":{\"Name\":\"foo\"},\"old_values\":{}}]\n2022-05-19 06:49:20.866495 +0000 UTC | DELETE | Players | [{\"keys\":{\"PlayerId\":\"29\"},\"new_values\":{},\"old_values\":{\"Name\":\"foo\"}}]\n2022-05-20 13:44:32.486447 +0000 UTC | UPDATE | Players | [{\"keys\":{\"PlayerId\":\"23\"},\"new_values\":{\"Name\":\"bar\"},\"old_values\":{\"Name\":\"foo\"}}]\n...\n```\n\n### JSON format\n\nWith `-f json` option, you can get the results in JSON.\n\n```\n$ spanner-change-streams-tail -p myproject -i myinstance -d mydb -s mystream -f json\nReading the stream...\n{\"commit_timestamp\":\"2022-05-19T06:46:12.536575Z\",\"record_sequence\":\"00000000\",\"server_transaction_id\":\"NjQxOTE0MDE0MzM1MDQ4NTQ5NQ==\",\"is_last_record_in_transaction_in_partition\":true,\"table_name\":\"Players\",\"column_types\":[{\"name\":\"PlayerId\",\"type\":{\"code\":\"INT64\"},\"is_primary_key\":true,\"ordinal_position\":1},{\"name\":\"Name\",\"type\":{\"code\":\"STRING\"},\"is_primary_key\":false,\"ordinal_position\":2}],\"mods\":[{\"keys\":{\"PlayerId\":\"22\"},\"new_values\":{\"Name\":\"foo\"},\"old_values\":{}}],\"mod_type\":\"INSERT\",\"value_capture_type\":\"OLD_AND_NEW_VALUES\",\"number_of_records_in_transaction\":1,\"number_of_partitions_in_transaction\":1}\n{\"commit_timestamp\":\"2022-05-19T09:45:59.480799Z\",\"record_sequence\":\"00000000\",\"server_transaction_id\":\"MTIwNjc4MTEyNTU3NDc1MDk5MjA=\",\"is_last_record_in_transaction_in_partition\":true,\"table_name\":\"Players\",\"column_types\":[{\"name\":\"PlayerId\",\"type\":{\"code\":\"INT64\"},\"is_primary_key\":true,\"ordinal_position\":1},{\"name\":\"Name\",\"type\":{\"code\":\"STRING\"},\"is_primary_key\":false,\"ordinal_position\":2}],\"mods\":[{\"keys\":{\"PlayerId\":\"23\"},\"new_values\":{\"Name\":\"bar\"},\"old_values\":{}}],\"mod_type\":\"INSERT\",\"value_capture_type\":\"OLD_AND_NEW_VALUES\",\"number_of_records_in_transaction\":1,\"number_of_partitions_in_transaction\":1}\n{\"commit_timestamp\":\"2022-05-20T13:45:27.682335Z\",\"record_sequence\":\"00000000\",\"server_transaction_id\":\"MTE1NTE3OTU3NzM5MjEyMzkxMzI=\",\"is_last_record_in_transaction_in_partition\":true,\"table_name\":\"Players\",\"column_types\":[{\"name\":\"PlayerId\",\"type\":{\"code\":\"INT64\"},\"is_primary_key\":true,\"ordinal_position\":1},{\"name\":\"Name\",\"type\":{\"code\":\"STRING\"},\"is_primary_key\":false,\"ordinal_position\":2}],\"mods\":[{\"keys\":{\"PlayerId\":\"23\"},\"new_values\":{\"Name\":\"bar\"},\"old_values\":{\"Name\":\"foo\"}}],\"mod_type\":\"UPDATE\",\"value_capture_type\":\"OLD_AND_NEW_VALUES\",\"number_of_records_in_transaction\":1,\"number_of_partitions_in_transaction\":1}\n...\n```\n\n### JSON format with jq\n\nYou can use `jq` command to modify the results.\n\n```\n$ spanner-change-streams-tail -p myproject -i myinstance -d mydb -s mystream -f json | jq '{ts:.commit_timestamp, type:.mod_type, table:.table_name}'\nReading the stream...\n{\n  \"ts\": \"2022-05-20T08:13:45.695039Z\",\n  \"type\": \"INSERT\",\n  \"table\": \"Players\"\n}\n{\n  \"ts\": \"2022-05-20T08:14:17.668655Z\",\n  \"type\": \"DELETE\",\n  \"table\": \"Players\"\n}\n{\n  \"ts\": \"2022-05-20T13:46:27.347695Z\",\n  \"type\": \"UPDATE\",\n  \"table\": \"Players\"\n}\n...\n```\n\n### Start \u0026 End timestamp\n\nWith `--start` and `--end` options, you can specify the time boundary of the records that be read. Both options must\nbe [RFC3339](https://datatracker.ietf.org/doc/html/rfc3339) format.\n\n```\n$ spanner-change-streams-tail -p myproject -i myinstance -d mydb -s mystream --start='2022-05-19T14:28:00Z' --end='2022-05-19T15:04:00Z'\nReading the stream...\n2022-05-19 14:28:50.566943 +0000 UTC | INSERT | Players | [{\"keys\":{\"PlayerId\":\"29\"},\"new_values\":{\"Name\":\"foo\"},\"old_values\":{}}]\n2022-05-19 15:03:07.866495 +0000 UTC | DELETE | Players | [{\"keys\":{\"PlayerId\":\"29\"},\"new_values\":{},\"old_values\":{\"Name\":\"foo\"}}]\n2022-05-19 15:03:28.907391 +0000 UTC | UPDATE | Players | [{\"keys\":{\"PlayerId\":\"20\"},\"new_values\":{\"Name\":\"abc\"},\"old_values\":{\"Name\":\"foo\"}}]\n```\n\n### Verbose output\n\nWith `-v, --verbose` option, you can get the Heartbeat and Child Partitions records as well. Also, each result includes\nthe `partition_token` that associates with the change record.\n\n```\n$ spanner-change-streams-tail -p myproject -i myinstance -d mydb -s mystream --verbose\nReading the stream...\n{\"partition_token\":\"\",\"change_record\":[{\"data_change_record\":[],\"heartbeat_record\":[],\"child_partitions_record\":[{\"start_timestamp\":\"2022-05-20T08:23:10.12375Z\",\"record_sequence\":\"00000001\",\"child_partitions\":[{\"token\":\"AUKmAmgw5S0xbORt3X6EPHBTEXRL5H7VVRh1T7I0xeX_M04SnhhFYBOjQuQZ3AHCh6jGc3gsxAqOHRMHyinqts18NY-JY7Ym5fvSoAGouuSmH6Gff1LspwazfdBRY8_G1enbeBuQNa8b1AEG_KsuhFJCdsr6_Q\",\"parent_partition_tokens\":[]}]}]}]}\n{\"partition_token\":\"\",\"change_record\":[{\"data_change_record\":[],\"heartbeat_record\":[],\"child_partitions_record\":[{\"start_timestamp\":\"2022-05-20T08:23:10.12375Z\",\"record_sequence\":\"00000002\",\"child_partitions\":[{\"token\":\"AUKmAmi65l6TU-0EGTTAj9zLPBU_aJJ1Jsy3JLIkWIH-SSb_nXfTb6X4CLmTQFSkZj-QL_NiGi3p0jGZNQZ8C1WF01GkgvIQ7Qaf4XFxVqSBgPuXBzdpLiye58fmj_Dz2lnV_LYTtPgQcdvOUGJU\",\"parent_partition_tokens\":[]}]}]}]}\n{\"partition_token\":\"AUKmAmgw5S0xbORt3X6EPHBTEXRL5H7VVRh1T7I0xeX_M04SnhhFYBOjQuQZ3AHCh6jGc3gsxAqOHRMHyinqts18NY-JY7Ym5fvSoAGouuSmH6Gff1LspwazfdBRY8_G1enbeBuQNa8b1AEG_KsuhFJCdsr6_Q\",\"change_record\":[{\"data_change_record\":[],\"heartbeat_record\":[{\"timestamp\":\"2022-05-20T08:23:20.123938Z\"}],\"child_partitions_record\":[]}]}\n{\"partition_token\":\"AUKmAmi65l6TU-0EGTTAj9zLPBU_aJJ1Jsy3JLIkWIH-SSb_nXfTb6X4CLmTQFSkZj-QL_NiGi3p0jGZNQZ8C1WF01GkgvIQ7Qaf4XFxVqSBgPuXBzdpLiye58fmj_Dz2lnV_LYTtPgQcdvOUGJU\",\"change_record\":[{\"data_change_record\":[],\"heartbeat_record\":[{\"timestamp\":\"2022-05-20T08:23:20.123904Z\"}],\"child_partitions_record\":[]}]}\n...\n```\n\n### Visualize partitions\n\nWith `--visualize-partitions` option, you can get the visualized partitions in Graphviz DOT format. You also need to\nspecify `--start` and `--end` options to specify the time bound for visualization.\n\n```\n$ spanner-change-streams-tail -p myproject -i myinstance -d mydb -s mystream --start=\"2022-05-23T17:20:00+09:00\" --end=\"2022-05-23T19:20:00+09:00\" --visualize-partitions\nReading the stream and analyzing partitions...\n\ndigraph {\n  node [shape=record];\n  \"AUKmAmidgXbEhh5eV1sR\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmidgXbEhh5eV1sR}|{2022-05-23T09:27:12Z}|{00000000}}\"];\n  \"AUKmAmi9L9YIb2qduDyp\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmi9L9YIb2qduDyp}|{2022-05-23T09:03:33Z}|{00000000}}\"];\n  \"AUKmAmj15z9icOuxjLip\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmj15z9icOuxjLip}|{2022-05-23T09:53:48Z}|{00000000}}\"];\n  \"AUKmAmjTD8SgGdkyPRqR\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmjTD8SgGdkyPRqR}|{2022-05-23T08:20:00Z}|{00000001}}\"];\n  \"AUKmAmjJ_KI60k-_yBcg\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmjJ_KI60k-_yBcg}|{2022-05-23T09:25:49Z}|{00000000}}\"];\n  \"AUKmAmhpe-NDUcBTm4MH\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmhpe-NDUcBTm4MH}|{2022-05-23T09:53:18Z}|{00000000}}\"];\n  \"AUKmAmj8bEIE227zJGuZ\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmj8bEIE227zJGuZ}|{2022-05-23T10:19:25Z}|{00000000}}\"];\n  \"AUKmAmgDoM2U4AQTeLCK\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmgDoM2U4AQTeLCK}|{2022-05-23T10:19:25Z}|{00000000}}\"];\n  \"root\" [label=\"{token|start_timestamp|record_sequence}|{{root}|{0001-01-01T00:00:00Z}|{}}\"];\n  \"AUKmAmj_kYtI0skOqool\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmj_kYtI0skOqool}|{2022-05-23T08:39:33Z}|{00000000}}\"];\n  \"AUKmAmhnVDPUd6zZn-Vs\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmhnVDPUd6zZn-Vs}|{2022-05-23T08:39:33Z}|{00000000}}\"];\n  \"AUKmAmiF2oA66F_yWhIL\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmiF2oA66F_yWhIL}|{2022-05-23T09:27:12Z}|{00000000}}\"];\n  \"AUKmAmieKUi4_ECN8qCf\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmieKUi4_ECN8qCf}|{2022-05-23T08:20:00Z}|{00000002}}\"];\n  \"AUKmAmivn5arzRwNTqm-\" [label=\"{token|start_timestamp|record_sequence}|{{AUKmAmivn5arzRwNTqm-}|{2022-05-23T08:31:03Z}|{00000000}}\"];\n  \"root\" -\u003e \"AUKmAmieKUi4_ECN8qCf\"\n  \"AUKmAmjTD8SgGdkyPRqR\" -\u003e \"AUKmAmivn5arzRwNTqm-\"\n  \"AUKmAmieKUi4_ECN8qCf\" -\u003e \"AUKmAmhnVDPUd6zZn-Vs\"\n  \"AUKmAmi9L9YIb2qduDyp\" -\u003e \"AUKmAmiF2oA66F_yWhIL\"\n  \"AUKmAmi9L9YIb2qduDyp\" -\u003e \"AUKmAmidgXbEhh5eV1sR\"\n  \"root\" -\u003e \"AUKmAmjTD8SgGdkyPRqR\"\n  \"AUKmAmivn5arzRwNTqm-\" -\u003e \"AUKmAmjJ_KI60k-_yBcg\"\n  \"AUKmAmj_kYtI0skOqool\" -\u003e \"AUKmAmi9L9YIb2qduDyp\"\n  \"AUKmAmhnVDPUd6zZn-Vs\" -\u003e \"AUKmAmi9L9YIb2qduDyp\"\n  \"AUKmAmjJ_KI60k-_yBcg\" -\u003e \"AUKmAmj15z9icOuxjLip\"\n  \"AUKmAmhpe-NDUcBTm4MH\" -\u003e \"AUKmAmgDoM2U4AQTeLCK\"\n  \"AUKmAmieKUi4_ECN8qCf\" -\u003e \"AUKmAmj_kYtI0skOqool\"\n  \"AUKmAmiF2oA66F_yWhIL\" -\u003e \"AUKmAmhpe-NDUcBTm4MH\"\n  \"AUKmAmidgXbEhh5eV1sR\" -\u003e \"AUKmAmhpe-NDUcBTm4MH\"\n  \"AUKmAmhpe-NDUcBTm4MH\" -\u003e \"AUKmAmj8bEIE227zJGuZ\"\n}\n```\n\n![Partitions](./partitions.png)\n\n## Go library\n\nThis repository also has `changestreams` package that can be used as a Go library to read the change streams from your\nGo application. You can get more details from\nthe [Godoc](https://pkg.go.dev/github.com/cloudspannerecosystem/spanner-change-streams-tail/changestreams).\n\nNote that `changestreams` package has limited scalability. If you need more scalable, reliable solution, you can use an\nofficial [Dataflow connector](https://cloud.google.com/spanner/docs/change-streams/use-dataflow).\n\n## Disclaimer\n\nPlease feel free to report issues and send pull requests, but note that this application is not officially supported as\npart of the Cloud Spanner product.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudspannerecosystem%2Fspanner-change-streams-tail","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudspannerecosystem%2Fspanner-change-streams-tail","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudspannerecosystem%2Fspanner-change-streams-tail/lists"}