{"id":17318400,"url":"https://github.com/olavloite/spanner-jdbc-converter","last_synced_at":"2025-04-14T15:22:43.576Z","repository":{"id":24741789,"uuid":"86078967","full_name":"olavloite/spanner-jdbc-converter","owner":"olavloite","description":"A tool for converting any JDBC compliant database to a Google Cloud Spanner database","archived":false,"fork":false,"pushed_at":"2022-09-08T01:14:46.000Z","size":75,"stargazers_count":6,"open_issues_count":1,"forks_count":1,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-03-28T04:12:44.543Z","etag":null,"topics":["conversion","google-cloud-spanner","jdbc"],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/olavloite.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-03-24T14:50:50.000Z","updated_at":"2022-02-02T08:28:05.000Z","dependencies_parsed_at":"2023-01-14T01:31:39.623Z","dependency_job_id":null,"html_url":"https://github.com/olavloite/spanner-jdbc-converter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olavloite%2Fspanner-jdbc-converter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olavloite%2Fspanner-jdbc-converter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olavloite%2Fspanner-jdbc-converter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/olavloite%2Fspanner-jdbc-converter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/olavloite","download_url":"https://codeload.github.com/olavloite/spanner-jdbc-converter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248904729,"owners_count":21180846,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["conversion","google-cloud-spanner","jdbc"],"created_at":"2024-10-15T13:19:43.037Z","updated_at":"2025-04-14T15:22:43.541Z","avatar_url":"https://github.com/olavloite.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# DEPRECATED\nThis project is no longer actively maintained.\n\nIt is recommended that you use the [officially supported open source JDBC driver for Google Cloud Spanner](https://github.com/googleapis/java-spanner-jdbc) in combination with the [corresponding Hibernate dialect](https://github.com/GoogleCloudPlatform/google-cloud-spanner-hibernate).\n\n## spanner-jdbc-converter\nA generic tool for converting JDBC-compliant databases to Google Cloud Spanner and vice-versa. The converter will convert both your data model and data to Google Cloud Spanner. It has currently only been tested with PostgreSQL source databases.\nIt can also convert databases in the other direction, i.e. download a Google Cloud Spanner database into a PostgreSQL database. This way you can create a local backup of your Google Cloud Spanner database.\n\n## How it works\nThe tool does the conversion in these stages:\u003cbr\u003e\n1. Copy the definition of all tables from the source database to Google Cloud Spanner. The tool uses a generic mapping for column data types. The generic mapping can be configured to suit your specific needs. Only tables with a primary key can be converted. \u003cbr\u003e\n2. Copy the data of all tables from the source database to Google Cloud Spanner.\u003cbr\u003e\n\n\nThe conversion of both table definitions and data can be run in two different modes: SkipExisting and DropAndRecreate. \u003cbr\u003e\nIn SkipExisting mode, tables that are already present in the destination Google Cloud Spanner database will be skipped, even if there are structural differences between the source and destination (i.e. it does not update the definition of the table). \u003cbr\u003e\nIn SkipExisting mode, data will only be copied if the destination table is empty. \u003cbr\u003e\nIn DropAndRecreate mode for table definition conversion, existing destination tables will be dropped and re-created during conversion, regardless of the contents of the table. During data conversion, this mode causes a \"DELETE FROM TABLE\" statement to be issued on all tables that are not empty.\n\n## Parallelism\nData copy is performed as much as possible using parallel workers in order to speed up the process. The optimal settings depend on your local resources (number of CPU's, memory, etc.) and the number of nodes on your Cloud Spanner instance.\n\nYou can use these configuration options to set the degree of parallelism and other performance considerations (with default values):\n\nDataConverter.batchSize=1500000\t\t\t// The number of bytes in each commit to Cloud Spanner\nDataConverter.maxNumberOfWorkers=10\t\t// The maximum number of upload workers for one table running in parallel.\nDataConverter.numberOfTableWorkers=10\t// The number of table workers running in parallel.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folavloite%2Fspanner-jdbc-converter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Folavloite%2Fspanner-jdbc-converter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Folavloite%2Fspanner-jdbc-converter/lists"}