{"id":42482613,"url":"https://github.com/cloudspannerecosystem/spanner-migration-example","last_synced_at":"2026-01-28T11:15:43.835Z","repository":{"id":222214196,"uuid":"750623252","full_name":"cloudspannerecosystem/spanner-migration-example","owner":"cloudspannerecosystem","description":null,"archived":false,"fork":false,"pushed_at":"2024-02-21T00:22:27.000Z","size":60,"stargazers_count":0,"open_issues_count":1,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2024-04-15T12:16:12.117Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cloudspannerecosystem.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null}},"created_at":"2024-01-31T01:42:43.000Z","updated_at":"2024-02-12T22:48:23.000Z","dependencies_parsed_at":"2024-02-20T03:22:19.281Z","dependency_job_id":null,"html_url":"https://github.com/cloudspannerecosystem/spanner-migration-example","commit_stats":null,"previous_names":["cloudspannerecosystem/spanner-migration-example"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/cloudspannerecosystem/spanner-migration-example","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-migration-example","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-migration-example/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-migration-example/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-migration-example/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cloudspannerecosystem","download_url":"https://codeload.github.com/cloudspannerecosystem/spanner-migration-example/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cloudspannerecosystem%2Fspanner-migration-example/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":28844861,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-01-28T10:53:21.605Z","status":"ssl_error","status_checked_at":"2026-01-28T10:53:20.789Z","response_time":57,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2026-01-28T11:15:43.256Z","updated_at":"2026-01-28T11:15:43.814Z","avatar_url":"https://github.com/cloudspannerecosystem.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# PostgreSQL to Spanner Migration Example\n\nThis is the sample application used to exemplify a migration from an open-source PostgreSQL database (Cloud SQL) to a Spanner PostgreSQL dialect database.\n\nIt uses the open-source JDBC driver for both databases.\n\n## Database Schema\n\n### Open-source PostgreSQL (Cloud SQL)\n\n```sql\n-- When converting to Spanner:\n--   * Table does not contain a primary key column, one must be added\n--   * SERIAL type does not exist in Spanner\nCREATE TABLE singers (\n  singer_id   SERIAL UNIQUE,\n  first_name  VARCHAR(1024),\n  last_name   VARCHAR(1024)\n);\n\n-- When converting to Spanner:\n--   * singer_id has int4 type, which must be converted to int8 in Spanner\n--   * SERIAL type does not exist in Spanner\n--   * album_title UNIQUE must be converted to an UNIQUE index in Spanner\n--   * FOREIGN KEY relation can be converted to Spanner INTERLEAVED table\nCREATE TABLE albums (\n  singer_id     int,\n  album_id      SERIAL,\n  album_title   VARCHAR UNIQUE,\n  PRIMARY KEY (singer_id, album_id),\n  CONSTRAINT fk_singers\n    FOREIGN KEY(singer_id) REFERENCES singers(singer_id)\n    ON DELETE CASCADE\n);\n\n-- When converting to Spanner:\n--   * SERIAL type does not exist in Spanner\n--   * song_data json type must be converted to jsonb in Spanner\n--   * FOREIGN KEY relation can be converted to Spanner INTERLEAVED table\nCREATE TABLE songs (\n  singer_id     int,\n  album_id      int,\n  song_id       SERIAL,\n  song_name     VARCHAR,\n  song_data     json,\n  PRIMARY KEY (singer_id, album_id, song_id),\n  CONSTRAINT fk_albums\n    FOREIGN KEY(singer_id, album_id) REFERENCES albums(singer_id, album_id)\n    ON DELETE CASCADE\n);\n\n-- When converting to Spanner:\n--   * Duplicated index, already taken care of by songs primary key\nCREATE INDEX singer_album_song ON songs(singer_id, album_id);\n```\n\n### Spanner PostgreSQL\n\nThis is the migrated schema on Spanner.\n\n```sql\nCREATE SEQUENCE singer_id_seq BIT_REVERSED_POSITIVE START COUNTER WITH 1;\nCREATE SEQUENCE album_id_seq BIT_REVERSED_POSITIVE START COUNTER WITH 1;\nCREATE SEQUENCE song_id_seq BIT_REVERSED_POSITIVE START COUNTER WITH 1;\n\nCREATE TABLE singers (\n  singer_id bigint DEFAULT nextval('singer_id_seq'::text) NOT NULL,\n  first_name character varying(1024),\n  last_name character varying(1024),\n  PRIMARY KEY(singer_id)\n);\n\nCREATE TABLE albums (\n  singer_id bigint NOT NULL,\n  album_id bigint DEFAULT nextval('album_id_seq'::text) NOT NULL,\n  album_title character varying,\n  PRIMARY KEY(singer_id, album_id)\n) INTERLEAVE IN PARENT singers;\n\nCREATE UNIQUE INDEX albums_album_title_key ON albums (album_title) WHERE (album_title IS NOT NULL);\n\nCREATE TABLE songs (\n  singer_id bigint NOT NULL,\n  album_id bigint NOT NULL,\n  song_id bigint DEFAULT nextval('song_id_seq'::text) NOT NULL,\n  song_name character varying,\n  song_data jsonb,\n  PRIMARY KEY(singer_id, album_id, song_id)\n) INTERLEAVE IN PARENT albums;\n```\n\n## Application Updates\n\nThe original application, which worked only with the open-source PostgreSQL database, had to be updated to work with Spanner PostgreSQL dialect as well.\n\nBelow we go over the necessary modifications.\n\nFirst, we parameterized the application to boot up using either Cloud SQL or Spanner. For this purpose we introduced the [DatabaseChoice](src/main/java/com/google/DatabaseChoice.java) abstraction. In a production scenario we would most likely have used a feature flag here.\nWe modified our [docker-compose.yml](docker-compose.yml) file to input the database choice on initialization:\n\n```yaml\nservices:\n  app-cloudsql\n    ...\n    command: [ \"java\", \"-jar\", \"app.jar\", \"cloudsql\" ]\n  app-spanner:\n    ...\n    command: [ \"java\", \"-jar\", \"app.jar\", \"spanner\" ]\n```\n\nSecondly, we proceeded to set up a Spanner connection. In order to keep using the PostgreSQL drivers in our application (instead of Spanner specific drivers), we had to configure [PGAdapter](https://github.com/GoogleCloudPlatform/pgadapter/tree/postgresql-dialect?tab=readme-ov-file#google-cloud-spanner-pgadapter). PGAdapter serves as a proxy that translates the PostgreSQL wire-protocol into the equivalent for Spanner databases that use the PostgreSQL interface. Our application connects to PGAdapter instead of Spanner. There are [multiple ways to use PGAdapter](https://cloud.google.com/spanner/docs/pgadapter#execution-env). We used the [distroless Docker image](https://github.com/GoogleCloudPlatform/pgadapter/tree/postgresql-dialect?tab=readme-ov-file#distroless-docker-image), as it is independent of the implementation of our application. The changes are reflected in our [docker-compose.yml](docker-compose.yml) file:\n\n```yaml\nservices:\n  ...\n  pgadapter:\n    image: gcr.io/cloud-spanner-pg-adapter/pgadapter-distroless\n    ...\n    \n  app-spanner:\n    depends_on:\n      - pgadapter\n    command: [ \"java\", \"-jar\", \"app.jar\", \"spanner\" ]\n    ...\n```\n\nWith that, we configured the connection of JDBC to PGAdapter, by following this [guide](https://github.com/GoogleCloudPlatform/pgadapter/blob/postgresql-dialect/docs/jdbc.md).\n\nNext, we inspected the in memory representation of our query results, since some types were migrated in the Spanner schema. As a reminder, our Spanner schema used `jsonb`s instead of `json`s and `int8`s instead of `int4`s. When handling `json` values, we were using Java `String`s, so we didn’t need any modifications here as `jsonb` objects can be stored in `String`s. On the other hand, when handling `int4` values, we were storing them into memory using the primitive `int` type. This domain is smaller than the new `int8` column type, so we need to make sure we use primitive `long`s instead.\n\nFinally, we had to take a look at the queries we were issuing. The only problem identified was that we are performing a `CAST` from `text` to `json` when inserting the `song_data` column for a [Song](src/main/java/com/google/models/Song.java). Since we are using the `jsonb` type now, we need to modify the casting accordingly.\n\n```sql\n-- PostgreSQL\nINSERT INTO songs (singer_id, album_id, song_id, song_name, song_data)\nVALUES (?, ?, DEFAULT, ?, CAST(? AS JSON))\nRETURNING song_id;\n\n-- Updated (Spanner)\nINSERT INTO songs (singer_id, album_id, song_id, song_name, song_data)\nVALUES (?, ?, DEFAULT, ?, CAST(? AS JSONB))\nRETURNING song_id\n```\n\n## How to Run\n\n### Open-source PostgreSQL (Cloud SQL)\n\nFirst set up a Cloud SQL PostgreSQL [instance](https://cloud.google.com/sql/docs/postgres/create-instance), [database](https://cloud.google.com/sql/docs/postgres/create-manage-databases) and [user](https://cloud.google.com/sql/docs/postgres/create-manage-users).\n\nNext copy the `env.sample` into a `.env` file. This will be used by `docker-compose`. In the copied file configure the following information:\n\n```shell\nCLOUDSQL_INSTANCE_CONNECTION_NAME=\u003ccloudsql instance connection name\u003e\nCLOUDSQL_DATABASE=\u003cdatabase name\u003e\nCLOUDSQL_USERNAME=\u003cdatabase username\u003e\nCLOUDSQL_PASSWORD=\u003cdatabase password\u003e\n```\n\nYou can then start the application as follows:\n\n```shell\ndocker-compose up app-cloudsql\n```\n\nYou can stop the application like so:\n\n```shell\ndocker-compose down app-cloudsql\n```\n\n### Spanner PostgreSQL\n\nFirst set up a Spanner [instance](https://cloud.google.com/spanner/docs/create-query-database-console#create-instance) and [PostgreSQL database](https://cloud.google.com/spanner/docs/create-query-database-console#create-database) (don't forget to select the PostgreSQL database dialect).\n\nNext copy the `env.sample` into a `.env` file. This will be used by `docker-compose`. In the copied file configure the following information:\n\n```shell\nSPANNER_PROJECT=\u003cgcp project-id\u003e\nSPANNER_INSTANCE=\u003cspanner instance-id\u003e\nSPANNER_DATABASE=\u003cspanner database-id\u003e\n```\n\nYou can then start the application as follows:\n\n```shell\ndocker-compose up app-spanner\n```\nYou can stop the application like so:\n\n```shell\ndocker-compose down pgadapter app-spanner\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudspannerecosystem%2Fspanner-migration-example","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcloudspannerecosystem%2Fspanner-migration-example","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcloudspannerecosystem%2Fspanner-migration-example/lists"}