{"id":15068890,"url":"https://github.com/squidmin/java11-spring-gradle-bigquery-reference","last_synced_at":"2026-01-02T23:07:04.621Z","repository":{"id":157749615,"uuid":"623961759","full_name":"squidmin/java11-spring-gradle-bigquery-reference","owner":"squidmin","description":"Java v11 ⋅ Spring v2 ⋅ Gradle ⋅ BigQuery","archived":false,"fork":false,"pushed_at":"2023-08-18T02:27:37.000Z","size":214,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-01-22T19:45:54.950Z","etag":null,"topics":["bigquery","gradle","gradle-java","java","java-gradle","java11","java11-spring-boot","spring","spring-boot-2","spring-mvc","spring-rest"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/squidmin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-04-05T13:04:07.000Z","updated_at":"2023-08-31T00:13:29.000Z","dependencies_parsed_at":null,"dependency_job_id":"52581141-d56e-4b61-9a80-bf924037f964","html_url":"https://github.com/squidmin/java11-spring-gradle-bigquery-reference","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava11-spring-gradle-bigquery-reference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava11-spring-gradle-bigquery-reference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava11-spring-gradle-bigquery-reference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava11-spring-gradle-bigquery-reference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/squidmin","download_url":"https://codeload.github.com/squidmin/java11-spring-gradle-bigquery-reference/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243837015,"owners_count":20355814,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","gradle","gradle-java","java","java-gradle","java11","java11-spring-boot","spring","spring-boot-2","spring-mvc","spring-rest"],"created_at":"2024-09-25T01:39:38.122Z","updated_at":"2026-01-02T23:07:04.615Z","avatar_url":"https://github.com/squidmin.png","language":"Java","readme":"# java11-spring-gradle-bigquery-reference\n\n\n\u003cdetails\u003e\n\u003csummary\u003eAbout\u003c/summary\u003e\n\nMade with:\n- **Adoptium Temurin OpenJDK 11.0.17**\n- **Spring Boot v2.7.10**\n- **Gradle 7.6.1**\n- **IntelliJ IDEA 2023.1 (Ultimate Edition)**\n\n\u003c/details\u003e\n\n\n---\n\n\n## Install \u0026 build\n\n\u003cdetails\u003e\n\u003csummary\u003eDownload, install, and initialize the gcloud SDK on your local machine\u003c/summary\u003e\n\nRefer to the \u003ca href=\"https://cloud.google.com/sdk/docs/install#other_installation_options\"\u003e`gcloud` CLI documentation\u003c/a\u003e to complete this step.\n\nInstall the `gcloud` SDK to the user's home directory (e.g., `/Users/USERNAME/google-cloud-sdk`).\n\nWhen it's finished installing, add the `gcloud` executable to your system's `$PATH` and run the command:\n\n```shell\ngcloud init\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003egcloud CLI: Application Default Credentials (ADC) usage\u003c/summary\u003e\n\n```shell\ngcloud auth login\ngcloud auth application-default login\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003egcloud CLI: Generate an Application Default Credentials (ADC) access token\u003c/summary\u003e\n\nIf you're running the application locally, you can use the following command to generate an access token using Application Default Credentials (ADC):\n\n```shell\ngcloud auth application-default print-access-token\n```\n\n```shell\nexport GCP_ADC_ACCESS_TOKEN=\"$(gcloud auth application-default print-access-token)\"\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003egcloud CLI: Generate an access token for service account impersonation\u003c/summary\u003e\n\nRun this command to generate an access token for a specific GCP service account:\n\n```shell\nexport GCP_SA_ACCESS_TOKEN=$(gcloud auth print-access-token --impersonate-service-account='GCP_SA_EMAIL_ADDRESS')\n```\n\n**Replace the following**:\n- `GCP_SA_EMAIL_ADDRESS`: the email address of the service account to impersonate.\n\nExample:\n\n```shell\nexport GCP_SA_ACCESS_TOKEN=$(gcloud auth print-access-token --impersonate-service-account='sa-developer@your-sa-name.iam.gserviceaccount.com')\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eCreate and store a service account key\u003c/summary\u003e\n\nThis section refers to usage of a GCP service account key (.json) file stored on your local file system.\n\nTo map a local `gcloud` installation to a volume on a container instance running the application, include the `-v` parameter in the `docker run` command used to start a container instance, as described below.\n\n### macOS\n\nAssuming the user's service account key file is stored in the same directory as their local `gcloud` installation:\n\n`/Users/USERNAME/.config/gcloud`\n\n```shell\nexport LOCAL_GCLOUD_AUTH_DIRECTORY=$HOME/.config/gcloud\n```\n\nand the target volume on the container instance is:\n\n`/root/.config/gcloud`\n\n```shell\nexport CONTAINER_GCLOUD_AUTH_DIRECTORY=/root/.config/gcloud\n```\n\nthe command to run the container instance would be:\n\n```shell\ndocker run --rm -it \\\n  -e GCP_SA_KEY_PATH=$GCP_SA_KEY_PATH \\\n  -e GCP_ADC_ACCESS_TOKEN=$GCP_ADC_ACCESS_TOKEN \\\n  -e GCP_SA_ACCESS_TOKEN=$GCP_SA_ACCESS_TOKEN \\\n  -e GCP_DEFAULT_USER_PROJECT_ID=$GCP_DEFAULT_USER_PROJECT_ID \\\n  -e GCP_DEFAULT_USER_DATASET=$GCP_DEFAULT_USER_DATASET \\\n  -e GCP_DEFAULT_USER_TABLE=$GCP_DEFAULT_USER_TABLE \\\n  -e GCP_SA_PROJECT_ID=$GCP_SA_PROJECT_ID \\\n  -e GCP_SA_DATASET=$GCP_SA_DATASET \\\n  -e GCP_SA_TABLE=$GCP_SA_TABLE \\\n  -v ${LOCAL_GCLOUD_AUTH_DIRECTORY}:${CONTAINER_GCLOUD_AUTH_DIRECTORY} \\\n  -v ${LOCAL_MAVEN_REPOSITORY}:${CONTAINER_MAVEN_REPOSITORY} \\\n  java11-spring-gradle-bigquery-reference\n```\n\n**Replace the following** in the path to the `gcloud` directory:\n\n- `USERNAME`: the current OS user's username\n\nso that the path to the service account key file is correct, e.g.:\n\n`/Users/squidmin/.config/gcloud/sa-private-key.json`\n\nRead \u003ca href=\"https://cloud.google.com/iam/docs/keys-create-delete#iam-service-account-keys-create-gcloud\"\u003ehere\u003c/a\u003e for more information about creating service account keys.\n\nRead \u003ca href=\"\"\u003ehere\u003c/a\u003e for more information about run config CLI arguments.\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eActivate GCP service account\u003c/summary\u003e\n\n```shell\ngcloud auth activate-service-account --key-file=GCP_SA_KEY_FILE\n```\n\n**Replace the following**:\n- `GCP_SA_KEY_FILE`: path to the user's service account key file.\n\nExample:\n\n```shell\ngcloud auth activate-service-account --key-file='/Users/squidmin/.config/gcloud/sa-private-key.json'\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eSet the active GCP project\u003c/summary\u003e\n\n```shell\ngcloud config set project ${GCP_DEFAULT_USER_PROJECT_ID}\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eList available gcloud SDK components\u003c/summary\u003e\n\n```shell\ngcloud components list\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eUpdate gcloud SDK components\u003c/summary\u003e\n\n```shell\ngcloud components update\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eCLI reference table: Run configuration\u003c/summary\u003e\n\n\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eBuild JAR\u003c/summary\u003e\n\n```shell\n./gradlew clean build\n```\n\n```shell\n./gradlew clean build -x test\n```\n\n```shell\n./gradlew clean build testClasses -x test\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eAdd manifest file\u003c/summary\u003e\n\n```shell\njar -cmvf \\\n  ./build/tmp/jar/MANIFEST.MF \\\n  ./build/libs/java11-spring-gradle-bigquery-reference-0.0.1-SNAPSHOT.jar \\\n  ./build/classes/java/main/org/squidmin/java/spring/gradle/bigquery/JavaSpringGradleBigQueryReferenceApplication.class\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eBuild container image\u003c/summary\u003e\n\n```shell\ndocker build \\\n  --build-arg GCP_SA_KEY_PATH=${GCP_SA_KEY_PATH} \\\n  --build-arg GCP_DEFAULT_USER_PROJECT_ID=${GCP_DEFAULT_USER_PROJECT_ID} \\\n  --build-arg GCP_DEFAULT_USER_DATASET=${GCP_DEFAULT_USER_DATASET} \\\n  --build-arg GCP_DEFAULT_USER_TABLE=${GCP_DEFAULT_USER_TABLE} \\\n  --build-arg GCP_SA_PROJECT_ID=${GCP_SA_PROJECT_ID} \\\n  --build-arg GCP_SA_DATASET=${GCP_SA_DATASET} \\\n  --build-arg GCP_SA_TABLE=${GCP_SA_TABLE} \\\n  -t java11-spring-gradle-bigquery-reference .\n```\n\n\u003c/details\u003e\n\n\n---\n\n\n## Commands\n\n\n### Java application\n\n\u003cdetails\u003e\n\u003csummary\u003eRun container\u003c/summary\u003e\n\n```shell\ndocker run --rm -it \\\n  -e GCP_SA_KEY_PATH=$GCP_SA_KEY_PATH \\\n  -e GCP_ADC_ACCESS_TOKEN=$GCP_ADC_ACCESS_TOKEN \\\n  -e GCP_SA_ACCESS_TOKEN=$GCP_SA_ACCESS_TOKEN \\\n  -e GCP_DEFAULT_USER_PROJECT_ID=$GCP_DEFAULT_USER_PROJECT_ID \\\n  -e GCP_DEFAULT_USER_DATASET=$GCP_DEFAULT_USER_DATASET \\\n  -e GCP_DEFAULT_USER_TABLE=$GCP_DEFAULT_USER_TABLE \\\n  -e GCP_SA_PROJECT_ID=$GCP_SA_PROJECT_ID \\\n  -e GCP_SA_DATASET=$GCP_SA_DATASET \\\n  -e GCP_SA_TABLE=$GCP_SA_TABLE \\\n  -v ${LOCAL_GCLOUD_AUTH_DIRECTORY}:${CONTAINER_GCLOUD_AUTH_DIRECTORY} \\\n  -v ${LOCAL_MAVEN_REPOSITORY}:${CONTAINER_MAVEN_REPOSITORY} \\\n  java11-spring-gradle-bigquery-reference\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eRun jar\u003c/summary\u003e\n\n```shell\nexec java -jar \\\n  -Dspring.profiles.active=local \\\n  ./build/libs/java11-spring-gradle-bigquery-reference-0.0.1-SNAPSHOT.jar\n```\n\n\u003c/details\u003e\n\n\n### `bq` CLI\n\n\u003cdetails\u003e\n\u003csummary\u003eList datasets\u003c/summary\u003e\n\n```shell\nbq ls --filter labels.key:value \\\n  --max_results integer \\\n  --format=prettyjson \\\n  --project_id project_id\n```\n\n**Replace the following**:\n- `key:value`: a label key and value, if applicable.\n- `integer`: an integer representing the number of datasets to list.\n- `project_id`: the name of the GCP project to target.\n\n**Examples**:\n\n```shell\nbq ls --format=pretty\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eCreate a dataset\u003c/summary\u003e\n\nRefer to the \u003ca href=\"https://cloud.google.com/bigquery/docs/datasets#create-dataset\"\u003eGCP documentation for creating datasets\u003c/a\u003e.\n\n**Examples**:\n\n```shell\nbq --location=us mk \\\n  --dataset \\\n  --default_partition_expiration=3600 \\\n  --default_table_expiration=3600 \\\n  --description=\"An example.\" \\\n  --label=test_label_1:test_value_1 \\\n  --label=test_label_2:test_value_2 \\\n  --max_time_travel_hours=168 \\\n  --storage_billing_model=LOGICAL \\\n  lofty-root-378503:test_dataset_integration\n```\n\nThe Cloud Key Management Service (KMS) key parameter (`KMS_KEY_NAME`) can be specified.\nThis parameter is used to pass the name of the default Cloud Key Management Service key used to protect newly created tables in this dataset.\nYou cannot create a Google-encrypted table in a dataset with this parameter set.\n\n```shell\nbq --location=us mk \\\n  --dataset \\\n  --default_kms_key=KMS_KEY_NAME \\\n  ...\n  lofty-root-378503:test_dataset_integration\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eDelete a dataset\u003c/summary\u003e\n\nRefer to the \u003ca href=\"https://cloud.google.com/bigquery/docs/managing-datasets#delete_a_dataset\"\u003eGCP documentation for deleting a dataset\u003c/a\u003e.\n\n#### Examples:\n\nRemove all tables in the dataset (`-r` flag):\n\n```shell\nbq rm -r -f -d lofty-root-378503:test_dataset_integration\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eCreate a table with a configured schema\u003c/summary\u003e\n\n**Create an empty table with an inline schema definition**\n\n```shell\nbq mk --table project_id:dataset.table schema\n```\n\n**Replace the following**:\n- `project_id`: the name of the GCP project to target.\n- `dataset`: the name of the BigQuery dataset to target.\n- `table`: the name of the BigQuery table to target.\n- `schema`: an inline schema definition.\n\nExample:\n\n```shell\nbq mk --table \\\n  lofty-root-378503:test_dataset_integration.test_table_integration \\\n  id:STRING,creation_timestamp:DATETIME,last_update_timestamp:DATETIME,column_a:STRING,column_b:BOOL\n```\n\n### Specify the schema in a JSON schema file\n\nFor an example JSON schema file, refer to: `/schema/example.json`.\n\n**Create an empty table**\n\n```shell\nbq mk --table \\\n  project_id:dataset.table \\\n  path_to_schema_file\n```\n\nExample:\n\n```shell\nbq mk --table \\\n  lofty-root-378503:test_dataset_integration.test_table_integration \\\n  ./schema/example.json\n```\n\n**Create a table with CSV data**\n\n```shell\nbq --location=location load \\\n  --source_format=format \\\n  project_id:dataset.table \\\n  path_to_data_file \\\n  path_to_schema_file\n```\n\nExample:\n\n```shell\nbq --location=us load \\\n  --source_format=CSV \\\n  lofty-root-378503:test_dataset_integration.test_table_integration \\\n  ./csv/example.csv \\\n  ./schema/example.json\n```\n\nRefer to the BigQuery documentation: \u003ca href=\"https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv#details_of_loading_csv_data\"\u003eDetails of loading CSV data\u003c/a\u003e.\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eDelete a table\u003c/summary\u003e\n\n```shell\nbq rm --table test_dataset_integration.test_table_integration\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eShow table schema\u003c/summary\u003e\n\nExample:\n\n```shell\nbq show \\\n  --schema \\\n  --format=prettyjson \\\n  lofty-root-378503:test_dataset_integration.test_table_integration\n```\n\nThe table schema can be written to a file:\n\n```shell\nbq show \\\n  --schema \\\n  --format=prettyjson \\\n  lofty-root-378503:test_dataset_integration.test_table_integration \\ \u003e ./schema/example_show-write.json\n```\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eModify table schemas\u003c/summary\u003e\n\n```shell\nbq update \\\n  lofty-root-378503:test_dataset_integration.test_table_integration \\\n  ./schema/example_update.json\n```\n\nRefer to the \u003ca href=\"https://cloud.google.com/bigquery/docs/managing-table-schemas\"\u003eGCP documentation on modifying table schemas.\u003c/a\u003e.\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eInsert data into a table\u003c/summary\u003e\n\n**Examples**:\n\nInsert for known values:\n\n```shell\nbq insert test_dataset_integration.test_table_integration ./json/example.json\n```\n\nSpecify a template suffix (`--template_suffix` or `-x`):\n\n```shell\nbq insert --ignore_unknown_values \\\n  --template_suffix=_insert \\\n  test_dataset_integration.test_table_integration \\\n  ./json/example.json\n```\n\nRefer to the \u003ca href=\"\"\u003e`bq insert` documentation\u003c/a\u003e.\n\n\u003c/details\u003e\n\n\n\u003cdetails\u003e\n\u003csummary\u003eRun an interactive query\u003c/summary\u003e\n\n```shell\nbq query \\\n  --use_legacy_sql=false \\\n  'query_string'\n```\n\nExample:\n\n```shell\nbq query \\\n  --use_legacy_sql=false \\\n  'SELECT\n    id, fieldC\n  FROM\n    `lofty-root-378503.test_dataset_integration.test_table_integration`\n  LIMIT\n    3;'\n```\n\n\u003c/details\u003e\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsquidmin%2Fjava11-spring-gradle-bigquery-reference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsquidmin%2Fjava11-spring-gradle-bigquery-reference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsquidmin%2Fjava11-spring-gradle-bigquery-reference/lists"}