{"id":15866364,"url":"https://github.com/squidmin/java17-spring-gradle-bigquery-reference","last_synced_at":"2025-08-01T22:43:15.939Z","repository":{"id":189045574,"uuid":"679917337","full_name":"squidmin/java17-spring-gradle-bigquery-reference","owner":"squidmin","description":"Java v17⋅ Spring v3 ⋅ Gradle ⋅ BigQuery","archived":false,"fork":false,"pushed_at":"2024-02-13T23:29:09.000Z","size":334,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-06-17T13:52:40.337Z","etag":null,"topics":["bigquery","gradle","java","java-17-gradle","java17","java17-spring-boot","spring-boot-3"],"latest_commit_sha":null,"homepage":"","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/squidmin.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-08-17T23:22:28.000Z","updated_at":"2024-01-30T15:44:14.000Z","dependencies_parsed_at":"2023-08-18T03:35:21.372Z","dependency_job_id":"a962ab63-87cb-48b4-9ab4-c44a13f2697d","html_url":"https://github.com/squidmin/java17-spring-gradle-bigquery-reference","commit_stats":null,"previous_names":["squidmin/java17-spring-gradle-bigquery-reference"],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/squidmin/java17-spring-gradle-bigquery-reference","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava17-spring-gradle-bigquery-reference","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava17-spring-gradle-bigquery-reference/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava17-spring-gradle-bigquery-reference/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava17-spring-gradle-bigquery-reference/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/squidmin","download_url":"https://codeload.github.com/squidmin/java17-spring-gradle-bigquery-reference/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/squidmin%2Fjava17-spring-gradle-bigquery-reference/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":268308972,"owners_count":24230163,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-01T02:00:08.611Z","response_time":67,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","gradle","java","java-17-gradle","java17","java17-spring-boot","spring-boot-3"],"created_at":"2024-10-05T23:20:28.022Z","updated_at":"2025-08-01T22:43:15.817Z","avatar_url":"https://github.com/squidmin.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# java17-spring-gradle-bigquery-reference\n\n[![Java CI with Gradle](https://github.com/squidmin/java17-spring-gradle-bigquery-reference/actions/workflows/spring-boot-ci.yml/badge.svg)](https://github.com/squidmin/java17-spring-gradle-bigquery-reference/actions/workflows/spring-boot-ci.yml)\n\n\u003cdetails\u003e\n\u003csummary\u003eAbout\u003c/summary\u003e\n\nMade with:\n- **Adoptium Temurin OpenJDK 17.0.8**\n- **Spring Boot v3.1.2**\n- **Gradle 8.2.1**\n- **IntelliJ IDEA 2023.1 (Ultimate Edition)**\n\n\u003c/details\u003e\n\n---\n\n## Install \u0026 build\n\n\u003cdetails\u003e\n\u003csummary\u003eDownload, install, and initialize the gcloud SDK on your local machine\u003c/summary\u003e\n\nRefer to the \u003ca href=\"https://cloud.google.com/sdk/docs/install#other_installation_options\"\u003e`gcloud` CLI documentation\u003c/a\u003e to complete this step.\n\nInstall the `gcloud` SDK to the user's home directory (e.g., `/Users/USERNAME/google-cloud-sdk`).\n\nWhen it's finished installing, add the `gcloud` executable to your system's `$PATH` and run the command:\n\n```shell\ngcloud init\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003egcloud CLI: Application Default Credentials (ADC) usage\u003c/summary\u003e\n\n```shell\ngcloud auth login\ngcloud auth application-default login\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003egcloud CLI: Generate an Application Default Credentials (ADC) access token\u003c/summary\u003e\n\nIf you're running the application locally, you can use the following command to generate an access token using Application Default Credentials (ADC):\n\n```shell\ngcloud auth application-default print-access-token\n```\n\n```shell\nexport GCP_ACCESS_TOKEN=\"$(gcloud auth application-default print-access-token)\"\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003egcloud CLI: Generate an access token for service account impersonation\u003c/summary\u003e\n\nRun this command to generate an access token for a specific GCP service account:\n\n```shell\nexport GCP_ACCESS_TOKEN=$(gcloud auth print-access-token --impersonate-service-account='GCP_SA_EMAIL_ADDRESS')\n```\n\n**Replace the following**:\n- `GCP_SA_EMAIL_ADDRESS`: the email address of the service account to impersonate.\n\nExample:\n\n```shell\nexport GCP_ACCESS_TOKEN=$(gcloud auth print-access-token --impersonate-service-account='gcp-tekton-sa@lift-with-your-legs-123456.iam.gserviceaccount.com')\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCreate and store a service account key\u003c/summary\u003e\n\nThis section refers to usage of a GCP service account key (.json) file stored on your local file system.\n\nTo map a local `gcloud` installation to a volume on a container instance running the application, include the `-v` parameter in the `docker run` command used to start a container instance, as described below.\n\n### macOS\n\nAssuming the user's service account key file is stored in the same directory as their local `gcloud` installation:\n\n`/Users/USERNAME/.config/gcloud`\n\n```shell\nexport LOCAL_GCLOUD_AUTH_DIRECTORY=$HOME/.config/gcloud\n```\n\nand the target volume on the container instance is:\n\n`/root/.config/gcloud`\n\n```shell\nexport CONTAINER_GCLOUD_AUTH_DIRECTORY=/root/.config/gcloud\n```\n\nthe command to run the container instance would be:\n\n```shell\ndocker run --rm -it \\\n  -e GCP_SA_KEY_PATH=${GCP_SA_KEY_PATH} \\\n  -e GCP_ACCESS_TOKEN=${GCP_ACCESS_TOKEN} \\\n  -e GCP_PROJECT_ID=${GCP_PROJECT_ID} \\\n  -e BQ_DATASET=${BQ_DATASET} \\\n  -e BQ_TABLE=${BQ_TABLE} \\\n  -v ${LOCAL_GCLOUD_AUTH_DIRECTORY}:${CONTAINER_GCLOUD_AUTH_DIRECTORY} \\\n  -v ${LOCAL_MAVEN_REPOSITORY}:${CONTAINER_MAVEN_REPOSITORY} \\\n  java17-spring-gradle-bigquery-reference\n```\n\n**Replace the following** in the path to the `gcloud` directory:\n\n- `USERNAME`: the current OS user's username\n\nso that the path to the service account key file is correct, e.g.:\n\n`/Users/squidmin/.config/gcloud/sa-private-key.json`\n\nRead \u003ca href=\"https://cloud.google.com/iam/docs/keys-create-delete#iam-service-account-keys-create-gcloud\"\u003ehere\u003c/a\u003e for more information about creating service account keys.\n\nRead \u003ca href=\"\"\u003ehere\u003c/a\u003e for more information about run config CLI arguments.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eActivate GCP service account\u003c/summary\u003e\n\n```shell\ngcloud auth activate-service-account --key-file=${GCP_SA_KEY_PATH}\n```\n\n**Replace the following**:\n- `GCP_SA_KEY_PATH`: path to the user's service account key file.\n\nExample:\n\n```shell\ngcloud auth activate-service-account --key-file='/Users/squidmin/.config/gcloud/sa-private-key.json'\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eSet the active GCP project\u003c/summary\u003e\n\n```shell\ngcloud config set project ${GCP_PROJECT_ID}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eList available gcloud SDK components\u003c/summary\u003e\n\n```shell\ngcloud components list\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eUpdate gcloud SDK components\u003c/summary\u003e\n\n```shell\ngcloud components update\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCLI reference table: Run configuration\u003c/summary\u003e\n\nTODO\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eBuild JAR\u003c/summary\u003e\n\n```shell\n./gradlew clean build\n```\n\n```shell\n./gradlew clean build -x test\n```\n\n```shell\n./gradlew clean build testClasses -x test\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eAdd manifest file\u003c/summary\u003e\n\n```shell\njar -cmvf \\\n  ./build/tmp/jar/MANIFEST.MF \\\n  ./build/libs/java17-spring-gradle-bigquery-reference-${REVISION}.jar \\\n  ./build/classes/java/main/org/squidmin/java/spring/gradle/bigquery/Java17SpringGradleBigQueryReferenceApplication.class\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eBuild container image\u003c/summary\u003e\n\n```shell\ndocker build \\\n  --build-arg GCP_SA_KEY_PATH=${GCP_SA_KEY_PATH} \\\n  --build-arg GCP_PROJECT_ID=${GCP_PROJECT_ID} \\\n  --build-arg BQ_DATASET=${BQ_DATASET} \\\n  --build-arg BQ_TABLE=${BQ_TABLE}\n  -t java17-spring-gradle-bigquery-reference .\n```\n\n\u003c/details\u003e\n\n---\n\n## Commands\n\n### Java application\n\n\u003cdetails\u003e\n\u003csummary\u003eRun container\u003c/summary\u003e\n\n```shell\ndocker run --rm -it \\\n  -e GCP_SA_KEY_PATH=${GCP_SA_KEY_PATH} \\\n  -e GCP_ACCESS_TOKEN=${GCP_ACCESS_TOKEN} \\\n  -e GCP_PROJECT_ID=${GCP_PROJECT_ID} \\\n  -e BQ_DATASET=${BQ_DATASET} \\\n  -e BQ_TABLE=${BQ_TABLE} \\\n  -v ${LOCAL_GCLOUD_AUTH_DIRECTORY}:${CONTAINER_GCLOUD_AUTH_DIRECTORY} \\\n  -v ${LOCAL_MAVEN_REPOSITORY}:${CONTAINER_MAVEN_REPOSITORY} \\\n  java17-spring-gradle-bigquery-reference\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRun jar\u003c/summary\u003e\n\n```shell\nexec java -jar \\\n  -Dspring.profiles.active=local \\\n  ./build/libs/java17-spring-gradle-bigquery-reference-${REVISION}.jar\n```\n\n\u003e **Why use `exec` with Java applications?**\n\u003e \n\u003e **In Docker containers**: Commonly seen in Dockerfiles for Java applications.\n\u003e It ensures that the Java application receives Unix signals directly because it's running as the container's PID 1 process.\n\u003e This is important for graceful shutdown and handling other system signals.\n\u003e \n\u003e **In scripts**: To ensure that the Java application is the only running process and to handle signals and exit codes directly, without a shell in between.\n\u003e \n\u003e In summary, `exec java -jar` os a powerful combination used to execute Java applications packaged as JAR files directly as the main process, ensuring that they handle system signals directly and that their lifecycle is tightly coupled with the lifecycle of the shell or container they're running in.\n\n\u003c/details\u003e\n\n### `bq` CLI\n\n\u003cdetails\u003e\n\u003csummary\u003eList datasets\u003c/summary\u003e\n\n```shell\nbq ls --filter labels.key:value \\\n  --max_results ${MAX_RESULTS} \\\n  --format=prettyjson \\\n  --project_id ${GCP_PROJECT_ID}\n```\n\n**Replace the following**:\n- `key:value`: a label key and value, if applicable.\n- `MAX_RESULTS`: an integer representing the number of datasets to list.\n- `GCP_PROJECT_ID`: the name of the GCP project to target.\n\n**Examples**:\n\n```shell\nbq ls --format=pretty\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCreate a dataset\u003c/summary\u003e\n\nRefer to the \u003ca href=\"https://cloud.google.com/bigquery/docs/datasets#create-dataset\"\u003eGCP documentation for creating datasets\u003c/a\u003e.\n\n**Examples**:\n\n```shell\nbq --location=us mk \\\n  --dataset \\\n  --default_partition_expiration=3600 \\\n  --default_table_expiration=3600 \\\n  --description=\"An example.\" \\\n  --label=test_label_1:test_value_1 \\\n  --label=test_label_2:test_value_2 \\\n  --max_time_travel_hours=168 \\\n  --storage_billing_model=LOGICAL \\\n  ${GCP_PROJECT_ID}:${BQ_DATASET}\n```\n\nThe Cloud Key Management Service (KMS) key parameter (`KMS_KEY_NAME`) can be specified.\nThis parameter is used to pass the name of the default Cloud Key Management Service key used to protect newly created tables in this dataset.\nYou cannot create a Google-encrypted table in a dataset with this parameter set.\n\n```shell\nbq --location=us mk \\\n  --dataset \\\n  --default_kms_key=${KMS_KEY_NAME} \\\n  ... \\\n  ${GCP_PROJECT_ID}:${BQ_DATASET}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eDelete a dataset\u003c/summary\u003e\n\nRefer to the \u003ca href=\"https://cloud.google.com/bigquery/docs/managing-datasets#delete_a_dataset\"\u003eGCP documentation for deleting a dataset\u003c/a\u003e.\n\n#### Examples:\n\nRemove all tables in the dataset (`-r` flag):\n\n```shell\nbq rm -r -f -d ${GCP_PROJECT_ID}:${BQ_TABLE}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eCreate a table with a configured schema\u003c/summary\u003e\n\n**Create an empty table with an inline schema definition**\n\n```shell\nbq mk --table ${GCP_PROJECT_ID}:${BQ_DATASET}.${BQ_TABLE} ${SCHEMA}\n```\n\n**Replace the following**:\n- `GCP_PROJECT_ID`: the name of the GCP project to target.\n- `BQ_DATASET`: the name of the BigQuery dataset to target.\n- `BQ_TABLE`: the name of the BigQuery table to target.\n- `SCHEMA`: an inline schema definition.\n\nExample:\n\n```shell\nbq mk --table \\\n  example-project-id:test_dataset_integration.test_table_integration \\\n  id:STRING,creation_timestamp:DATETIME,last_update_timestamp:DATETIME,column_a:STRING,column_b:BOOL\n```\n\n### Specify the schema in a JSON schema file\n\nFor an example JSON schema file, refer to: `/schema/example.json`.\n\n**Create an empty table**\n\n```shell\nbq mk --table \\\n  ${GCP_PROJECT_ID}:${BQ_DATASET}.${BQ_TABLE} \\\n  path_to_schema_file\n```\n\nExample:\n\n```shell\nbq mk --table \\\n  example-project-id:test_dataset_integration.test_table_integration \\\n  ./schema/example.json\n```\n\n**Create a table with CSV data**\n\n```shell\nbq --location=location load \\\n  --source_format=${FORMAT} \\\n  ${GCP_PROJECT_ID}:${BQ_DATASET}.${BQ_TABLE} \\\n  path_to_data_file \\\n  path_to_schema_file\n```\n\nExample:\n\n```shell\nbq --location=us load \\\n  --source_format=CSV \\\n  example-project-id:test_dataset_integration.test_table_integration \\\n  ./csv/example.csv \\\n  ./schema/example.json\n```\n\nRefer to the BigQuery documentation: \u003ca href=\"https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv#details_of_loading_csv_data\"\u003eDetails of loading CSV data\u003c/a\u003e.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eDelete a table\u003c/summary\u003e\n\n```shell\nbq rm --table ${BQ_DATASET}.${BQ_TABLE}\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eShow table schema\u003c/summary\u003e\n\nExample:\n\n```shell\nbq show \\\n  --schema \\\n  --format=prettyjson \\\n  example-project-id:test_dataset_integration.test_table_integration\n```\n\nThe table schema can be written to a file:\n\n```shell\nbq show \\\n  --schema \\\n  --format=prettyjson \\\n  example-project-id:test_dataset_integration.test_table_integration \\ \u003e ./schema/example_show-write.json\n```\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eModify table schemas\u003c/summary\u003e\n\n```shell\nbq update \\\n  ${GCP_PROJECT_ID}:test_dataset_integration.test_table_integration \\\n  ./schema/example_update.json\n```\n\nRefer to the \u003ca href=\"https://cloud.google.com/bigquery/docs/managing-table-schemas\"\u003eGCP documentation on modifying table schemas.\u003c/a\u003e.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eInsert data into a table\u003c/summary\u003e\n\n**Examples**:\n\nInsert for known values:\n\n```shell\nbq insert test_dataset_integration.test_table_integration ./json/example.json\n```\n\nSpecify a template suffix (`--template_suffix` or `-x`):\n\n```shell\nbq insert --ignore_unknown_values \\\n  --template_suffix=_insert \\\n  test_dataset_integration.test_table_integration \\\n  ./json/example.json\n```\n\nRefer to the \u003ca href=\"https://cloud.google.com/bigquery/docs/reference/bq-cli-reference#bq_insert\"\u003e`bq insert` documentation\u003c/a\u003e.\n\n\u003c/details\u003e\n\n\u003cdetails\u003e\n\u003csummary\u003eRun an interactive query\u003c/summary\u003e\n\n```shell\nbq query \\\n  --use_legacy_sql=false \\\n  'query_string'\n```\n\nExample:\n\n```shell\nbq query \\\n  --use_legacy_sql=false \\\n  'SELECT\n    id, fieldC\n  FROM\n    `example-project-id.test_dataset_integration.test_table_integration`\n  LIMIT\n    3;'\n```\n\n\u003c/details\u003e\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsquidmin%2Fjava17-spring-gradle-bigquery-reference","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fsquidmin%2Fjava17-spring-gradle-bigquery-reference","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fsquidmin%2Fjava17-spring-gradle-bigquery-reference/lists"}