{"id":15221816,"url":"https://github.com/googlecloudplatform/bigquery-dlp-remote-function","last_synced_at":"2025-10-20T00:33:04.477Z","repository":{"id":56819411,"uuid":"525992926","full_name":"GoogleCloudPlatform/bigquery-dlp-remote-function","owner":"GoogleCloudPlatform","description":"Use Remote Functions to tokenize data with DLP in BigQuery using SQL","archived":false,"fork":false,"pushed_at":"2025-04-02T15:17:23.000Z","size":335,"stargazers_count":22,"open_issues_count":4,"forks_count":6,"subscribers_count":45,"default_branch":"main","last_synced_at":"2025-04-19T22:18:05.000Z","etag":null,"topics":["bigquery","cloud-run","data-loss-prevention","dlp","google-cloud"],"latest_commit_sha":null,"homepage":"https://cloud.google.com/dlp/docs/deidentify-bq-tutorial","language":"Java","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GoogleCloudPlatform.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null}},"created_at":"2022-08-18T00:00:46.000Z","updated_at":"2025-04-04T12:56:52.000Z","dependencies_parsed_at":"2022-09-21T11:10:46.267Z","dependency_job_id":"39a4bd9f-88d8-47d8-8fa2-5a8cfa456cc3","html_url":"https://github.com/GoogleCloudPlatform/bigquery-dlp-remote-function","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/GoogleCloudPlatform/bigquery-dlp-remote-function","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fbigquery-dlp-remote-function","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fbigquery-dlp-remote-function/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fbigquery-dlp-remote-function/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fbigquery-dlp-remote-function/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GoogleCloudPlatform","download_url":"https://codeload.github.com/GoogleCloudPlatform/bigquery-dlp-remote-function/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GoogleCloudPlatform%2Fbigquery-dlp-remote-function/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":279955692,"owners_count":26250547,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-10-19T02:00:07.647Z","response_time":64,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bigquery","cloud-run","data-loss-prevention","dlp","google-cloud"],"created_at":"2024-09-28T15:07:48.575Z","updated_at":"2025-10-20T00:33:04.146Z","avatar_url":"https://github.com/GoogleCloudPlatform.png","language":"Java","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Use custom encryption in BigQuery using Remote Function, Cloud Run and Data Loss Prevention\n\n*Summary: Learn how to use Remote Functions to de/re-identify data with DLP in BigQuery using SQL.*\n\n[![Build and Validate](https://github.com/GoogleCloudPlatform/bigquery-dlp-remote-function/actions/workflows/main.yml/badge.svg)](https://github.com/GoogleCloudPlatform/bigquery-dlp-remote-function/actions/workflows/main.yml)\n[![codecov](https://codecov.io/gh/GoogleCloudPlatform/bigquery-dlp-remote-function/branch/main/graph/badge.svg?token=B25A0dD36P)](https://codecov.io/gh/GoogleCloudPlatform/bigquery-dlp-remote-function)\n\nThis document discusses how to detect and de-identify sensitive data like personally identifiable information (PII) in\nBigQuery tables with simple SQL based functions, using Cloud Data Loss Prevention\n[(Cloud DLP)](https://cloud.google.com/dlp). De-identification techniques like encryption lets you preserve the utility\nof your data for joining or analytics while reducing the risk of handling the data by de-identifying/obfuscating the raw\nsensitive identifiers.\n\nEnterprises may have policy or regulatory requirements to only store de-identified data in their DataWarehouses, and a need\nto quickly re-identify the de-identify data for reports generation. To minimize the risk of handling large volumes of\nsensitive data, you can use an\n[automated data transformation pipeline](https://github.com/GoogleCloudPlatform/auto-data-tokenize) to create\nde-identified datasets that can be used for migrating from on-premises to cloud. You can use this project to either\nreplace that pipeline with a SQL query for de-identification and quick re-identification or only for re-identification.\n\nCloud DLP can inspect the data for sensitive information when the dataset has not been characterized, by using\n[more than 100 built-in classifiers](https://cloud.google.com/dlp/docs/infotypes-reference). Using DLP to de-identify the\ndata requires complex data pipelines/applications. This solution aims to help your data analysts/engineers/scientists\nto achieve the same outcome through Simple SQL functions using\nBigQuery [Remote Functions](https://cloud.google.com/blog/products/data-analytics/extending-bigquery-functions) which\nare a powerful way to extend the functionality of BigQuery.\n\nThis document demonstrates a reference implementation of de-identifying structured data in\n[BigQuery](https://cloud.google.com/bigquery) using DLP and remote function\n(hosted on [Cloud Run](https://cloud.google.com/run)).\n\nThis document is intended for audience whose responsibilities include data security, data processing, or data\nanalytics. This guide assumes that you're familiar with data processing and data privacy, without the need to be an\nexpert and run simple shell and SQL scripts.\n\n## Objectives\n\n* Deploy Cloud Run service providing DLP based encryption functionality\n* Create BigQuery Remote functions that use DLP de-identify templates\n* Verify data encryption in BigQuery using a SQL query\n\n## Costs\n\nThis tutorial uses billable components of Google Cloud, including the following:\n\n* [BigQuery](https://cloud.google.com/bigquery/pricing)\n* [Cloud Build](https://cloud.google.com/build/pricing)\n* [Cloud Run](https://cloud.google.com/run/pricing)\n* [Cloud Secret Manager](https://cloud.google.com/secret-manager/pricing)\n* [Data Loss Prevention](https://cloud.google.com/dlp/pricing)\n\nUse the [pricing calculator](https://cloud.google.com/products/calculator) to generate a cost estimate based on your\nprojected usage.\n\n## Architecture\n\n![bigquery external tokenization functions](ext_bq_fn_dlp.svg)\n\n## Before you begin\n\nFor this tutorial, you need a Google Cloud [project](https://cloud.google.com/resource-manager/docs/cloud-platform-resource-hierarchy#projects). To make\ncleanup easiest at the end of the tutorial, we recommend that you create a new project for this tutorial.\n\n1.  [Create a Google Cloud project](https://console.cloud.google.com/projectselector2/home/dashboard).\n1.  Make sure that [billing is enabled](https://support.google.com/cloud/answer/6293499#enable-billing) for your Google\n    Cloud project.\n1.  [Open Cloud Shell](https://console.cloud.google.com/?cloudshell=true).\n\n    At the bottom of the Cloud Console, a [Cloud Shell](https://cloud.google.com/shell/docs/features) session opens and\n    displays a command-line prompt. Cloud Shell is a shell environment with the Cloud SDK already installed, including\n    the [gcloud](https://cloud.google.com/sdk/gcloud/) command-line tool, and with values already set for your current\n    project. It can take a few seconds for the session to initialize.\n\n1.  In Cloud Shell, clone the source repository and go to the directory for this tutorial:\n\n        git clone https://github.com/GoogleCloudPlatform/bigquery-dlp-remote-function.git\n        cd bigquery-dlp-remote-function/\n\n2.  Enable all the required Google Cloud APIs\n\n    ```shell\n    gcloud services enable \\\n    artifactregistry.googleapis.com \\\n    bigquery.googleapis.com \\\n    bigqueryconnection.googleapis.com \\\n    cloudbuild.googleapis.com \\\n    cloudkms.googleapis.com \\\n    containerregistry.googleapis.com \\\n    dlp.googleapis.com \\\n    run.googleapis.com \\\n    secretmanager.googleapis.com\n    ```\n\n## Deployment script\n\n1.  Authenticate using User [Application Default Credentials (\"ADCs\")](https://cloud.google.com/sdk/gcloud/reference/auth/application-default) as a primary authentication method.\n    ```shell\n    gcloud auth application-default login\n    ```\n\n2.  Initialize and run the Terraform script to create all resources:\n\n    ```shell\n    terraform init \u0026\u0026 \\\n    terraform apply\n    ```\n\n3.  Once the script successfully completes resources creation,\n    visit [BigQuery Console](https://console.cloud.google.com/bigquery)\n    to run the test SQL script\n\n    ```sql\n    SELECT\n    pii_column,\n    fns.dlp_freetext_encrypt(pii_column) AS dlp_encrypted,\n    fns.dlp_freetext_decrypt(fns.dlp_freetext_encrypt(pii_column)) AS dlp_decrypted\n    FROM\n     UNNEST(\n       [\n           'My name is John Doe. My email is john@doe.com',\n           'Some non PII data',\n           '212-233-4532',\n           'some script with simple number 1234']) AS pii_column\n      ```\n\n      The query will produce a result similar to following table:\n\n      \u003ctable\u003e\n          \u003cthead\u003e\n              \u003cth\u003eRow\u003c/th\u003e\n              \u003cth\u003epii_column\u003c/th\u003e\n              \u003cth\u003edlp_encrypted\u003c/th\u003e\n              \u003cth\u003edlp_decrypted\u003c/th\u003e\n          \u003c/thead\u003e\n          \u003ctbody\u003e\n              \u003ctr\u003e\n                  \u003ctd\u003e1\u003c/td\u003e\n                  \u003ctd\u003eMy name is John Doe. My email is john@doe.com\u003c/td\u003e\n                  \u003ctd\u003eMy name is John Doe. My email is BQ_TRF_EMAIL(40):AQy6lGvwKR+AiiRqJpEr+nBzZUzOcjXkXamUugU=\u003c/td\u003e\n                  \u003ctd\u003eMy name is John Doe. My email is john@doe.com\u003c/td\u003e\n              \u003c/tr\u003e\n              \u003ctr\u003e\n                  \u003ctd\u003e2\u003c/td\u003e\n                  \u003ctd\u003eSome non PII data\u003c/td\u003e\n                  \u003ctd\u003eSome non PII data\u003c/td\u003e\n                  \u003ctd\u003eSome non PII data\u003c/td\u003e\n              \u003c/tr\u003e\n              \u003ctr\u003e\n                  \u003ctd\u003e3\u003c/td\u003e\n                  \u003ctd\u003e212-233-4532\u003c/td\u003e\n                  \u003ctd\u003eBQ_TRF_PH(40):AbovCRcD0RwhqTJh1SuQmEfXERQoAmOKxwC2jc8=\u003c/td\u003e\n                  \u003ctd\u003e212-233-4532\u003c/td\u003e\n              \u003c/tr\u003e\n              \u003ctr\u003e\n                  \u003ctd\u003e4\u003c/td\u003e\n                  \u003ctd\u003esome script with simple number 1234\u003c/td\u003e\n                  \u003ctd\u003esome script with simple number 1234\u003c/td\u003e\n                  \u003ctd\u003esome script with simple number 1234\u003c/td\u003e\n              \u003c/tr\u003e\n          \u003c/tbody\u003e\n      \u003c/table\u003e\n\n## Detailed Deployment steps\n\nIn case you want to customize the deployment, please use following steps:\n\n### Setting up your environment\n\n1.  Enable APIs for Compute Engine, Cloud Storage, Dataproc, and Cloud SQL services:\n\n    ```shell\n    gcloud services enable --project \"${PROJECT_ID}\" \\\n    artifactregistry.googleapis.com \\\n    bigquery.googleapis.com \\\n    bigqueryconnection.googleapis.com \\\n    cloudbuild.googleapis.com \\\n    cloudkms.googleapis.com \\\n    containerregistry.googleapis.com \\\n    dlp.googleapis.com \\\n    iam.googleapis.com \\\n    run.googleapis.com \\\n    secretmanager.googleapis.com\n    ```\n\n1.  In Cloud Shell, set the [Cloud Region](https://cloud.google.com/compute/docs/regions-zones#available) that you want to create your BigQuery and Cloud Run resources in:\n\n    ```shell\n    PROJECT_ID=\"\u003cPROJECT_ID\u003e\"\n    REGION=\"\u003cREGION_ID\u003e\"\n    ARTIFACT_REGISTRY_NAME=\"\u003cARTIFACT_DOCKER_REGISTRY_NAME\u003e\"\n    CLOUD_RUN_SERVICE_NAME=\"bq-transform-fns\"\n    ```\n\n### Create Service Account for Cloud Run service\n\nFor fine-grained access control, using a specialized service account for\nindividual services is recommended.\n\n1.  Create a service Account:\n\n    ```shell\n    RUNNER_SA_NAME=\"${CLOUD_RUN_SERVICE_NAME}-runner\"\n    RUNNER_SA_EMAIL=\"${RUNNER_SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com\"\n\n    gcloud iam service-accounts create \"${RUNNER_SA_NAME}\" \\\n    --project=\"${PROJECT_ID}\" \\\n    --description \"Runner for BigQuery remote function execution\" \\\n    --display-name \"${RUNNER_SA_NAME}\"\n    ```\n\n1.  Grant permissions to the service account to access DLP\n\n    ```shell\n    gcloud projects add-iam-policy-binding \"${PROJECT_ID}\" \\\n    --member=\"serviceAccount:${RUNNER_SA_EMAIL}\" \\\n    --role='roles/dlp.deidentifyTemplatesReader'\n\n    gcloud projects add-iam-policy-binding \"${PROJECT_ID}\" \\\n    --member=\"serviceAccount:${RUNNER_SA_EMAIL}\" \\\n    --role='roles/dlp.user'\n    ```\n\n### Create Artifact Registry\nThis is a containerized SpringBoot application.\nCreate an [Artifact Registry](https://cloud.google.com/artifact-registry) to store the application's container image\n\n```shell\ngcloud artifacts repositories create \"${ARTIFACT_REGISTRY_NAME}\" \\\n--repository-format=docker \\\n--location=\"${REGION}\" \\\n--description=\"Docker repository for Bigquery Functions\" \\\n--project=\"${PROJECT_ID}\"\n```\n\n### Deploy Cloud Run service\n\n1. Build the application container image using [Cloud Build](https://cloud.google.com/build):\n    ```shell\n    gcloud builds submit . \\\n    --project=\"${PROJECT_ID}\" \\\n    --substitutions=_CONTAINER_IMAGE_NAME=\"${REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REGISTRY_NAME}/${CLOUD_RUN_SERVICE_NAME}:latest\" \\\n    --machine-type=e2-highcpu-8\n    ```\n\n2. Deploy Cloud Run by compiling and deploying Container :\n\n    ```shell\n    gcloud beta run deploy ${CLOUD_RUN_SERVICE_NAME} \\\n    --image=\"${REGION}-docker.pkg.dev/${PROJECT_ID}/${ARTIFACT_REGISTRY_NAME}/${CLOUD_RUN_SERVICE_NAME}:latest\" \\\n    --execution-environment=gen2 \\\n    --platform=managed \\\n    --region=\"${REGION}\" \\\n    --service-account=\"${RUNNER_SA_EMAIL}\" \\\n    --update-env-vars=PROJECT_ID=${PROJECT_ID} \\\n    --no-allow-unauthenticated \\\n    --project ${PROJECT_ID}\n    ```\n\n1.  Retrieve and save the Cloud Run URL:\n\n    ```shell\n    RUN_URL=\"$(gcloud run services describe ${CLOUD_RUN_SERVICE_NAME} --region ${REGION} --project ${PROJECT_ID} --format=\"get(status.address.url)\")\"\n    ```\n\n### Create DLP Didentify Templates\n\nDLP [Deidentify templates](https://cloud.google.com/dlp/docs/creating-templates-deid) makes managing deidentification and inspection easy to document and manage.\n\n```shell\nDEID_TEMPLATE=$(curl -X POST \\\n-H \"Authorization: Bearer `gcloud auth print-access-token`\" \\\n-H \"Accept: application/json\" \\\n-H \"Content-Type: application/json\" \\\n-H \"X-Goog-User-Project: ${PROJECT_ID}\" \\\n--data-binary \"@sample_dlp_deid_config.json\" \\\n\"https://dlp.googleapis.com/v2/projects/${PROJECT_ID}/locations/${REGION}/deidentifyTemplates\")\n\nDEID_TEMPLATE_NAME=$(echo ${DEID_TEMPLATE} | jq -r '.name')\n```\n\n\u003e **Note:** Recommended practice is to use\n\u003e [KMS Wrapped Key for DLP de-identification](https://cloud.google.com/dlp/docs/create-wrapped-key).\n\u003e\n\u003e This document uses unwrapped key for simplification of demo purpose.\n\n### Create BigQuery Remote Functions\n\n1.  Create BigQuery connection for accessing Cloud Run:\n\n    ```shell\n    bq mk --connection \\\n    --display_name='External transform function connection' \\\n    --connection_type=CLOUD_RESOURCE \\\n    --project_id=\"${PROJECT_ID}\" \\\n    --location=\"${REGION}\" \\\n    ext-${CLOUD_RUN_SERVICE_NAME}\n    ```\n\n1.  Find the BigQuery Service Account used for the connection:\n\n    ```shell\n    CONNECTION_SA=\"$(bq --project_id ${PROJECT_ID} --format json show --connection ${PROJECT_ID}.${REGION}.ext-${CLOUD_RUN_SERVICE_NAME} | jq '.cloudResource.serviceAccountId')\"\n    ```\n\n1.  Grant the BigQuery connection Service Account Cloud Run Invoker role for accessing the Cloud Run:\n\n    ```shell\n    gcloud projects add-iam-policy-binding ${PROJECT_ID} \\\n    --member=\"serviceAccount:${CONNECTION_SA}\" \\\n    --role='roles/run.invoker'\n    ```\n\n### Create BQ Dataset for remote functions\n\n1.  Define the BigQuery dataset to create remote functions:\n\n    ```shell\n    BQ_FUNCTION_DATASET=\"fns\"\n    ```\n\n1.  Create the dataset if it doesn't exist:\n\n    ```shell\n    bq mk --dataset \\\n    --project_id ${PROJECT_ID} \\\n    --location ${REGION} \\\n    ${BQ_FUNCTION_DATASET}\n    ```\n\n### Create BigQuery Remote functions for DLP\n\n1.  Create DLP de-identification function:\n\n    DLP uses default inspection template, if you want to use your custom [Inspection template](https://cloud.google.com/dlp/docs/concepts-templates#the_inspecttemplate_and_deidentifytemplate_objects),\n    please add following to the `user_defined_context`: `('dlp-inspect-template', '\u003cfull inspection template name\u003e')`\n\n    ```shell\n    bq query --project_id ${PROJECT_ID} \\\n    --use_legacy_sql=false \\\n    \"CREATE OR REPLACE FUNCTION ${BQ_FUNCTION_DATASET}.dlp_freetext_encrypt(v STRING)\n    RETURNS STRING\n    REMOTE WITH CONNECTION \\`${PROJECT_ID}.${REGION}.ext-${CLOUD_RUN_SERVICE_NAME}\\`\n    OPTIONS (endpoint = '${RUN_URL}', user_defined_context = [('mode', 'deidentify'),('algo','dlp'),('dlp-deid-template','${DEID_TEMPLATE_NAME}')]);\"\n    ```\n\n1.  Create DLP re-identification function:\n\n    DLP uses default inspection template, if you want to use your custom [Inspection template](https://cloud.google.com/dlp/docs/concepts-templates#the_inspecttemplate_and_deidentifytemplate_objects),\n    please add following to the `user_defined_context`: `('dlp-inspect-template', '\u003cfull inspection template name\u003e')`\n\n    ```shell\n    bq query --project_id ${PROJECT_ID} \\\n    --use_legacy_sql=false \\\n    \"CREATE OR REPLACE FUNCTION ${BQ_FUNCTION_DATASET}.dlp_freetext_decrypt(v STRING)\n    RETURNS STRING\n    REMOTE WITH CONNECTION \\`${PROJECT_ID}.${REGION}.ext-${CLOUD_RUN_SERVICE_NAME}\\`\n    OPTIONS (endpoint = '${RUN_URL}', user_defined_context = [('mode', 'reidentify'),('algo','dlp'),('dlp-deid-template','${DEID_TEMPLATE_NAME}')]);\"\n    ```\n\n\n## Verify De-identification and Re-identification\n\nExecute the following query to observe that the remote function is deidentifying\nand reidentifying the data using SQL:\n\n1.  Using [BigQuery Workspace](https://console.cloud.google.com/bigquery)\n\n    ```sql\n    SELECT\n        pii_column,\n        fns.dlp_freetext_encrypt(pii_column) AS dlp_encrypted,\n        fns.dlp_freetext_decrypt(fns.dlp_freetext_encrypt(pii_column)) AS dlp_decrypted\n    FROM\n        UNNEST(\n        [\n            'My name is John Doe. My email is john@doe.com',\n            'Some non PII data',\n            '212-233-4532',\n            'some script with simple number 1234']) AS pii_column\n    ```\n\n1.  Using Cloud Shell run the following command to run the query on BigQuery:\n\n    ```shell\n    bq query --project_id ${PROJECT_ID} \\\n    --use_legacy_sql=false \\\n    \"\n    SELECT\n      pii_column,\n      ${BQ_FUNCTION_DATASET}.dlp_freetext_encrypt(pii_column) AS dlp_encrypted,\n      ${BQ_FUNCTION_DATASET}.dlp_freetext_decrypt(${BQ_FUNCTION_DATASET}.dlp_freetext_encrypt(pii_column)) AS dlp_decrypted\n    FROM\n      UNNEST(\n        [\n          'My name is John Doe. My email is john@doe.com',\n          'Some non PII data'\n        ]) AS pii_column\"\n    ```\n\n    Expected Output:\n\n\n    |  pii_column  |  dlp_encrypted  | dlp_decrypted | aes_encrypted  |  aes_decrypted  |\n    | -----------  | -----------  | ----------- | -----------  | -----------  |\n    | My name is John Doe. My email is john@doe.com | My name is John Doe. My email is BQ_TOK_FN(40):AQy6lGvwKR+AiiRqJpEr+nBzZUzOcjXkXamUugU= | My name is John Doe. My email is john@doe.com | qWnwDEY3bDTCV/azveHnxUm24z/DvUcVLZWHyN4Rg16ISvqswcuYXNXsOyI4/d8M | My name is John Doe. My email is john@doe.com |\n    | Some non PII data                              | Some non PII data                                                                       | Some non PII data                                                | 37Agm90CqzGXwerJxai/Tf89ffRN9FpPkuuDW+rz7ns=                     | Some non PII data                             |\n\n## Limitations\n\n *  The de-identification and re-identification processing occurs through a Cloud Run service.\n    Please provision the Cloud Run CPU/Memory in accordance with your compute requirements\n *  When using DLP consider following:\n    *  Use at least `--cpu=4 --memory=8Gi` for Cloud Run instance\n    *  When using DLP consider following limits for throughput and cost considerations:\n    *  Limit to 10000 or fewer items for DLP processing\n *  Ensure BigQuery, Cloud Run service and DLP templates are in the same cloud region\n\n## Cleaning up\n\n\u003cdiv style=\"color: lightcoral\"\u003e\n\u003cb\u003eCaution:\u003c/b\u003e Deleting a project has the following effects:\n\u003cul style=\"color: lightcoral\"\u003e\n    \u003cli\u003e Everything in the project is deleted. If you used an existing project for this tutorial, when you delete it, you also delete any other work you've done in the project. \u003c/li\u003e\n    \u003cli\u003e Custom project IDs are lost. When you created this project, you might have created a custom project ID that you want to use in the future. To preserve the URLs that use the project ID, such as an appspot.com URL, delete selected resources inside the project instead of deleting the whole project. \u003c/li\u003e\n\u003c/ul\u003e\nIf you plan to explore multiple tutorials and quickstarts, reusing projects can help you avoid exceeding project quota limits.\n\u003c/div\u003e\n\nTo avoid incurring charges to your Google Cloud account for the resources used in this tutorial, you can delete the project:\n\n1.  In the Cloud Console, go to the [**Manage resources** page](https://console.cloud.google.com/iam-admin/projects).\n1.  In the project list, select the project that you want to delete and then click **Delete** ![delete](bin_icon.png).\n1.  In the dialog, type the project ID and then click **Shut down** to delete the project.\n\n\n## What's next\n\n* Learn more about [BigQuery Remote Functions](https://cloud.google.com/bigquery/docs/reference/standard-sql/remote-functions)\n* Learn more about [Cloud DLP](https://cloud.google.com/dlp)\n* Learn more about [Cloud KMS](https://cloud.google.com/kms)\n* Learn about [Inspecting storage and databases for sensitive data](https://cloud.google.com/dlp/docs/inspecting-storage)\n* Handling [De-identification and re-identification of PII in large-scale datasets using DLP](https://cloud.google.com/solutions/de-identification-re-identification-pii-using-cloud-dlp)\n\n## Disclaimer\n**License**: Apache 2.0\n\nThis is not an official Google product.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecloudplatform%2Fbigquery-dlp-remote-function","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgooglecloudplatform%2Fbigquery-dlp-remote-function","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgooglecloudplatform%2Fbigquery-dlp-remote-function/lists"}