https://github.com/chees/datastore2bigquery

Export Google Datastore to BigQuery
https://github.com/chees/datastore2bigquery

backup bigquery datastore gcloud gcp google-cloud-storage

Last synced: 3 months ago
JSON representation

Export Google Datastore to BigQuery

• Host: GitHub
• URL: https://github.com/chees/datastore2bigquery
• Owner: chees
• Created: 2017-09-09T18:53:32.000Z (over 8 years ago)
• Default Branch: master
• Last Pushed: 2019-05-22T14:34:08.000Z (over 6 years ago)
• Last Synced: 2025-04-14T20:46:42.449Z (8 months ago)
• Topics: backup, bigquery, datastore, gcloud, gcp, google-cloud-storage
• Language: Shell
• Homepage:
• Size: 3.91 KB
• Stars: 31
• Watchers: 3
• Forks: 10
• Open Issues: 4
• Metadata Files:
  • Readme: README.md

Awesome Lists containing this project

README

          
# Export Google Datastore to BigQuery

See this screen recording on YouTube for the demo:

[![YouTube video](https://img.youtube.com/vi/dGyQCE3bWkU/0.jpg)](https://www.youtube.com/watch?v=dGyQCE3bWkU "Google Cloud Datastore to BigQuery")

This script exports a Datastore kind to Cloud Storage, imports it into BigQuery and then deletes the Cloud Storage files.

Usage:

    ./datastore2bigquery.sh PROJECT KIND [BQDATASET]

Example:

    ./datastore2bigquery.sh code-cooking Food

This script requires the `bq` and `gsutil` gcloud components, so install those first if you haven't already:

    gcloud components install bq gsutil