https://github.com/kieras/google-cloud-datastore-utils
Utilities for Google Cloud Datastore.
https://github.com/kieras/google-cloud-datastore-utils
google-cloud google-cloud-datastore google-cloud-platform
Last synced: 2 months ago
JSON representation
Utilities for Google Cloud Datastore.
- Host: GitHub
- URL: https://github.com/kieras/google-cloud-datastore-utils
- Owner: kieras
- License: mit
- Created: 2018-01-08T16:14:52.000Z (about 8 years ago)
- Default Branch: develop
- Last Pushed: 2019-11-05T05:45:21.000Z (over 6 years ago)
- Last Synced: 2025-08-18T12:01:09.608Z (7 months ago)
- Topics: google-cloud, google-cloud-datastore, google-cloud-platform
- Language: Python
- Size: 36.1 KB
- Stars: 8
- Watchers: 4
- Forks: 2
- Open Issues: 6
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Google Cloud Datastore Utils
Utilities for Google Cloud Datastore.
Supported commands:
- **export**: Export entities from any project/namespace to a file.
- **import**: Import entities from a file into any project/namespace.
## Installation
Simply run:
```bash
pipsi install gcdu
```
---
## Usage
To use it:
```bash
gcdu --help
```
Export command:
```bash
gcdu export -p [project] -n [namespace] -k [comma separated list of datastore kinds]
```
Import command:
```bash
gcdu import -p [project] -n [namespace] -k [comma separated list of datastore kinds]
```
## Authentication
You can execute the program with your user authenticated in the using `gcloud`:
```bash
gcloud auth application-default login
gcloud config set project PROJECT_ID
```
If you want, you can use a Service Account to authenticate by exporting the following variable:
```bash
export GOOGLE_APPLICATION_CREDENTIALS='service-account-key.json'
```
## Example execution of an import
```bash
gcdu import --project my-project-stage --namespace e2e-20190313-192340 --data-dir=resources/database --kinds=AuditLog,Budget,BudgetBigqueryIntegration,BudgetChannel,BudgetCostCenter,BudgetEntity,BudgetGLCode,BudgetLocation,BudgetProduct,CurrencyCode,CurrencyEntity,Entry,FeatureToggle,LoadVersionControl,Settings,Submission,SubmissionForEmailCron,SubmissionLinePreview,SubmissionSequence,Team,Template,Transaction,User,Checkpoint,CostCenter,Header,OptionEntry,Question,QuestionAlias,BudgetQuarterSnapshot
```
Command output:
```bash
Executing import. Project=my-project-stage, Namespace=e2e-20190313-192340, Kinds=[u'AuditLog', u'Budget', u'BudgetBigqueryIntegration', u'BudgetChannel', u'BudgetCostCenter', u'BudgetEntity', u'BudgetGLCode', u'BudgetLocation', u'BudgetProduct', u'CurrencyCode', u'CurrencyEntity', u'Entry', u'FeatureToggle', u'LoadVersionControl', u'Settings', u'Submission', u'SubmissionForEmailCron', u'SubmissionLinePreview', u'SubmissionSequence', u'Team', u'Template', u'Transaction', u'User', u'Checkpoint', u'CostCenter', u'Header', u'OptionEntry', u'Question', u'QuestionAlias', u'BudgetQuarterSnapshot'].
Starting tasks...
Done. 30 tasks started.
Executing...
Done. Kind=Template
Done. Kind=Header
Done. Kind=LoadVersionControl
Done. Kind=Question
Done. Kind=BudgetEntity
Done. Kind=Checkpoint
Done. Kind=User
Done. Kind=BudgetLocation
Done. Kind=Settings
Done. Kind=BudgetCostCenter
Done. Kind=FeatureToggle
Done. Kind=BudgetChannel
Done. Kind=BudgetGLCode
Done. Kind=Team
Done. Kind=BudgetProduct
Done. Kind=OptionEntry
Done. Kind=CostCenter
Done. Kind=Budget
Done. Kind=QuestionAlias
Done. Kind=Transaction
Done. Kind=CurrencyEntity
Done. Kind=CurrencyCode
Done. Kind=SubmissionSequence
Done. Kind=BudgetQuarterSnapshot
Done. Kind=SubmissionForEmailCron
Done. Kind=SubmissionLinePreview
Done. Kind=Submission
Done. Kind=AuditLog
Done. Kind=Entry
Done. Kind=BudgetBigqueryIntegration
Finished!
```
Notice that we previously exported the database (not shown here), and now we are importing it into another project and namespace.
Another nice thing is that export/import executes in parallel, starting a task for each entity separately.