https://github.com/mondo-mob/spring-gae-datastore-backup
Google App Engine Datastore Backup Service
https://github.com/mondo-mob/spring-gae-datastore-backup
Last synced: 4 months ago
JSON representation
Google App Engine Datastore Backup Service
- Host: GitHub
- URL: https://github.com/mondo-mob/spring-gae-datastore-backup
- Owner: mondo-mob
- Created: 2020-04-06T01:22:59.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2022-03-31T19:05:16.000Z (over 3 years ago)
- Last Synced: 2025-01-16T06:58:28.249Z (6 months ago)
- Language: Java
- Size: 12.7 KB
- Stars: 0
- Watchers: 11
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: readme.md
Awesome Lists containing this project
README
Backup Service (Java 8)
=======================
This app provides functionality to perform a Google Cloud Datastore backup.
It should be deployed as a separate App Engine service called "backup-service".
### Running locally
mvn appengine:run
### Deploying
CLOUDSDK_CORE_PROJECT={PROJECT_ID} mvn appengine:deploy
where `{PROJECT_ID}` is your GCP project id.
Usage
=====
Configure a cron and a task queue in your default service as per the instructions here: https://github.com/3wks/spring-boot-gae/blob/master/README-DATASTORE-BACKUP.md
Note that the cron and task must both be configured with `target: backup-service` to ensure they hit this service.
You cannot add `cron.xml` or `queue.xml` to this service, it must be added to your default service (ie: your main application).
Backup and Import to BigQuery
=======
As well as regular backups you can configure automatic export and import into BigQuery.
This is a full import/export and any existing data in BigQuery will be deleted for the requested kinds.
### Endpoint
Task endpoint: `/task/bigquery/datastore-export-and-load`
Params:
| Property | Description | Required |
| ------------- |------------- | -----------|
| name | name for the datastore export | N |
| dataset | the target dataset for the import in BigQuery | Y |
| kinds | the datastore kinds to be exported | Y |
| namespaceId | optional namespace id to filter by. Only a single namespace is supported. | N |
### Usage
In your default service schedule a cron to trigger an ETL job with target of `backup-service`.
e.g. for XML config
```
/task/bigquery/datastore-export-and-load?name=ExportToBigQuery&dataset=backup_data&kinds=Kind1,Kind2,Kind3
Datastore export and load to BigQuery
backup-service
every day 03:00
Australia/NSW
```
or yaml
```
- description: "Datastore export and load to BigQuery"
url: /task/bigquery/datastore-export-and-load?name=ExportToBigQuery&dataset=backup_data&kinds=Kind1,Kind2,Kind3
target: backup-service
schedule: every day 03:00
timezone: Australia/NSW
```