Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/thinkingmachines/bqup
(pronounced BACK-wup) BigQuery backup scripts
https://github.com/thinkingmachines/bqup
Last synced: about 1 month ago
JSON representation
(pronounced BACK-wup) BigQuery backup scripts
- Host: GitHub
- URL: https://github.com/thinkingmachines/bqup
- Owner: thinkingmachines
- License: mit
- Archived: true
- Created: 2018-04-10T05:52:04.000Z (over 6 years ago)
- Default Branch: master
- Last Pushed: 2023-03-10T03:20:04.000Z (over 1 year ago)
- Last Synced: 2024-04-24T12:32:53.796Z (7 months ago)
- Language: Python
- Homepage: https://pypi.org/project/bqup/
- Size: 217 KB
- Stars: 30
- Watchers: 33
- Forks: 11
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Contributing: CONTRIBUTING.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
Awesome Lists containing this project
README
> **Note**
> If this is your first time checking out bqup, you might want to try out [Dataform](https://dataform.co/) or [dbt](https://www.getdbt.com/) instead!
# bqup
bqup is a backup tool for BigQuery projects. It can export a BigQuery
project's structure and source code while mimicking the hierarchy of
datasets and tables.
For the full story of why we made bqup, check out our [blog
post](https://stories.thinkingmachin.es/coding-safely-in-the-cloud/)!
## Installation
bqup can be installed using `pip`.
```
pip install bqup
```
Alternatively, you can also clone the repository then run `install`.
```
git clone https://github.com/thinkingmachines/bqup.git
cd bqup
python3 setup.py install
```
## Usage
### Command line options
You can list down the options by running `bqup --help`.
```text
bqup [-p PROJECT_ID] [-d TARGET_DIR] [-fvxr] [-e REGEX]
Options:
-p PROJECT_ID, --project PROJECT_ID Project ID to load. If unspecified,
defaults to current project in
configuration.
-d TARGET_DIR, --dir TARGET_DIR The target directory where the project
will be written. Defaults to current
timestamp.
-f --force Overwrite target directory if it exists.
-v --verbose Print a summary of the loaded project.
-x --schema Export table schemata as json.
-r --routine Include routines in export.
-e REGEX, --regex REGEX Regex pattern to filter datasets to be exported.
```
### Development
1. Set up `gcloud` to run with your personal account (aka run with scissors).
2. Set up
[application-default](https://cloud.google.com/sdk/gcloud/reference/auth/application-default/login).
```
gcloud auth application-default login
```
3. Install `wheel`.
```
pip3 install wheel
```
4. Install bqup.
```
pip3 install -e .
```
Alternatively, you can also install it using:
```
python3 setup.py develop
```
5. Run bqup (see [Usage](#usage)).
### Setting up regular backups
Check out [bqup-starter](https://github.com/thinkingmachines/bqup-starter) to set up regular bqups using GitHub's workflows!
### Distribution
Run `make test` to try a test upload.
Run `make dist` to upload a distribution.
Both of these will call `make build`, which rebuilds the package locally.
## Contributors
Thanks to all these wonderful people who've helped out with bqup:
Jess
Ram
Pepe Berba
Tim Pron
Enzo
Ardie
- [bhuesemann](https://github.com/bhuesemann)
- [bzieba-spartez](https://github.com/bzieba-spartez)
- [profwacko](https://github.com/profwacko)
- [urtho](https://github.com/urtho)
- [ajkruse](https://github.com/ajkruse)
## Disclaimers
bqup is **no longer being maintained**
- There are currently no plans to maintain this project.
- Thinking Machines does not make any guarantees about the quality of the
software.