https://github.com/gitpod-samples/template-dbt-bigquery
A dbt template, configured to launch a dbt Project in the Cloud for a user specified BigQuery project
https://github.com/gitpod-samples/template-dbt-bigquery
Last synced: 3 months ago
JSON representation
A dbt template, configured to launch a dbt Project in the Cloud for a user specified BigQuery project
- Host: GitHub
- URL: https://github.com/gitpod-samples/template-dbt-bigquery
- Owner: gitpod-samples
- Created: 2023-04-03T11:57:20.000Z (about 2 years ago)
- Default Branch: main
- Last Pushed: 2023-08-23T08:59:11.000Z (almost 2 years ago)
- Last Synced: 2025-01-24T12:45:32.296Z (5 months ago)
- Size: 22.5 KB
- Stars: 0
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# dbt + BigQuery in Gitpod Demo Project
This demo project showcases how to run [dbt](https://www.getdbt.com/)'s core module on top of a BigQuery warehouse inside a [Gitpod](https://www.gitpod.io/) Cloud Development Environment. The project is based on the well-known [jaffle_shop example](https://github.com/dbt-labs/jaffle_shop/).
## Configuration
In order to connect with BigQuery, the following environment variables need to be set inside inside the development environment:
- `DBT_SERVICE_ACCOUNT`: the content of a valid service account JSON. Required permissions are _BigQuery User_ and _BigQuery Data Editor_
- `DBT_PROJECT`: ID of the GCP project in which the models will be persisted
- `DBT_DEV_DATASET`: Name of the dataset in which the models will be persisted, e.g. `dbt_dev`
- `DBT_LOCATION`: multi-regional or regional location of the data, e.g.: `US`, `EU`, `us-west2`
The easiest way to achieve this is by setting [user-specific environment variables in Gitpod](https://gitpod.io/user/variables). Find out more about environment variables in Gitpod [here](https://www.gitpod.io/docs/configure/projects/environment-variables).
The values of the environment variables are used inside dbt's `profiles.yml`. For more information about setting up dbt with BigQuery, click [here](https://docs.getdbt.com/reference/warehouse-setups/bigquery-setup).
## Next Steps
Click the button below to start a new development environment:
[](https://gitpod.io/#https://github.com/gitpod-samples/template-dbt-bigquery)
The validity of the dbt configuration will be tested inside the workspace. Run the following commands to test the functionality of the Jaffle shop demo:
1. `dbt seed`: to materialize the CSVs
2. `dbt run`: to run the models
3. `dbt test`: to test all models
If all commands run succesfully, you have completed the setup of a working dbt development environment on a BigQuery warehouse with Gitpod!🎉
## Get Started With Your Own Project
### A new project
Click the above "Open in Gitpod" button to start a new workspace. Once you're ready to push your first code changes, Gitpod will guide you to fork this project so you own it.
### An existing project
To get started with dbt on Gitpod, add a [`.gitpod.yml`](./.gitpod.yml) to any of your existing dbt projects. To learn more, please see the [Getting Started](https://www.gitpod.io/docs/getting-started) documentation.