https://github.com/getindata/dbt-workflows-factory
Creates dbt based GCP workflows.
https://github.com/getindata/dbt-workflows-factory
Last synced: about 1 year ago
JSON representation
Creates dbt based GCP workflows.
- Host: GitHub
- URL: https://github.com/getindata/dbt-workflows-factory
- Owner: getindata
- License: apache-2.0
- Created: 2022-10-06T08:15:59.000Z (over 3 years ago)
- Default Branch: main
- Last Pushed: 2023-07-10T07:26:40.000Z (almost 3 years ago)
- Last Synced: 2025-02-24T13:02:31.244Z (over 1 year ago)
- Language: Python
- Size: 264 KB
- Stars: 1
- Watchers: 5
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- License: LICENSE
- Code of conduct: CODE_OF_CONDUCT.md
- Security: SECURITY.md
- Support: SUPPORT.md
Awesome Lists containing this project
README
# dbt-workflows-factory
[](https://github.com/getindata/dbt-workflows-factory)
[](https://pypi.org/project/dbt-workflows-factory/)
[](https://pepy.tech/project/dbt-workflows-factory)
Creates dbt based GCP workflows.
## Installation
Use the package manager [pip](https://pip.pypa.io/en/stable/) to install [dbt-workflows-factory](https://pypi.org/project/dbt-workflows-factory/) for [dp (data-pipelines-cli)]:
```bash
pip install dbt-workflows-factory
```
### How to run
To call from cli, you can
```bash
python -m dbt_workflows_factory.cli convert \
--image-uri my-image-uri \
--gcs-key-volume-remote-path google-cloud-storage/path/ \
--gcs-key-volume-mount-path /etc/gcs-key/ \
--gcs-key-volume-container-mount-path /etc/gcs-key/:/etc/gcs-key/:ro \
--container-gcp-key-path /etc/gcs-key/path.json \
--container-gcp-project-id some-project-id \
--pretty \
tests/unit/dbt_workflows_factory/test_data/manifest.json > workflow_source.json
```
## Project Organization
- .devcontainer - This directory contains required files for creating a [Codespace](https://github.com/features/codespaces).
- .github
- workflows - Contains GitHub Actions used for building, testing and publishing.
- publish.yml - Publish wheels to [https://pypi.org/](https://pypi.org/)
- pull-request.yml - Build and Test pull requests before commiting to main.
- template-sync.yml - Update GitHub Repo with enhancments to base template
- docs - collect documents (default format .md)
- src - place new source code here
- python_package - sample package (this can be deleted when creating a new repository)
- tests - contains Python based test cases to validation src code
- .pre-commit-config.yaml - Contains various pre-check fixes for Python
- pyproject.toml - Python Project Declaration
## Publish to PyPi from GitHub
In order to publish to PyPi, a repostirory secret must be created, "PYPI_PASSWORD". In order to publish to the Test PyPi, a second secret must be added, "TEST_PYPI_PASSWORD".