https://github.com/datatovalue/datatovalue-tools
datatovalue-tools
https://github.com/datatovalue/datatovalue-tools
bigquery sql terraform
Last synced: 2 months ago
JSON representation
datatovalue-tools
- Host: GitHub
- URL: https://github.com/datatovalue/datatovalue-tools
- Owner: datatovalue
- License: apache-2.0
- Created: 2024-12-16T09:58:09.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-09T12:03:15.000Z (about 1 year ago)
- Last Synced: 2026-04-22T15:39:22.335Z (3 months ago)
- Topics: bigquery, sql, terraform
- Language: HCL
- Homepage:
- Size: 165 KB
- Stars: 11
- Watchers: 2
- Forks: 2
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE.md
Awesome Lists containing this project
README
# datatovalue-tools
The **`datatovalue-tools`** library extends Google BigQuery to simplify common Analytics Engineering use-cases which we encounter daily working with clients at [Data to Value](https://datatovalue.com/).
More specifically, the library comprises a set of utility functions to support data profiling, debugging, root-cause analysis and automation activities in data transformation and management. It is developed and maintained by the Engineering Team at Data to Value and is licensed under Apache 2.0.
If you are interested in making a contribution or suggestion, please contact `jim@datatovalue.com`.
## Functions
The following sets of functions are deployed in the **datatovalue-tools** Google Cloud project, across all available regions.
Function Set | Description
--- | ---
[Metadata Functions](docs/metadata_functions.md) | Functions supporting metadata-based analysis and automation.
[Monitor Functions](docs/monitor_functions.md) | Functions which are used to monitor BigQuery resources and resource status.
[Profile Functions](docs/profile_functions.md) | Functions to support data quality assurance activities by modelling, integrating and analysing table contents and metadata.
[JSON Functions](docs/json_functions.md) | Functions to support automatic schema parsing and JSON parser deployment, to support accurate data transfer and efficient data pipeline development.
[SQL Parsers](docs/sql_parsers.md) | Functions which parse SQL queries and derive the logical structure and dependency graph.
## Deployment
Functions are live and deployed across in the `datatovalue-tools` BigQuery project for all global regions and multi-regions. Note that the dataset names contain underscores instead of dashes (e.g. functions in `europe-north1` are in the dataset `datatovalue-tools.europe_north1`).
## Permissions
Functions can be called by any user with `BigQuery Data Viewer`, `BigQuery Data User` or higher permissions on your source data. All authenticated users are permitted to call functions in the `datatovalue-tools` regional datasets.