https://github.com/holistics/holistics_dbt
https://github.com/holistics/holistics_dbt
Last synced: 4 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/holistics/holistics_dbt
- Owner: holistics
- Created: 2024-06-21T05:22:12.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2024-07-12T03:40:11.000Z (almost 2 years ago)
- Last Synced: 2025-11-22T06:04:31.010Z (7 months ago)
- Language: Python
- Size: 63.5 KB
- Stars: 0
- Watchers: 4
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Holistics DBT
Holistics-DBT is a generic library for integrating DBT with Holistics.
Currently, it focuses on synchronizing Holistics data models with dbt using `manifest.json` and `catalog.json` files
**Supported databases:**
- BigQuery
- (more to come later)
## Install
- `pip install holistics-dbt`
## Usage
- Import the `AmlGen` and `Dbt` classes
- Provide the path to your `manifest.json` and `catalog.json` files
- The `catalog.json` can be obtained by running `dbt docs generate`
- Run the `AmlGen#gen_table_models` to generate the Holistics data models
- Call `to_s` on each model to get the AML text
```python
from holistics_dbt.aml_gen import AmlGen
from holistics_dbt.dbt import Dbt
dbt = Dbt(
manifest_path='target/manifest.json',
catalog_path='target/catalog.json'
)
aml_gen = AmlGen(dbt_artifact=dbt)
table_models = aml_gen.gen_table_models()
for table_model in table_models:
print(table_model.to_s())
```
Output:
```
Model model_ecommerce_dbt_model_cities {
type: 'table'
label: ''
description: ''
owner: ''
data_source_name: 'airy-berm-145910'
table_name: '`ecommerce`.`model_cities`'
dimension id {
type: 'number'
hidden: false
definition: @sql {{ #SOURCE.id }};;
label: 'id'
}
dimension name {
type: 'text'
hidden: false
definition: @sql {{ #SOURCE.name }};;
label: 'name'
}
}
```
## Customization
The `gen_table_models` return a list of `AmlTableModel` objects which are just plain Python objects. You can see the class definition at [aml_table_model.py](https://github.com/holistics/holistics_dbt/blob/master/holistics_dbt/aml_table_model.py).
Feel free to modify the class to suit your needs, and then call `to_s` to get the AML codes.