https://github.com/ydb-platform/dbt-ydb
https://github.com/ydb-platform/dbt-ydb
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/ydb-platform/dbt-ydb
- Owner: ydb-platform
- Created: 2025-04-21T09:48:56.000Z (about 1 year ago)
- Default Branch: main
- Last Pushed: 2025-06-09T15:39:51.000Z (about 1 year ago)
- Last Synced: 2025-07-08T05:41:16.110Z (11 months ago)
- Language: Python
- Size: 126 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# dbt-ydb
**dbt-ydb** is a plugin for [dbt](https://www.getdbt.com/) that provides support for working with [YDB](https://ydb.tech). **dbt-ydb** adapter is in preview stage and does not currently support all dbt features. The sections below list the supported features and known limitations.
## Installation
To install plugin, execute the following command:
```bash
pip install dbt-ydb
```
## Supported features
- [x] Table materialization
- [x] View materialization
- [x] Seeds
- [x] Docs generate
- [x] Tests
- [x] Incremental materializations (`merge` strategy only)
- [x] Snapshots
## Limitations
* `YDB` does not support CTE
* `YDB` requires a primary key to be specified for its tables. See the configuration section for instructions on how to set it.
* `source()` macro requires you to specify a `schema`. Use `/` if your source is in root folder.
## Usage
### Profile Configuration
To configure YDB connection, fill `profile.yml` file as below:
```
profile_name:
target: dev
outputs:
dev:
type: ydb
host: [localhost] # YDB host
port: [2136] # YDB port
database: [/local] # YDB database
schema: [] # Optional subfolder for DBT models
secure: [False] # If enabled, grpcs protocol will be used
root_certificates_path: [] # Optional path to root certificates file
# Static Credentials
username: []
password: []
# Access Token Credentials
token: []
# Service Account Credentials
service_account_credentials_file: []
```
### Model Configuration
#### View
| Option | Description | Required | Default |
| ------ | ----------- | -------- | ------- |
#### Table
| Option | Description | Required | Default |
| ------ | ----------- | -------- | ------- |
| `primary_key` | Primary key expression to use during table creation | `yes` | |
| `store_type` | Type of table. Available options are `row` and `column` | `no` | `row` |
| `auto_partitioning_by_size` | Enable automatic partitioning by size. Available options are `ENABLED` and `DISABLED` | `no` | |
| `auto_partitioning_partition_size_mb` | Partition size in megabytes for automatic partitioning | `no` | |
| `ttl` | Time-to-live (TTL) expression for automatic data expiration | `no` | |
#### Incremental
| Option | Description | Required | Default |
| ------ | ----------- | -------- | ------- |
| `incremental_strategy` | Strategy of incremental materialization. Current adapter supports only `merge` strategy, which will use `YDB`'s `UPSERT` operation. | `no` | `default` |
| `primary_key` | Primary key expression to use during table creation | `yes` | |
| `store_type` | Type of table. Available options are `row` and `column` | `no` | `row` |
| `auto_partitioning_by_size` | Enable automatic partitioning by size. Available options are `ENABLED` and `DISABLED` | `no` | |
| `auto_partitioning_partition_size_mb` | Partition size in megabytes for automatic partitioning | `no` | |
| `ttl` | Time-to-live (TTL) expression for automatic data expiration | `no` | |
##### Example table configuration
```sql
{{ config(
primary_key='id, created_at',
store_type='row',
auto_partitioning_by_size='ENABLED',
auto_partitioning_partition_size_mb=256,
ttl='Interval("P30D") on created_at'
) }}
select
id,
name,
created_at
from {{ ref('source_table') }}
```
#### Seed
| Option | Description | Required | Default |
| ------ | ----------- | -------- | ------- |
| `primary_key` | Primary key expression to use during table creation | `no` | The first column of CSV will be used as default. |