https://github.com/timeplus-io/dbt-proton
dbt adapter for Timeplus
https://github.com/timeplus-io/dbt-proton
Last synced: 12 months ago
JSON representation
dbt adapter for Timeplus
- Host: GitHub
- URL: https://github.com/timeplus-io/dbt-proton
- Owner: timeplus-io
- License: apache-2.0
- Created: 2023-10-17T20:59:34.000Z (over 2 years ago)
- Default Branch: main
- Last Pushed: 2024-12-21T01:38:59.000Z (over 1 year ago)
- Last Synced: 2024-12-21T02:29:30.333Z (over 1 year ago)
- Language: Python
- Homepage:
- Size: 34.2 KB
- Stars: 4
- Watchers: 4
- Forks: 0
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/silentsokolov/dbt-clickhouse/actions/workflows/build.yml)
# dbt-proton
This plugin ports [dbt](https://getdbt.com) functionality to [Timeplus Proton](https://github.com/timeplus-io/proton).
### Installation
Use your favorite Python package manager to install the app from PyPI, e.g.
```bash
pip install dbt-proton
```
### Development
Follow the [dbt Documentation])(https://docs.getdbt.com/docs/core/pip-install) to install dbt with pip.
```shell
python3.10 -m venv proton-dbt-env
source proton-dbt-env/bin/activate
pip install dbt-core
pip install -r dev_requirements.txt
```
Then run `pip install -e .` to install the current dev code.
Run `pytest tests/unit/test_adapter.py` to run basic tests.
Run `pytest tests/integration/proton.dbtspec` to run integration tests.
### Supported features
- [x] Table materialization
- [x] View materialization
- [x] Incremental materialization
- [x] Seeds
- [x] Sources
- [x] Docs generate
- [x] Tests
- [x] Snapshots (experimental)
- [ ] Ephemeral materialization
# Usage Notes
### Database
The dbt model `database.schema.table` is not compatible with ClickHouse because ClickHouse does not support a `schema`.
So we use a simple model `schema.table`, where `schema` is the ClickHouse's database. Please, don't use `default` database!
### Model Configuration
| Option | Description | Required? |
|----------------|------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------|
| engine | The table engine (type of table) to use when creating tables | Optional (default: `MergeTree()`) |
| order_by | A tuple of column names or arbitrary expressions. This allows you to create a small sparse index that helps find data faster. | Optional (default: `tuple()`) |
| partition_by | A partition is a logical combination of records in a table by a specified criterion. The partition key can be any expression from the table columns. | Optional |
### Example Profile
```
your_profile_name:
target: dev
outputs:
dev:
type: proton
schema: [database name] # default default
host: [db.url.proton] # default localhost
# optional
port: [port] # default 8463
user: [user]
password: [abc123]
cluster: [cluster name]
verify: [verify] # default False
secure: [secure] # default False
connect_timeout: [10] # default 10
send_receive_timeout: [300] # default 300
sync_request_timeout: [5] # default 5
compress_block_size: [1048576] # default 1048576
compression: ['lz4'] # default '' (disable)
```