https://github.com/intake/intake-dremio
Plugin for Intake to read from Dremio data lake engines
https://github.com/intake/intake-dremio
Last synced: 9 months ago
JSON representation
Plugin for Intake to read from Dremio data lake engines
- Host: GitHub
- URL: https://github.com/intake/intake-dremio
- Owner: intake
- License: bsd-3-clause
- Created: 2021-05-06T15:35:17.000Z (about 5 years ago)
- Default Branch: master
- Last Pushed: 2024-06-13T08:49:35.000Z (almost 2 years ago)
- Last Synced: 2025-08-16T04:53:27.010Z (9 months ago)
- Language: Python
- Homepage:
- Size: 79.1 KB
- Stars: 3
- Watchers: 5
- Forks: 2
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# Intake-Dremio
Dremio Plugin for Intake based on pyarrow Flight
## User Installation
To install the intake-dremio plugin, execute the following command
```
conda install -c conda-forge intake-dremio
```
or:
```
pip install intake-dremio
```
When installing from the `pyviz` channel and `conda-forge` conda channels, you must ensure that `libarrow-flight` is also installed:
```
conda install -c pyviz -c conda-forge intake-dremio libarrow-flight
```
This is because `conda-forge` has split up pyarrow into individual packages as of `pyarrow 16`.
## Example
An Intake catalog referencing a Dremio dataset consists of the `uri` pointing to the Dremio instance along with a username and password and a SQL expression (`sql_expr`), e.g.:
```yaml
sources:
dremio_vds:
driver: dremio
args:
uri: grpc+tcp://{{ env(DREMIO_USER) }}:{{ env(DREMIO_PASSWORD) }}@x.x.x.x:32010
sql_expr: SELECT * FROM TABLE ORDER BY "timestamp" ASC
```