An open API service indexing awesome lists of open source software.

https://github.com/datalpia/laketower

Oversee your lakehouse
https://github.com/datalpia/laketower

apache-iceberg arrow data deltalake duckdb lakehouse sql

Last synced: about 4 hours ago
JSON representation

Oversee your lakehouse

Awesome Lists containing this project

README

          

# ๐Ÿ—ผ Laketower

> Oversee your lakehouse

[![PyPI](https://img.shields.io/pypi/v/laketower.svg)](https://pypi.org/project/laketower/)
[![Python Versions](https://img.shields.io/pypi/pyversions/laketower?logo=python&logoColor=white)](https://pypi.org/project/laketower/)
[![CI/CD](https://github.com/datalpia/laketower/actions/workflows/ci-cd.yml/badge.svg)](https://github.com/datalpia/laketower/actions/workflows/ci-cd.yml)
[![License](https://img.shields.io/github/license/datalpia/laketower)](https://github.com/datalpia/laketower/blob/main/LICENSE)

Utility application to explore and manage tables in your data lakehouse, especially tailored for data pipelines local development.

## Features

- Delta Lake table format support
- Remote tables support (S3, ADLS)
- Inspect table metadata
- Inspect table schema
- Inspect table history
- Get table statistics
- Import data into a table from CSV files
- View table content with a simple query builder
- Query all registered tables with DuckDB SQL dialect
- Execute saved queries
- Export query results to CSV files
- Static and versionable YAML configuration
- Web application
- CLI application

## Installation

Using `pip` (or any other Python package manager):

```bash
pip install laketower
```

Using `uvx`:

```bash
uvx laketower
```

## Usage

### Configuration

Laketower configuration is based on a static YAML configuration file allowing to:

- List all tables to be registered

Format:

```yaml
settings:
max_query_rows: 1000
web:
hide_tables: false

storage_credentials:
:
s3: # mutually exclusive with adls
access_key_id:
secret_access_key:
region:
endpoint_url:
allow_http: false
adls: # mutually exclusive with s3
account_name:
access_key:
sas_key:
tenant_id:
client_id:
client_secret:
msi_endpoint:
use_azure_cli: false

tables:
- name:
uri:
format: {delta}
storage_credential: # optional, references storage_credentials

queries:
- name:
title:
description:
totals_row: true
parameters:
:
default:
sql:
```

Current limitations:

- `tables.uri`:
- Local paths are supported (`./path/to/table`, `/abs/path/to/table`, `file:///abs/path/to/table`)
- Remote paths to S3 (`s3:///`) and ADLS (`abfss:///`)
- `tables.format`: only `delta` is allowed

Example from the provided demo:

```yaml
tables:
- name: sample_table
uri: demo/sample_table
format: delta
- name: weather
uri: demo/weather
format: delta

queries:
- name: all_data
title: All data
sql: |
select
sample_table.*,
weather.*
from
sample_table,
weather
limit 10
- name: daily_avg_temperature
title: Daily average temperature
sql: |
select
date_trunc('day', time) as day,
round(avg(temperature_2m)) as avg_temperature
from
weather
group by
day
order by
day asc
```

Support for environment variables substitution is also supported within the YAML
configuration using a object containing a single key `env` with the name of the
environment variable to be injected. The value of the variable can contain JSON
and will be decoded in a best effort manner (default to string value). For instance:

```yaml
# export TABLE_URI=path/to/table

tables:
- name: sample_table
uri:
env: TABLE_URI
format: delta
```

For string values that only need partial substitution, use the `${VAR_NAME}`
inline syntax instead. Multiple variables in a single value are supported:

```yaml
# export BUCKET=my-bucket
# export PREFIX=my-prefix

tables:
- name: sample_table
uri: s3://${BUCKET}/${PREFIX}/sample_table
format: delta
```

The two syntaxes are independent: `{env: VAR}` replaces the entire value (any
type), while `${VAR}` interpolates within a string.

#### Config Includes

Large or multi-environment setups can split the configuration across multiple
YAML files using the `include` directive. Included files are deep-merged before
the main file, so the main file always wins on conflict.

```yaml
# queries.yml (shared across environments)
queries:
- name: all_data
title: All data
sql: "select * from my_table"
```

```yaml
# laketower.production.yml
include:
- queries.yml

storage_credentials:
s3_creds:
s3:
access_key_id: ${AWS_ACCESS_KEY_ID}
secret_access_key: ${AWS_SECRET_ACCESS_KEY}

tables:
- name: my_table
uri: s3://production-bucket/tables/my_table
format: delta
storage_credential: s3_creds
```

Merge semantics:

- **Lists** (`tables`, `queries`, ...): included file items come first, main file items appended
- **Dicts** (`settings`, `storage_credentials`, ...): recursively deep-merged, main file values win on conflict
- **Scalars**: main file wins

Rules:

- `include` paths are relative to the directory of the file declaring them
- Multiple files are merged in order (first listed = lowest priority)
- Environment variable substitution works in included files
- Included files themselves do not support `include` (no recursive includes)

#### Storage Credentials

Storage credentials are defined once under the top-level `storage_credentials`
key as a named registry, then referenced by name from each table via the
`storage_credential` field. This avoids repeating the same credentials across
multiple tables.

##### Remote S3 Tables

Configuring S3 tables (AWS, MinIO, Cloudflare R2, Scaleway Object Storage, โ€ฆ):

```yaml
storage_credentials:
my_s3:
s3:
access_key_id: access-key-id
secret_access_key: secret-access-key
region: s3-region
endpoint_url: http://s3.domain.com
allow_http: false

tables:
- name: delta_table_s3
uri: s3:///path/to/table
format: delta
storage_credential: my_s3
```

Depending on your object storage location and configuration, one might have to
set part or all the available `s3` parameters. The only required ones
are `access_key_id` and `secret_access_key`.

As a security best practice, avoid writing secrets directly in static
configuration files. Use environment variable substitution instead:

```yaml
storage_credentials:
my_s3:
s3:
access_key_id: access-key-id
secret_access_key:
env: S3_SECRET_ACCESS_KEY
region: s3-region
endpoint_url: http://s3.domain.com
allow_http: false

tables:
- name: delta_table_s3
uri: s3:///path/to/table
format: delta
storage_credential: my_s3
```

##### Remote ADLS Tables

Configuring Azure ADLS tables:

```yaml
storage_credentials:
my_adls:
adls:
account_name: adls-account-name
access_key: adls-access-key
sas_key: adls-sas-key
tenant_id: adls-tenant-id
client_id: adls-client-id
client_secret: adls-client-secret
msi_endpoint: https://msi.azure.com
use_azure_cli: false

tables:
- name: delta_table_adls
uri: abfss:///path/to/table
format: delta
storage_credential: my_adls
```

Depending on your object storage location and configuration, one might have to
set part or all the available `adls` parameters. The only required one
is `account_name`.

As a security best practice, avoid writing secrets directly in static
configuration files. Use environment variable substitution instead:

```yaml
storage_credentials:
my_adls:
adls:
account_name: adls-account-name
access_key:
env: ADLS_ACCESS_KEY

tables:
- name: delta_table_adls
uri: abfss:///path/to/table
format: delta
storage_credential: my_adls
```

#### Predefined Query Parameters

Predefined queries allows for specifying named parameters that can then be used
with an SQL statement using the `$param_name` syntax.

When a query parameter is left blank by the user, it is treated as `NULL` by the
SQL engine. Use `COALESCE` to provide a fallback so the filter becomes a no-op
instead of returning an error or empty results:

```yaml
queries:
- name: daily_avg_temperature_params
title: Daily average temperature with parameters
parameters:
start_date:
default: "2025-01-01"
end_date:
default: "2025-01-31"
sql: |
select
date_trunc('day', time) as day,
round(avg(temperature_2m)) as avg_temperature
from
weather
where
day between coalesce($start_date::timestamp, timestamp '-infinity')
and coalesce($end_date::timestamp, timestamp 'infinity')
group by
day
order by
day asc
```

In this example:

- Blank `start_date` leads to `timestamp '-infinity'` (no lower bound)
- Blank `end_date` leads to `timestamp 'infinity'` (no upper bound)
- If both parameters are blank, all rows are returned

### Web Application

The easiest way to get started is to launch the Laketower web application:

```bash
$ laketower -c demo/laketower.yml web
```

By default, the web application will run on host `127.0.0.1` and port `8000`.
If some custom setup is required (especially for cloud deployment), this configuration
can be customized at runtime:

```bash
$ laketower -c demo/laketower.yml web --host 0.0.0.0 --port 5000
```

#### Screenshots

![Laketower UI - Tables Overview](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_overview.png)
![Laketower UI - Tables View](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_view.png)
![Laketower UI - Tables Statistics](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_statistics.png)
![Laketower UI - Tables History](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_history.png)
![Laketower UI - Tables Import](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_import.png)
![Laketower UI - Tables Query](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_query.png)
![Laketower UI - Queries View](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/queries_view.png)

### CLI

Laketower provides a CLI interface:

```bash
$ laketower --help

usage: laketower [-h] [--version] [--config CONFIG] {web,config,tables,queries} ...

options:
-h, --help show this help message and exit
--version show program's version number and exit
--config, -c CONFIG Path to the Laketower YAML configuration file (default: laketower.yml)

commands:
{web,config,tables,queries}
web Launch the web application
config Work with configuration
tables Work with tables
queries Work with queries
```

By default, a YAML configuration file named `laketower.yml` will be looked for.
A custom path can be specified with the `-c` / `--config` argument.

#### Show resolved YAML configuration

Print the fully resolved configuration after include merging, useful for debugging composed setups.

```bash
$ laketower -c demo/laketower.yml config show
```

Pass `--with-env-vars-substitution` to also resolve environment variable references:

```bash
$ laketower -c demo/laketower.yml config show --with-env-vars-substitution
```

#### Validate YAML configuration

```bash
$ laketower -c demo/laketower.yml config validate

โ•ญโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฎ
โ”‚ Configuration is valid โ”‚
โ•ฐโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ•ฏ
Config(
tables=[
ConfigTable(name='sample_table', uri='demo/sample_table', table_format=),
ConfigTable(name='weather', uri='demo/weather', table_format=)
]
)
```

#### List all registered tables

```bash
$ laketower -c demo/laketower.yml tables list

tables
โ”œโ”€โ”€ sample_table
โ”‚ โ”œโ”€โ”€ format: delta
โ”‚ โ””โ”€โ”€ uri: demo/sample_table
โ””โ”€โ”€ weather
โ”œโ”€โ”€ format: delta
โ””โ”€โ”€ uri: demo/weather
```

#### Display a given table metadata

```bash
$ laketower -c demo/laketower.yml tables metadata sample_table

sample_table
โ”œโ”€โ”€ name: Demo table
โ”œโ”€โ”€ description: A sample demo Delta table
โ”œโ”€โ”€ format: delta
โ”œโ”€โ”€ uri: /Users/romain/Documents/dev/datalpia/laketower/demo/sample_table/
โ”œโ”€โ”€ id: c1cb1cf0-1f3f-47b5-a660-3cc800edd341
โ”œโ”€โ”€ version: 3
โ”œโ”€โ”€ created at: 2025-02-05 22:27:39.579000+00:00
โ”œโ”€โ”€ partitions:
โ””โ”€โ”€ configuration: {}
```

#### Display a given table schema

```bash
$ laketower -c demo/laketower.yml tables schema weather

weather
โ”œโ”€โ”€ time: timestamp[us, tz=UTC]
โ”œโ”€โ”€ city: string
โ”œโ”€โ”€ temperature_2m: float
โ”œโ”€โ”€ relative_humidity_2m: float
โ””โ”€โ”€ wind_speed_10m: float
```

#### Display a given table history

```bash
$ uv run laketower -c demo/laketower.yml tables history weather

weather
โ”œโ”€โ”€ version: 2
โ”‚ โ”œโ”€โ”€ timestamp: 2025-02-05 22:27:46.425000+00:00
โ”‚ โ”œโ”€โ”€ client version: delta-rs.0.23.1
โ”‚ โ”œโ”€โ”€ operation: WRITE
โ”‚ โ”œโ”€โ”€ operation parameters
โ”‚ โ”‚ โ””โ”€โ”€ mode: Append
โ”‚ โ””โ”€โ”€ operation metrics
โ”‚ โ”œโ”€โ”€ execution_time_ms: 4
โ”‚ โ”œโ”€โ”€ num_added_files: 1
โ”‚ โ”œโ”€โ”€ num_added_rows: 168
โ”‚ โ”œโ”€โ”€ num_partitions: 0
โ”‚ โ””โ”€โ”€ num_removed_files: 0
โ”œโ”€โ”€ version: 1
โ”‚ โ”œโ”€โ”€ timestamp: 2025-02-05 22:27:45.666000+00:00
โ”‚ โ”œโ”€โ”€ client version: delta-rs.0.23.1
โ”‚ โ”œโ”€โ”€ operation: WRITE
โ”‚ โ”œโ”€โ”€ operation parameters
โ”‚ โ”‚ โ””โ”€โ”€ mode: Append
โ”‚ โ””โ”€โ”€ operation metrics
โ”‚ โ”œโ”€โ”€ execution_time_ms: 4
โ”‚ โ”œโ”€โ”€ num_added_files: 1
โ”‚ โ”œโ”€โ”€ num_added_rows: 408
โ”‚ โ”œโ”€โ”€ num_partitions: 0
โ”‚ โ””โ”€โ”€ num_removed_files: 0
โ””โ”€โ”€ version: 0
โ”œโ”€โ”€ timestamp: 2025-02-05 22:27:39.722000+00:00
โ”œโ”€โ”€ client version: delta-rs.0.23.1
โ”œโ”€โ”€ operation: CREATE TABLE
โ”œโ”€โ”€ operation parameters
โ”‚ โ”œโ”€โ”€ metadata: {"configuration":{},"createdTime":1738794459722,"description":"Historical and forecast weather data from
โ”‚ โ”‚ open-meteo.com","format":{"options":{},"provider":"parquet"},"id":"a9615fb1-25cc-4546-a0fe-1cb534c514b2","name":"Weather","partitionCol
โ”‚ โ”‚ umns":[],"schemaString":"{\"type\":\"struct\",\"fields\":[{\"name\":\"time\",\"type\":\"timestamp\",\"nullable\":true,\"metadata\":{}},
โ”‚ โ”‚ {\"name\":\"city\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"temperature_2m\",\"type\":\"float\",\"nullable\":
โ”‚ โ”‚ true,\"metadata\":{}},{\"name\":\"relative_humidity_2m\",\"type\":\"float\",\"nullable\":true,\"metadata\":{}},{\"name\":\"wind_speed_1
โ”‚ โ”‚ 0m\",\"type\":\"float\",\"nullable\":true,\"metadata\":{}}]}"}
โ”‚ โ”œโ”€โ”€ protocol: {"minReaderVersion":1,"minWriterVersion":2}
โ”‚ โ”œโ”€โ”€ mode: ErrorIfExists
โ”‚ โ””โ”€โ”€ location: file:///Users/romain/Documents/dev/datalpia/laketower/demo/weather
โ””โ”€โ”€ operation metrics
```

#### Get statistics of a given table

Get basic statistics on all columns of a given table:

```bash
$ laketower -c demo/laketower.yml tables statistics weather

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ column_name โ”ƒ count โ”ƒ avg โ”ƒ std โ”ƒ min โ”ƒ max โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ time โ”‚ 576 โ”‚ None โ”‚ None โ”‚ 2025-01-26 01:00:00+01 โ”‚ 2025-02-12 00:00:00+01 โ”‚
โ”‚ city โ”‚ 576 โ”‚ None โ”‚ None โ”‚ Grenoble โ”‚ Grenoble โ”‚
โ”‚ temperature_2m โ”‚ 576 โ”‚ 5.2623263956047595 โ”‚ 3.326529069892729 โ”‚ 0.0 โ”‚ 15.1 โ”‚
โ”‚ relative_humidity_2m โ”‚ 576 โ”‚ 78.76909722222223 โ”‚ 15.701802163559918 โ”‚ 29.0 โ”‚ 100.0 โ”‚
โ”‚ wind_speed_10m โ”‚ 576 โ”‚ 7.535763886032833 โ”‚ 10.00898058743763 โ”‚ 0.0 โ”‚ 42.4 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```

Specifying a table version yields according results:

```bash
$ laketower -c demo/laketower.yml tables statistics --version 0 weather

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”“
โ”ƒ column_name โ”ƒ count โ”ƒ avg โ”ƒ std โ”ƒ min โ”ƒ max โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ time โ”‚ 0 โ”‚ None โ”‚ None โ”‚ None โ”‚ None โ”‚
โ”‚ city โ”‚ 0 โ”‚ None โ”‚ None โ”‚ None โ”‚ None โ”‚
โ”‚ temperature_2m โ”‚ 0 โ”‚ None โ”‚ None โ”‚ None โ”‚ None โ”‚
โ”‚ relative_humidity_2m โ”‚ 0 โ”‚ None โ”‚ None โ”‚ None โ”‚ None โ”‚
โ”‚ wind_speed_10m โ”‚ 0 โ”‚ None โ”‚ None โ”‚ None โ”‚ None โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```

#### Import data into a given table

Import a CSV dataset into a table in append mode:

```bash
$ laketower -c demo/laketower.yml tables import weather --file data.csv --mode append --format csv --delimiter ',' --encoding 'utf-8'
```

`--mode` argument can be one of:
- `append`: append rows to the table (default)
- `overwrite`: replace all rows with the ones from the input file

`--format` argument can be one of:
- `csv`: CSV file format (default)
- `xlsx`: Excel file format (requires `laketower[excel]`, imports the first sheet)

`--delimiter` argument can be:
- Any single character (only valid for CSV file format)
- Default is _comma_ (`','`)

`--encoding` argument can be:
- Any [standard Python encoding](https://docs.python.org/3/library/codecs.html#standard-encodings),
- Default is `'utf-8'`
- Only applies to CSV file format

#### View a given table

Using a simple query builder, the content of a table can be displayed.
Optional arguments:

- `--cols `: select which columns to display
- `--sort-asc `: sort by a column name in ascending order
- `--sort-desc `: sort by a column name in descending order
- `--limit ` (default 10): limit the number of rows
- `--version`: time-travel to table revision number

```bash
$ laketower -c demo/laketower.yml tables view weather

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ time โ”ƒ city โ”ƒ temperature_2m โ”ƒ relative_humidity_2m โ”ƒ wind_speed_10m โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 2025-02-05 01:00:00+01:00 โ”‚ Grenoble โ”‚ 2.0 โ”‚ 84.0 โ”‚ 4.0 โ”‚
โ”‚ 2025-02-05 02:00:00+01:00 โ”‚ Grenoble โ”‚ 2.0999999046325684 โ”‚ 83.0 โ”‚ 1.5 โ”‚
โ”‚ 2025-02-05 03:00:00+01:00 โ”‚ Grenoble โ”‚ 1.600000023841858 โ”‚ 86.0 โ”‚ 1.100000023841858 โ”‚
โ”‚ 2025-02-05 04:00:00+01:00 โ”‚ Grenoble โ”‚ 1.899999976158142 โ”‚ 80.0 โ”‚ 4.199999809265137 โ”‚
โ”‚ 2025-02-05 05:00:00+01:00 โ”‚ Grenoble โ”‚ 1.899999976158142 โ”‚ 81.0 โ”‚ 3.299999952316284 โ”‚
โ”‚ 2025-02-05 06:00:00+01:00 โ”‚ Grenoble โ”‚ 1.399999976158142 โ”‚ 88.0 โ”‚ 4.300000190734863 โ”‚
โ”‚ 2025-02-05 07:00:00+01:00 โ”‚ Grenoble โ”‚ 1.7000000476837158 โ”‚ 87.0 โ”‚ 5.5 โ”‚
โ”‚ 2025-02-05 08:00:00+01:00 โ”‚ Grenoble โ”‚ 1.5 โ”‚ 82.0 โ”‚ 4.699999809265137 โ”‚
โ”‚ 2025-02-05 09:00:00+01:00 โ”‚ Grenoble โ”‚ 1.899999976158142 โ”‚ 80.0 โ”‚ 2.200000047683716 โ”‚
โ”‚ 2025-02-05 10:00:00+01:00 โ”‚ Grenoble โ”‚ 2.9000000953674316 โ”‚ 80.0 โ”‚ 0.800000011920929 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```

```bash
$ laketower -c demo/laketower.yml tables view weather --cols time city temperature_2m --limit 5 --sort-desc time

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ time โ”ƒ city โ”ƒ temperature_2m โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 2025-02-12 00:00:00+01:00 โ”‚ Grenoble โ”‚ 5.099999904632568 โ”‚
โ”‚ 2025-02-12 00:00:00+01:00 โ”‚ Grenoble โ”‚ 5.099999904632568 โ”‚
โ”‚ 2025-02-11 23:00:00+01:00 โ”‚ Grenoble โ”‚ 4.900000095367432 โ”‚
โ”‚ 2025-02-11 23:00:00+01:00 โ”‚ Grenoble โ”‚ 4.900000095367432 โ”‚
โ”‚ 2025-02-11 22:00:00+01:00 โ”‚ Grenoble โ”‚ 4.900000095367432 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```

```bash
$ laketower -c demo/laketower.yml tables view weather --version 1

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ time โ”ƒ city โ”ƒ temperature_2m โ”ƒ relative_humidity_2m โ”ƒ wind_speed_10m โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 2025-01-26 01:00:00+01:00 โ”‚ Grenoble โ”‚ 7.0 โ”‚ 87.0 โ”‚ 8.899999618530273 โ”‚
โ”‚ 2025-01-26 02:00:00+01:00 โ”‚ Grenoble โ”‚ 6.099999904632568 โ”‚ 87.0 โ”‚ 6.199999809265137 โ”‚
โ”‚ 2025-01-26 03:00:00+01:00 โ”‚ Grenoble โ”‚ 6.0 โ”‚ 86.0 โ”‚ 2.700000047683716 โ”‚
โ”‚ 2025-01-26 04:00:00+01:00 โ”‚ Grenoble โ”‚ 6.099999904632568 โ”‚ 82.0 โ”‚ 3.0999999046325684 โ”‚
โ”‚ 2025-01-26 05:00:00+01:00 โ”‚ Grenoble โ”‚ 5.5 โ”‚ 87.0 โ”‚ 3.299999952316284 โ”‚
โ”‚ 2025-01-26 06:00:00+01:00 โ”‚ Grenoble โ”‚ 5.199999809265137 โ”‚ 91.0 โ”‚ 2.200000047683716 โ”‚
โ”‚ 2025-01-26 07:00:00+01:00 โ”‚ Grenoble โ”‚ 4.800000190734863 โ”‚ 86.0 โ”‚ 3.0 โ”‚
โ”‚ 2025-01-26 08:00:00+01:00 โ”‚ Grenoble โ”‚ 4.900000095367432 โ”‚ 83.0 โ”‚ 1.100000023841858 โ”‚
โ”‚ 2025-01-26 09:00:00+01:00 โ”‚ Grenoble โ”‚ 4.0 โ”‚ 92.0 โ”‚ 3.0999999046325684 โ”‚
โ”‚ 2025-01-26 10:00:00+01:00 โ”‚ Grenoble โ”‚ 5.0 โ”‚ 86.0 โ”‚ 6.400000095367432 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
```

#### Query all registered tables

Query any registered tables using DuckDB SQL dialect!

```bash
$ laketower -c demo/laketower.yml tables query "select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather group by day order by day desc limit 3"

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ day โ”ƒ mean_temperature โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 2025-02-12 00:00:00+01:00 โ”‚ 5.099999904632568 โ”‚
โ”‚ 2025-02-11 00:00:00+01:00 โ”‚ 4.833333373069763 โ”‚
โ”‚ 2025-02-10 00:00:00+01:00 โ”‚ 2.1083333243926368 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
3 rows returned
Execution time: 33.72ms
```

Use named parameters within a giving query (note: escape `$` prefixes properly!):

```bash
$ laketower -c demo/laketower.yml tables query "select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather where day between \$start_date and \$end_date group by day order by day desc" -p start_date 2025-01-29 -p end_date 2025-01-31

โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ day โ”ƒ mean_temperature โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 2025-01-31 00:00:00+01:00 โ”‚ 5.683333257834117 โ”‚
โ”‚ 2025-01-30 00:00:00+01:00 โ”‚ 8.900000015894571 โ”‚
โ”‚ 2025-01-29 00:00:00+01:00 โ”‚ 7.770833313465118 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
4 rows returned
Execution time: 30.59ms
```

Export query results to CSV:

```bash
$ laketower -c demo/laketower.yml tables query --output results.csv "select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather group by day order by day desc limit 3"

Query results written to: results.csv
```

#### List saved queries

```bash
$ laketower -c demo/laketower.yml queries list

queries
โ”œโ”€โ”€ all_data
โ””โ”€โ”€ daily_avg_temperature
```

#### Execute saved queries

```bash
$ laketower -c demo/laketower.yml queries view daily_avg_temperature

โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ # โ”ƒ day โ”ƒ avg_temperature โ”ƒ
โ”กโ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 1 โ”‚ 2025-01-26 00:00:00+01:00 โ”‚ 8.0 โ”‚
โ”‚ 2 โ”‚ 2025-01-27 00:00:00+01:00 โ”‚ 13.0 โ”‚
โ”‚ 3 โ”‚ 2025-01-28 00:00:00+01:00 โ”‚ 7.0 โ”‚
โ”‚ 4 โ”‚ 2025-01-29 00:00:00+01:00 โ”‚ 8.0 โ”‚
โ”‚ 5 โ”‚ 2025-01-30 00:00:00+01:00 โ”‚ 9.0 โ”‚
โ”‚ 6 โ”‚ 2025-01-31 00:00:00+01:00 โ”‚ 6.0 โ”‚
โ”‚ 7 โ”‚ 2025-02-01 00:00:00+01:00 โ”‚ 4.0 โ”‚
โ”‚ 8 โ”‚ 2025-02-02 00:00:00+01:00 โ”‚ 4.0 โ”‚
โ”‚ 9 โ”‚ 2025-02-03 00:00:00+01:00 โ”‚ 4.0 โ”‚
โ”‚ 10 โ”‚ 2025-02-04 00:00:00+01:00 โ”‚ 3.0 โ”‚
โ”‚ 11 โ”‚ 2025-02-05 00:00:00+01:00 โ”‚ 3.0 โ”‚
โ”‚ 12 โ”‚ 2025-02-06 00:00:00+01:00 โ”‚ 2.0 โ”‚
โ”‚ 13 โ”‚ 2025-02-07 00:00:00+01:00 โ”‚ 6.0 โ”‚
โ”‚ 14 โ”‚ 2025-02-08 00:00:00+01:00 โ”‚ 7.0 โ”‚
โ”‚ 15 โ”‚ 2025-02-09 00:00:00+01:00 โ”‚ 5.0 โ”‚
โ”‚ 16 โ”‚ 2025-02-10 00:00:00+01:00 โ”‚ 2.0 โ”‚
โ”‚ 17 โ”‚ 2025-02-11 00:00:00+01:00 โ”‚ 5.0 โ”‚
โ”‚ 18 โ”‚ 2025-02-12 00:00:00+01:00 โ”‚ 5.0 โ”‚
โ”œโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ผโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ค
โ”‚ Total โ”‚ - โ”‚ 101.0 โ”‚
โ””โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
18 rows returned
Execution time: 27.02ms
```

Executing a predefined query with parameters (here `start_date` and `end_date`):

```bash
$ laketower -c demo/laketower.yml queries view daily_avg_temperature_params -p start_date 2025-02-01 -p end_date 2025-02-05

โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ณโ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”“
โ”ƒ # โ”ƒ day โ”ƒ avg_temperature โ”ƒ
โ”กโ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ•‡โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”ฉ
โ”‚ 1 โ”‚ 2025-02-01 00:00:00+01:00 โ”‚ 4.0 โ”‚
โ”‚ 2 โ”‚ 2025-02-02 00:00:00+01:00 โ”‚ 4.0 โ”‚
โ”‚ 3 โ”‚ 2025-02-03 00:00:00+01:00 โ”‚ 4.0 โ”‚
โ”‚ 4 โ”‚ 2025-02-04 00:00:00+01:00 โ”‚ 3.0 โ”‚
โ”‚ 5 โ”‚ 2025-02-05 00:00:00+01:00 โ”‚ 3.0 โ”‚
โ”‚ 6 โ”‚ 2025-02-06 00:00:00+01:00 โ”‚ 2.0 โ”‚
โ””โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”ดโ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”€โ”˜
6 rows returned
Execution time: 29.70ms
```

## License

Licensed under [Apache License 2.0](LICENSE)

Copyright (c) 2025 - present Romain Clement / Datalpia