Ecosyste.ms: Awesome
An open API service indexing awesome lists of open source software.
https://github.com/quackscience/duckdb-extension-clickhouse-sql
DuckDB Community Extension implementing ClickHouse SQL Dialect macros for DuckDB
https://github.com/quackscience/duckdb-extension-clickhouse-sql
chdb chsql clickhouse clickhouse-database clickhouse-server duckdb duckdb-community duckdb-extension duckdb-server quackpipe sql
Last synced: 14 days ago
JSON representation
DuckDB Community Extension implementing ClickHouse SQL Dialect macros for DuckDB
- Host: GitHub
- URL: https://github.com/quackscience/duckdb-extension-clickhouse-sql
- Owner: quackscience
- License: mit
- Created: 2024-07-06T10:30:09.000Z (4 months ago)
- Default Branch: main
- Last Pushed: 2024-10-23T21:53:32.000Z (22 days ago)
- Last Synced: 2024-10-24T00:17:42.554Z (22 days ago)
- Topics: chdb, chsql, clickhouse, clickhouse-database, clickhouse-server, duckdb, duckdb-community, duckdb-extension, duckdb-server, quackpipe, sql
- Language: C++
- Homepage: https://community-extensions.duckdb.org/extensions/chsql.html
- Size: 151 KB
- Stars: 34
- Watchers: 2
- Forks: 2
- Open Issues: 2
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DuckDB ClickHouse SQL extension
The DuckDB [chsql](https://community-extensions.duckdb.org/extensions/chsql.html) community extension implements popular **ClickHouse SQL** syntax macros and functions,
making it easier for users to transition between the two database systems ⭐ designed for [Quackpipe](https://github.com/metrico/quackpipe)
## Installation
**chsql** is distributed as a [DuckDB Community Extension](https://github.com/duckdb/community-extensions) and can be installed using SQL:
```sql
INSTALL chsql FROM community;
LOAD chsql;
```
If you previously installed the `chsql` extension, upgrade using the FORCE command
```sql
FORCE INSTALL chsql FROM community;
LOAD chsql;
```
## Usage Examples
Once installed, the [macro functions](https://community-extensions.duckdb.org/extensions/chsql.html#added-functions) provided by the extension can be used just like built-in functions.
Here's a random example out of 100s using the `IPv4StringToNum` and `IPv4NumToString` functions:
```sql
D INSTALL chsql FROM community;
D LOAD chsql;
D SELECT IPv4StringToNum('127.0.0.1'), IPv4NumToString(2130706433);
┌──────────────────────────────┬─────────────────────────────┐
│ ipv4stringtonum('127.0.0.1') │ ipv4numtostring(2130706433) │
│ int32 │ varchar │
├──────────────────────────────┼─────────────────────────────┤
│ 2130706433 │ 127.0.0.1 │
└──────────────────────────────┴─────────────────────────────┘
```
### Remote Queries
The built-in `ch_scan` function can be used to query remote ClickHouse servers using the HTTP/s API
```sql
D SELECT * FROM ch_scan("SELECT number * 2 FROM numbers(10)", "https://play.clickhouse.com");
```
## Supported Functions
👉 The [list of supported aliases](https://community-extensions.duckdb.org/extensions/chsql.html#added-functions) is available on the [dedicated extension page](https://community-extensions.duckdb.org/extensions/chsql.html)
👉 The combined list of [supported functions](https://quackpipe.fly.dev/?user=default#TE9BRCBjaHNxbDsgU0VMRUNUIERJU1RJTkNUIE9OKGZ1bmN0aW9uX25hbWUpIGZ1bmN0aW9uX25hbWUgYXMgbmFtZQpGUk9NIGR1Y2tkYl9mdW5jdGlvbnMoKSBXSEVSRSBuYW1lIElOIChTRUxFQ1QgbmFtZSBGUk9NIGNoX3NjYW4oJyBTRUxFQ1QgbmFtZSBGUk9NIHN5c3RlbS5mdW5jdGlvbnMnLCdodHRwczovL3BsYXkuY2xpY2tob3VzZS5jb20nKSBPUkRFUiBCWSBuYW1lKSBPUkRFUiBCWSBuYW1lOw==) can be obtained using an [SQL Join](https://quackpipe.fly.dev/?user=default#TE9BRCBjaHNxbDsgU0VMRUNUIERJU1RJTkNUIE9OKGZ1bmN0aW9uX25hbWUpIGZ1bmN0aW9uX25hbWUgYXMgbmFtZQpGUk9NIGR1Y2tkYl9mdW5jdGlvbnMoKSBXSEVSRSBuYW1lIElOIChTRUxFQ1QgbmFtZSBGUk9NIGNoX3NjYW4oJyBTRUxFQ1QgbmFtZSBGUk9NIHN5c3RlbS5mdW5jdGlvbnMnLCdodHRwczovL3BsYXkuY2xpY2tob3VzZS5jb20nKSBPUkRFUiBCWSBuYW1lKSBPUkRFUiBCWSBuYW1lOw==)
## Motivation
> Why is the DuckDB + chsql combo fun and useful
✔ DuckDB SQL is awesome and full of great functions.
✔ ClickHouse SQL is awesome and full of great functions.
✔ The DuckDB library is ~51M and modular. Can LOAD extensions.
❌ The ClickHouse monolith is ~551M and growing. No extensions.
✔ DuckDB is open source and _protected by a no-profit foundation._
❌ ClickHouse is open core and _controlled by for-profit corporation._
✔ DuckDB embedded is fast, mature and elegantly integrated in many languages.
❌ chdb is still experimental, unstable and _currently only supports Python_.
## Development
The extension is automatically build and distributed. This section is only required for development.
### Managing dependencies
DuckDB extensions uses VCPKG for dependency management. Enabling VCPKG is very simple: follow the [installation instructions](https://vcpkg.io/en/getting-started) or just run the following:
```shell
git clone https://github.com/Microsoft/vcpkg.git
./vcpkg/bootstrap-vcpkg.sh
export VCPKG_TOOLCHAIN_PATH=`pwd`/vcpkg/scripts/buildsystems/vcpkg.cmake
```
Note: VCPKG is only required for extensions that want to rely on it for dependency management. If you want to develop an extension without dependencies, or want to do your own dependency management, just skip this step. Note that the example extension uses VCPKG to build with a dependency for instructive purposes, so when skipping this step the build may not work without removing the dependency.
### Build steps
Clone the repository and fetch all required submodules:
```sh
git submodule update --init
```
Build the extension:
```sh
GEN=ninja make
```
The main binaries that will be built are:
```sh
./build/release/duckdb
./build/release/test/unittest
./build/release/extension/dynamic_sql_clickhouse/dynamic_sql_clickhouse.duckdb_extension
```
- `duckdb` is the binary for the duckdb shell with the extension code automatically loaded.
- `unittest` is the test runner of duckdb. Again, the extension is already linked into the binary.
- `dynamic_sql_clickhouse.duckdb_extension` is the loadable binary as it would be distributed.
## Running the extension
To run the extension code, simply start the shell with `./build/release/duckdb`.
Now we can use the features from the extension directly in DuckDB. See `/tests/sql` for a list of supported functions.
## Running the tests
Different tests can be created for DuckDB extensions. The primary way of testing DuckDB extensions should be the SQL tests in `./test/sql`. These SQL tests can be run using:
```sh
make test
```
###### Disclaimer
> DuckDB ® is a trademark of DuckDB Foundation. ClickHouse® is a trademark of ClickHouse Inc. All trademarks, service marks, and logos mentioned or depicted are the property of their respective owners. The use of any third-party trademarks, brand names, product names, and company names is purely informative or intended as parody and does not imply endorsement, affiliation, or association with the respective owners.