https://github.com/quackscience/duckdb-extension-kafquack
Experimental DuckDB Kafka Consumer Extension
https://github.com/quackscience/duckdb-extension-kafquack
duckdb kafka kafka-consumer redpanda
Last synced: 8 months ago
JSON representation
Experimental DuckDB Kafka Consumer Extension
- Host: GitHub
- URL: https://github.com/quackscience/duckdb-extension-kafquack
- Owner: quackscience
- License: mit
- Created: 2024-12-16T01:40:12.000Z (over 1 year ago)
- Default Branch: main
- Last Pushed: 2025-01-01T20:03:52.000Z (about 1 year ago)
- Last Synced: 2025-01-01T21:18:19.534Z (about 1 year ago)
- Topics: duckdb, kafka, kafka-consumer, redpanda
- Language: C++
- Homepage:
- Size: 22.5 KB
- Stars: 5
- Watchers: 1
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# DuckDB Kafka Extension
This extension can be used to consume Kafka messages from brokers through a DuckDB Table function.
> The extension is a WIP and not functional! Join if you're willing to contribute!
## Examples
#### Basic usage:
```sql
SELECT * FROM kafquack('localhost:9092', 'test-topic', 'test-group');
```
#### Secure usage:
```sql
SELECT * FROM kafquack(
'broker.some.cloud:9092',
'test-topic',
'test-group',
security_protocol := 'SASL_SSL',
sasl_mechanism := 'PLAIN',
username := 'your-key',
password := 'your-secret'
);
```
#### View
```sql
-- Create a view that consumes messages from Kafka
CREATE VIEW kafka_messages AS
SELECT * FROM kafquack('localhost:9092', 'test-topic', 'test-group');
-- Query messages
SELECT * FROM kafka_messages;
```