https://github.com/glassflow/clickhouse-etl
https://github.com/glassflow/clickhouse-etl
Last synced: 2 months ago
JSON representation
- Host: GitHub
- URL: https://github.com/glassflow/clickhouse-etl
- Owner: glassflow
- Created: 2025-04-22T08:22:30.000Z (2 months ago)
- Default Branch: main
- Last Pushed: 2025-04-22T12:43:14.000Z (2 months ago)
- Last Synced: 2025-04-22T12:55:20.622Z (2 months ago)
- Language: TypeScript
- Size: 458 KB
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 1
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
- awesome-clickhouse - glassflow/clickhouse-etl - GlassFlow is an open-source ETL tool for real-time data processing from Kafka to ClickHouse with features like deduplication and temporal joins. (Integrations / ETL and Data Processing)
README
## Clickhouse-etl-internal
This repository houses the components required for running a complete pipeline
from Kafka to Clickhouse.
To test in development mode execute:
`make run`
Ensure any credentials required are set in `.env` file. E.g. AWS client and secret IDs for connecting with MSK Kafka.
An example [docker-compose.yaml](./docker-compose.yaml) has been added to show how this would eventually work for end-users / clients.
All functionality related to the API resides under [glassflow-api](./glassflow-api) and development related only to the API can be continued there.
All functionality related to the Kafka to Nats bridge reside under [nats-kafka-bridge](./nats-kafka-bridge) and all related development can be continued there.