https://github.com/s0rg/clickhouse-kafka-compose
Example docker-compose with clickhouse and kafka or redpanda as event sources
https://github.com/s0rg/clickhouse-kafka-compose
clickhouse docker docker-compose example kafka redpanda
Last synced: about 14 hours ago
JSON representation
Example docker-compose with clickhouse and kafka or redpanda as event sources
- Host: GitHub
- URL: https://github.com/s0rg/clickhouse-kafka-compose
- Owner: s0rg
- License: mit
- Created: 2023-08-27T20:48:51.000Z (about 2 years ago)
- Default Branch: master
- Last Pushed: 2023-08-30T18:40:10.000Z (about 2 years ago)
- Last Synced: 2025-03-11T17:21:58.791Z (7 months ago)
- Topics: clickhouse, docker, docker-compose, example, kafka, redpanda
- Language: Python
- Homepage:
- Size: 17.6 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
[](https://github.com/s0rg/clickhouse-kafka-compose/blob/master/LICENSE)
# clickhouse-kafka-compose
Example docker-compose with clickhouse and kafka or redpanda as sources
# usage
Start original `kafka`-variant:
```shell
make kafka-up
```
Stop it:
```shell
make kafka-down
```
Stop and purge volumes:
```shell
make kafka-data-down
```
Same for `redpanda`-valiant
```shell
make redpanda-up
make redpanda-down
make redpanda-data-down
```
Send some data (stop it with `C-c`):
```shell
make produce
```
or with `kafkacat`
```shell
kcat -P -b 127.0.0.1:19092 -t event-queue < your-events.json
```
see: [init.sql](initdb.d/00-init.sql) for message structure details.
# exposed ports
Both variants:
- `127.0.0.1:19092` - message broker
- `127.0.0.1:9000` - clickhouse
Redpanda-only:
- `127.0.0.1:8080` - redpanda console (web ui)
# dependencies
Please, make sure you have installed [kafka-python](https://kafka-python.readthedocs.io/en/master/install.html) in order
to use provided `json-producer.py` script:
```shell
pip install kafka-python
```