https://github.com/gravitycloudai/clickhouse-onprem
Clickhouse Multi-Replica Starter Repository
https://github.com/gravitycloudai/clickhouse-onprem
clickhouse docker docker-compose sharding zookeeper
Last synced: over 1 year ago
JSON representation
Clickhouse Multi-Replica Starter Repository
- Host: GitHub
- URL: https://github.com/gravitycloudai/clickhouse-onprem
- Owner: GravityCloudAI
- Created: 2024-08-23T06:06:36.000Z (almost 2 years ago)
- Default Branch: main
- Last Pushed: 2024-08-23T06:25:02.000Z (almost 2 years ago)
- Last Synced: 2025-01-17T14:39:31.886Z (over 1 year ago)
- Topics: clickhouse, docker, docker-compose, sharding, zookeeper
- Homepage:
- Size: 33.2 KB
- Stars: 0
- Watchers: 0
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
Awesome Lists containing this project
README
# Clickhouse Cluster On-Prem Deployment
This docker compose is to enable Clickhouse on-prem deployment with Zookeeper and Multi-Replica Clickhouse with storage as Wasabi/S3 buckets
## Why Wasabi?
Wasabi is extremely low-cost compared to AWS S3 and does not charge any egress fee!
## How does this work?
1. Zookeeper syncs data between multiple clickhouse replicas
2. Clickhouse by defaults uses local disk to store the data and every 10s sync to Wasabi/S3.
> [!TIP]
> You can also update the storage config tiered for s3 as "cold" and use a TTL table in clickhouse to sync every few seconds.
## How to run?
`docker compose up -d`
## Configurations
1. Change your Bucket endpoint inside the `clickhouse-storage.xml`
```xml
https://s3.wasabisys.com/WASABI_BUCKET_NAME/data/
WASABI_ACCESS_KEY_ID
WASABI_SECRET_ACCESS_KEY
```
2. Set the Username and Password for Clickhouse in `docker-compose.yml`
```yaml
environment:
- CLICKHOUSE_USER=
- CLICKHOUSE_PASSWORD=
```
3. Add multi-replicas/shards in `cluster-config.xml`
```xml
clickhouse-2
9000
```