{"id":20674758,"url":"https://github.com/gravitycloudai/clickhouse-onprem","last_synced_at":"2025-03-10T18:27:39.483Z","repository":{"id":254394391,"uuid":"846401309","full_name":"GravityCloudAI/clickhouse-onprem","owner":"GravityCloudAI","description":"Clickhouse Multi-Replica Starter Repository","archived":false,"fork":false,"pushed_at":"2024-08-23T06:25:02.000Z","size":34,"stargazers_count":0,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"main","last_synced_at":"2025-01-17T14:39:31.886Z","etag":null,"topics":["clickhouse","docker","docker-compose","sharding","zookeeper"],"latest_commit_sha":null,"homepage":"","language":null,"has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/GravityCloudAI.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-08-23T06:06:36.000Z","updated_at":"2024-08-23T06:25:05.000Z","dependencies_parsed_at":"2024-08-23T07:26:56.492Z","dependency_job_id":"f090ecc1-ac0e-4a26-8531-a075f6bb5f6c","html_url":"https://github.com/GravityCloudAI/clickhouse-onprem","commit_stats":null,"previous_names":["gravitycloudai/clickhouse-onprem"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GravityCloudAI%2Fclickhouse-onprem","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GravityCloudAI%2Fclickhouse-onprem/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GravityCloudAI%2Fclickhouse-onprem/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/GravityCloudAI%2Fclickhouse-onprem/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/GravityCloudAI","download_url":"https://codeload.github.com/GravityCloudAI/clickhouse-onprem/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":242901627,"owners_count":20203976,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["clickhouse","docker","docker-compose","sharding","zookeeper"],"created_at":"2024-11-16T21:07:17.033Z","updated_at":"2025-03-10T18:27:39.452Z","avatar_url":"https://github.com/GravityCloudAI.png","language":null,"funding_links":[],"categories":[],"sub_categories":[],"readme":"# Clickhouse Cluster On-Prem Deployment\n\nThis docker compose is to enable Clickhouse on-prem deployment with Zookeeper and Multi-Replica Clickhouse with storage as Wasabi/S3 buckets\n\n## Why Wasabi?\nWasabi is extremely low-cost compared to AWS S3 and does not charge any egress fee!\n\n## How does this work?\n\n1. Zookeeper syncs data between multiple clickhouse replicas\n2. Clickhouse by defaults uses local disk to store the data and every 10s sync to Wasabi/S3.\n\u003e [!TIP]\n\u003e You can also update the storage config tiered for s3 as \"cold\" and use a TTL table in clickhouse to sync every few seconds.\n\n## How to run?\n`docker compose up -d`\n\n## Configurations\n1. Change your Bucket endpoint inside the `clickhouse-storage.xml`\n```xml\n    \u003cendpoint\u003ehttps://s3.wasabisys.com/WASABI_BUCKET_NAME/data/\u003c/endpoint\u003e\n    \u003caccess_key_id\u003eWASABI_ACCESS_KEY_ID\u003c/access_key_id\u003e\n    \u003csecret_access_key\u003eWASABI_SECRET_ACCESS_KEY\u003c/secret_access_key\u003e\n```\n2. Set the Username and Password for Clickhouse in `docker-compose.yml`\n```yaml\n    environment:\n      - CLICKHOUSE_USER=\u003cUSERNAME\u003e\n      - CLICKHOUSE_PASSWORD=\u003cPASSWORD\u003e\n```\n3. Add multi-replicas/shards in `cluster-config.xml`\n```xml\n    \u003cshard\u003e\n        \u003creplica\u003e\n            \u003chost\u003eclickhouse-2\u003c/host\u003e\n            \u003cport\u003e9000\u003c/port\u003e\n        \u003c/replica\u003e\n    \u003c/shard\u003e\n```\n\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgravitycloudai%2Fclickhouse-onprem","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fgravitycloudai%2Fclickhouse-onprem","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fgravitycloudai%2Fclickhouse-onprem/lists"}