{"id":17720389,"url":"https://github.com/kow3ns/kubernetes-zookeeper","last_synced_at":"2025-03-14T04:31:02.983Z","repository":{"id":21824287,"uuid":"88918231","full_name":"kow3ns/kubernetes-zookeeper","owner":"kow3ns","description":"This project contains tools to facilitate the deployment of Apache ZooKeeper on Kubernetes.","archived":false,"fork":false,"pushed_at":"2022-04-27T19:48:13.000Z","size":75,"stargazers_count":141,"open_issues_count":18,"forks_count":173,"subscribers_count":10,"default_branch":"master","last_synced_at":"2023-11-07T15:20:48.406Z","etag":null,"topics":["kubernetes","zookeeper"],"latest_commit_sha":null,"homepage":"https://kow3ns.github.io/kubernetes-zookeeper/","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/kow3ns.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2017-04-20T23:09:40.000Z","updated_at":"2023-10-11T08:02:46.000Z","dependencies_parsed_at":"2022-07-25T13:00:06.721Z","dependency_job_id":null,"html_url":"https://github.com/kow3ns/kubernetes-zookeeper","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kow3ns%2Fkubernetes-zookeeper","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kow3ns%2Fkubernetes-zookeeper/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kow3ns%2Fkubernetes-zookeeper/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/kow3ns%2Fkubernetes-zookeeper/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/kow3ns","download_url":"https://codeload.github.com/kow3ns/kubernetes-zookeeper/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":243526451,"owners_count":20305108,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["kubernetes","zookeeper"],"created_at":"2024-10-25T15:26:46.993Z","updated_at":"2025-03-14T04:30:57.975Z","avatar_url":"https://github.com/kow3ns.png","language":"Shell","funding_links":[],"categories":["Shell"],"sub_categories":[],"readme":"# Kubernetes ZooKeeper\nThis project contains tools to facilitate the deployment of \n[Apache ZooKeeper](https://zookeeper.apache.org/) on \n[Kubernetes](http://kubernetes.io/) using \n[StatefulSets](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/). \nIt requires Kubernetes 1.7 or greater.\n\n## Limitations\n1. Scaling is not currently supported. An ensemble's membership can not be updated in a safe way in \nZooKeeper 3.4.10 (The current stable release).\n1. Observers are currently not supported. Contributions are welcome.\n1. Persistent Volumes must be used. emptyDirs will likely result in a loss of data.\n\n## ZooKeeper Docker Image \nThe [docker](docker) directory contains the [Makefile](docker/Makefile) for a [Docker image](docker/Dockerfile) that \nruns a ZooKeeper server using some custom [scripts](docker/scripts/README.md).\n\n## Manifests\nThe [manifests](manifests) directory contains server Kubernetes [manifests](manifests/README.md) that can be used for \ndemonstration purposes or production deployments. If you primarily deploy manifests directly you can modify any of \nthese to fit your use case.\n\n## Helm\nThe [helm](helm/zookeeper) directory contains a [helm repository](helm/zookeeper/README.md) that deploys a ZooKeeper \nensemble.\n\n\n## Administration and Configuration\nRegardless of whether you use manifests or helm to deploy your ZooKeeper ensemble, there are some common administration \nand configuration items that you should be aware of.\n\n### Ensemble Size\nAs notes in the [limitation](#limitations) section, ZooKeeper membership can't be dynamically configured using the \nlatest stable version. You need to select an ensemble size that suites your use case. For demonstration purposes, or if \nyou are willing to tolerate at most one planned or unplanned failure, you should select an ensemble size of 3. This is \ndone by setting the `spec.replicas` field of the StatefulSet to 3,\n\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n  serviceName: zk-hs\n  replicas: 3\n  ...\n```\n\nand passing in 3 as `--servers` parameter to the `start-zookeeper` script.\n\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n...\n        command:\n        - sh\n        - -c\n        - \"start-zookeeper \\\n          --servers=3 \\\n...\n```\nFor production use cases, 5 servers may be desirable. This allows you to tolerate one planned **and** one unplanned \nfailure.\n\n### Memory\nWhile ZooKeeper periodically snapshots all of its data to its data directory, the entire working data set must fit on \nheap. The `--heap` parameter of the `start-zookeeper` script controls the heap size of the ZooKeeper servers, \n\n```yaml \napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\n...\n        command:\n        - sh\n        - -c\n        - \"start-zookeeper \\\n...\n          --heap=512M \\\n...\n```\n\nand the `spec.template.containers[0].resources.requests.memory` controls the memory allocated to the JVM process.\n\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n ...\n      containers:\n      - name: kubernetes-zookeeper\n        imagePullPolicy: Always\n        image: \"gcr.io/google_containers/kubernetes-zookeeper:1.0-3.4.10\"\n        resources:\n          requests:\n            memory: \"1Gi\"\n...\n```\n\nYou should probably not use heap sizes larger than 8 GiB. For production deployments you should consider setting the \nrequested memory to the maximum of 2 GiB and 1.5 times the size of the configured JVM heap.\n\n### CPUs\nZooKeeper is not a CPU intensive application. For a production deployment you should start with 2 CPUs and adjust as \nnecessary. For a demonstration deployment, you can set the CPUs as low as 0.5. The amount of CPU is configured by \nsetting the StatefulSet's `spec.template.containers[0].resources.requests.cpus`.\n\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n...\n      containers:\n      - name: kubernetes-zookeeper\n        imagePullPolicy: Always\n        image: \"gcr.io/google_containers/kubernetes-zookeeper:1.0-3.4.10\"\n        resources:\n          requests:\n            memory: \"4Gi\"\n            cpu: \"2\"\n...\n```\n\n### Networking\nThe Headless Service that controls the domain of the ensemble must have two ports. The `sever` port is used for \ninter-server communication, and the `leader-election` port is used to perform leader election.\n\n```yaml\napiVersion: v1\nkind: Service\nmetadata:\n  name: zk-hs\n  labels:\n    app: zk\nspec:\n  ports:\n  - port: 2888\n    name: server\n  - port: 3888\n    name: leader-election\n  clusterIP: None\n  selector:\n    app: zk\n```\n\nThese ports must correspond to the container ports in the StatefulSet's `.spec.template` and the parameters passed to \nthe `start-zookeeper` script.\n\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n  serviceName: zk-hs\n  replicas: 3\n  podManagementPolicy: Parallel\n  updateStrategy:\n    type: RollingUpdate\n  template:\n...\n        ports:\n        - containerPort: 2181\n          name: client\n        - containerPort: 2888\n          name: server\n        - containerPort: 3888\n          name: leader-election\n        command:\n        - sh\n        - -c\n        - \"start-zookeeper \\\n...\n          --election_port=3888 \\\n          --server_port=2888 \\\n...\n```\n\nThe Service used to load balance client connections has one port. \n\n```yaml \napiVersion: v1\nkind: Service\nmetadata:\n  name: zk-cs\n  labels:\n    app: zk\nspec:\n  ports:\n  - port: 2181\n    name: client\n  selector:\n    app: zk\n```\n\nThe `client` port must correspond to the container port specified in the StatefulSet's `.spec.template` and the \nparameter passed to the `start-zookeeper` script.\n\n```yaml\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n  serviceName: zk-hs\n  replicas: 3\n  podManagementPolicy: Parallel\n  updateStrategy:\n    type: RollingUpdate\n  template:\n...\n        ports:\n        - containerPort: 2181\n          name: client\n        - containerPort: 2888\n          name: server\n        - containerPort: 3888\n          name: leader-election\n        command:\n        - sh\n        - -c\n        - \"start-zookeeper \\\n...\n          --client_port=2181 \\\n...\n```\n\n### Storage\nCurrently, the use of Persistent Volumes to provide durable, network attached storage is mandatory. **If you use the \nprovided image with emptyDirs, you will likely suffer a data loss.** The `storage` field of the StatefulSet's \n`spec.volumeClaimTemplates` controls the storage the amount of storage allocated.\n\n```yaml\n  volumeClaimTemplates:\n    - metadata:\n      name: datadir\n    spec:\n      accessModes: [ \"ReadWriteOnce\" ]\n      resources:\n        requests:\n          storage: 20Gi\n```\n\nThe `volumeMounts` in the StatefulSet's `spec.template` control the mount point of the PersistentVolumes requested by \nthe PersistentVolumeClaims,\n\n```yaml\n  volumeMounts:\n  - name: datadir\n    mountPath: /var/lib/zookeeper\n```\n\nand the parameters passed to the `start-zookeeper` script instruct the ZooKeeper server to use the PersistentVolume \nbacked directory for its snapshots and write ahead log.\n\n\n    --data_dir          The directory where the ZooKeeper process will store its\n                        snapshots. The default is /var/lib/zookeeper/data. This \n                        directory must be backed by a persistent volume.\n\n    --data_log_dir      The directory where the ZooKeeper process will store its \n                        write ahead log. The default is \n                        /var/lib/zookeeper/data/log. This directory must be \n                        backed by a persistent volume.\n\n\nNote that, because we use network attached storage there is no benefit to using multiple PersistentVolumes to \nsegregate the snapshot and write ahead log to separate storage media.\n\n### ZooKeeper Time\nZooKeeper does not use wall clock time. Rather, it uses internal ticks that are based on an elapsed number of \nmilliseconds. The various timeouts for the ZooKeeper ensemble can be controlled by the parameters passed to the \n`start-zookeeper` script.\n\n    --tick_time         The length of a ZooKeeper tick in ms. The default is \n                        2000.\n\n    --init_limit        The number of Ticks that an ensemble member is allowed \n                        to perform leader election. The default is 10.\n\n    --sync_limit        The maximum session timeout that the ensemble will \n                        allows a client to request. The default is 5.\n\n    --max_session_timeout The maximum time in milliseconds for a client session \n                        timeout. The default value is 2 * tick time.\n\n    --min_session_timeout The minimum time in milliseconds for a client session \n                        timeout. The default value is 20 * tick time.\n\nIf you notice an excessive number of client timeouts or leader elections, you can tune these knobs as appropriate to \nincrease stability, at the cost of slower failure detection.\n\n### Snapshot Retention \nBy default, ZooKeeper will never delete its own snapshots, though it will compact its write ahead log. You must either \nimplement your own mechanism for retaining and removing snapshots, or configure the snapshot retention via the \n`start-zookeeper` script.\n\n    --snap_retain_count The maximum number of snapshots the ZooKeeper process \n                        will retain if purge_interval is greater than 0. The \n                        default is 3.\n\n    --purge_interval    The number of hours the ZooKeeper process will wait \n                        between purging its old snapshots. If set to 0 old \n                        snapshots will never be purged. The default is 0.\n                        \n### Pod Management Policy \nZooKeeper is not sensitive to the order in which Pods are started. All Pods in the StatefulSet may be launched and \nterminated in parallel. This is accomplished by setting the `spec.podManagementPolicy` to `Parallel`.\n\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n...\n  podManagementPolicy: Parallel\n...\n```\n\n### Update Strategy \nThe Update Strategy for the StatefulSet is set to `RollingUpdate`. This enables the rolling update feature for \nStatefulSets in Kubernetes 1.7, and it allows for modifications to the StatefulSet to be propagated to its Pods.\n\n```yaml\napiVersion: apps/v1beta1\nkind: StatefulSet\nmetadata:\n  name: zk\nspec:\n  ...\n  updateStrategy: \n    type: RollingUpdate\n  ...\n```\n\n### Liveness and Readiness\nThe `zookeeper-ready` script can be used to check the liveness and readiness of ZooKeeper process. The example below \ndemonstrates how to configure liveness and readiness probes for the Pods in the StatefulSet.\n\n```yaml\n  readinessProbe:\n    exec:\n      command:\n      - sh\n      - -c\n      - \"zookeeper-ready 2181\"\n    initialDelaySeconds: 15 \n    timeoutSeconds: 5\n  livenessProbe:\n    exec:\n      command:\n      - sh\n      - -c\n      - \"zookeeper-ready 2181\"\n    initialDelaySeconds: 15\n    timeoutSeconds: 5\n```\n\n### Spreading\nAn anti-affinity rule is specified to ensure that the ZooKeeper servers are spread across nodes in the Kubernetes \ncluster. This makes the ensemble resilient to node failures.\n\n```yaml\naffinity:\n    podAntiAffinity:\n      requiredDuringSchedulingIgnoredDuringExecution:\n        - labelSelector:\n            matchExpressions:\n              - key: \"app\"\n                operator: In\n                values: \n                - zk\n          topologyKey: \"kubernetes.io/hostname\"\n```\n\n### Logging \nThe Log Level configuration may be modified via the log_level flag supplied to the `start-zookeeper` script. However,\nthe location of the log output is not modifiable. The ZooKeeper process must be run in the foreground, and the log \ninformation will be shipped to stdout. This is considered to be a best practice for containerized applications, and it \nallows users to make use of the log rotation and retention infrastructure that already exists for K8s.\n\n### Metrics \nThe `zookeeper-metrics` script can be used to retrieve metrics from the ZooKeeper process and print them to stdout. A \nrecurring Kubernetes job can be used to collect these metrics and provide them to a collector.\n\n```bash\nbash$ kubectl exec zk-0 zookeeper-metrics\nzk_version\t3.4.10-1757313, built on 08/23/2016 06:50 GMT\nzk_avg_latency\t0\nzk_max_latency\t0\nzk_min_latency\t0\nzk_packets_received\t21\nzk_packets_sent\t20\nzk_num_alive_connections\t1\nzk_outstanding_requests\t0\nzk_server_state\tfollower\nzk_znode_count\t4\nzk_watch_count\t0\nzk_ephemerals_count\t0\nzk_approximate_data_size\t27\nzk_open_file_descriptor_count\t39\nzk_max_file_descriptor_count\t1048576\n\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkow3ns%2Fkubernetes-zookeeper","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkow3ns%2Fkubernetes-zookeeper","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkow3ns%2Fkubernetes-zookeeper/lists"}