{"id":30064566,"url":"https://github.com/agirish/drill-helm-charts","last_synced_at":"2025-08-08T05:15:45.939Z","repository":{"id":92465508,"uuid":"243808437","full_name":"Agirish/drill-helm-charts","owner":"Agirish","description":"Helm Charts to Deploy Apache Drill on Kubernetes","archived":false,"fork":false,"pushed_at":"2024-01-05T07:21:56.000Z","size":196,"stargazers_count":14,"open_issues_count":6,"forks_count":12,"subscribers_count":2,"default_branch":"master","last_synced_at":"2024-01-05T08:28:58.813Z","etag":null,"topics":["apache-drill","apache-zookeeper","drill","drill-clusters","drill-helm-charts","drill-pods","helm","helm-chart","helm-charts","kubernetes","kubernetes-cluster","zookeeper"],"latest_commit_sha":null,"homepage":null,"language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/Agirish.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null}},"created_at":"2020-02-28T16:37:51.000Z","updated_at":"2024-01-05T08:29:00.327Z","dependencies_parsed_at":"2023-06-02T13:15:46.580Z","dependency_job_id":null,"html_url":"https://github.com/Agirish/drill-helm-charts","commit_stats":null,"previous_names":[],"tags_count":0,"template":null,"template_full_name":null,"purl":"pkg:github/Agirish/drill-helm-charts","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agirish%2Fdrill-helm-charts","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agirish%2Fdrill-helm-charts/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agirish%2Fdrill-helm-charts/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agirish%2Fdrill-helm-charts/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/Agirish","download_url":"https://codeload.github.com/Agirish/drill-helm-charts/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/Agirish%2Fdrill-helm-charts/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":269366951,"owners_count":24405278,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-08-08T02:00:09.200Z","response_time":72,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-drill","apache-zookeeper","drill","drill-clusters","drill-helm-charts","drill-pods","helm","helm-chart","helm-charts","kubernetes","kubernetes-cluster","zookeeper"],"created_at":"2025-08-08T05:15:41.861Z","updated_at":"2025-08-08T05:15:45.923Z","avatar_url":"https://github.com/Agirish.png","language":"Shell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Helm Charts for Apache Drill\n\n## Overview\nThis repository contains a collection of files that can be used to deploy [Apache Drill](http://drill.apache.org/) on Kubernetes using Helm Charts. Supports single-node and [cluster](http://drill.apache.org/docs/installing-drill-in-distributed-mode/) modes.\n\n#### What are Helm and Charts?\n[Helm](https://helm.sh/) is a package manager for [Kubernetes](https://kubernetes.io/). [Charts](https://helm.sh/docs/topics/charts/) are a packaging format in Helm that can simplify deploying Kubernetes applications such as Drill Clusters.\n\n## Pre-requisites\n\n- A Kubernetes Cluster (this project is tested on [GKE](https://cloud.google.com/kubernetes-engine/) clusters)\n- [Helm](https://github.com/helm/helm#install) version 3 or greater\n- [Kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/) version 1.16.0 or greater\n\n## Chart Structure\nDrill Helm charts are organized as a collection of files inside of the `drill` directory. As Drill depends on Zookeeper for cluster co-ordination, a zookeeper chart is inside the [dependencies](drill/charts) directory. The Zookeeper chart follows a similar structure as the Drill chart.\n```\ndrill/   \n  Chart.yaml    # A YAML file with information about the chart\n  values.yaml   # The default configuration values for this chart\n  charts/       # A directory containing the ZK charts\n  templates/    # A directory of templates, when combined with values, will generate valid Kubernetes manifest files\n  ```\n### Templates\nHelm Charts contain `templates` which are used to generate Kubernetes manifest files. These are YAML-formatted resource descriptions that Kubernetes can understand. These templates contain 'variables', values for which  are picked up from the `values.yaml` file.\n\nDrill Helm Charts contain the following templates:\n```\ndrill/\n  ...\n  templates/\n    drill-rbac-*.yaml       # To enable RBAC for the Drill app\n    drill-service.yaml      # To create a Drill Service\n    drill-web-service.yaml  # To expose Drill's Web UI externally using a LoadBalancer. Works on cloud deployments only. \n    drill-statefulset.yaml  # To create a Drill cluster\n  charts/\n    zookeeper/\n      ...\n      templates/\n        zk-rbac.yaml        # To enable RBAC for the ZK app\n        zk-service.yaml     # To create a ZK Service\n        zk-statefulset.yaml # To create a ZK cluster. Currently only a single-node ZK (1 replica) is supported\n```\n### Values\nHelm Charts use `values.yaml` for providing default values to 'variables' used in the chart templates. These values may be overridden either by editing the `values.yaml` file or during `helm install`. For example, such as the namespace, number of drillbits and more to the `template` files\n\nPlease refer to the [values.yaml](drill/values.yaml) file for details on default values for Drill Helm Charts.\n\n## Usage\n### Install\n\n#### Simple Deploy\nDrill Helm Charts can be deployed as simply as follows: \n```\n# helm install \u003cUNIQUE_NAME\u003e drill/\nhelm install drill1 drill/\n```\n#### Override Drill Config\nOverridding the following two Drill configuration files is currently supported:\n- `drill/conf/drill-env.sh`\n- `drill/conf/drill-override.conf`\n\nPlease edit/replace them as needed. Please do NOT rename/delete.\n\nOnce the above configuration files are ready, please create the `drill-config-cm` configMap to upload them to Kubernetes. When a Drill chart is deployed, the files contained within this configMap will be downloaded to each container and used by the drill-bit process during start-up.\n```\n./scripts/createCM.sh\n```\nor\n```\nkubectl create configmap drill-config-cm --from-file=./drill/conf/drill-override.conf --from-file=./drill/conf/drill-env.sh\n```\nEnable config overriding by editing the drillConf section in `drill/values.yaml` file.\n#### Using Namespaces to Deploy Multple Drill Clusters\nKubernetes [Namespaces](https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/) can be used when more that one Drill Cluster needs to be created. We use the `default` namespace by default. To create a namespace, use the following command:\n```\n# kubectl create namespace \u003cNAMESPACE_NAME\u003e\nkubectl create namespace namespace2\n```\nThis NAMESPACE_NAME needs to be provided in `drill/values.yaml`. Or can be provided in the `helm install` command as follows:\n```\n# helm install \u003cHELM_INSTALL_RELEASE_NAME\u003e drill/ --set global.namespace=\u003cNAMESPACE_NAME\u003e\nhelm install drill2 drill/ --set global.namespace=namespace2 --set drill.id=drillcluster2\n```\nNote that installing the Drill Helm Chart also installs the dependent Zookeeper chart. So with current design, for each instance of a Drill cluster includes a single-node Zookeeper.\n#### List Pods\n```\n$ kubectl get pods\nNAME                       READY   STATUS    RESTARTS   AGE\ndrillcluster1-drillbit-0   1/1     Running   0          51s\ndrillcluster1-drillbit-1   1/1     Running   0          51s\nzk-0                       1/1     Running   0          51s\n\n$ kubectl get pods -n namespace2\nNAME                       READY   STATUS    RESTARTS   AGE\ndrillcluster2-drillbit-0   1/1     Running   0          47s\ndrillcluster2-drillbit-1   1/1     Running   0          47s\nzk-0                       1/1     Running   0          47s\n```\n#### List Services\n```\n$ kubectl get services\nNAME                    TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                                  AGE\ndrill-service           ClusterIP      10.15.242.217   \u003cnone\u003e            8047/TCP,31010/TCP,31011/TCP,31012/TCP   3m49s\ndrillcluster1-web-svc   LoadBalancer   10.15.250.97    34.71.235.149     8047:30019/TCP,31010:32513/TCP           3m49s\nzk-service              ClusterIP      10.15.243.254   \u003cnone\u003e            2181/TCP,2888/TCP,3888/TCP               3m49s\n\n$ kubectl get services -n namespace2\nNAME                    TYPE           CLUSTER-IP      EXTERNAL-IP       PORT(S)                                  AGE\ndrill-service           ClusterIP      10.15.246.116   \u003cnone\u003e            8047/TCP,31010/TCP,31011/TCP,31012/TCP   2m9s\ndrillcluster2-web-svc   LoadBalancer   10.15.249.214   130.211.220.239   8047:30019/TCP,31010:32513/TCP           2m9s\nzk-service              ClusterIP      10.15.246.218   \u003cnone\u003e            2181/TCP,2888/TCP,3888/TCP               2m9s\n```\n#### Access Drill Web UI\nFor cloud based deployments, we create a LoadBalancer type service with an EXTERNAL_IP address. Use this along with the HTTP port to access the Drill Web UI on a browser. Note that the URL is similar to a proxy which internally redirects to the Drill Web UI of any Drill pod. \n```\n# http://EXTERNAL_IP:PORT\nhttp://130.211.220.239:8047\n```\n![Drill Web UI via LoadBalancer for GKE](docs/images/apacheDrillExternalWebUI.jpg)\n\n### Upgrading Drill Charts\nCurrently only scaling up/down the number of Drill pods is supported as part of Helm Chart upgrades. To resize a Drill Cluster, edit the `drill/values.yaml` file and apply the changes as below:\n```\n# helm upgrade \u003cHELM_INSTALL_RELEASE_NAME\u003e drill/\nhelm upgrade drill1 drill/\n```\nAlternatively, provide the count as a part of the `upgrade` command:\n```\n# helm upgrade \u003cHELM_INSTALL_RELEASE_NAME\u003e drill/ --set drill.count=2\nhelm upgrade drill1 drill/ --set drill.count=2\n```\nIf autoscaling is enabled, \n```\n# helm upgrade \u003cHELM_INSTALL_RELEASE_NAME\u003e drill/ --set drill.count=\u003cNEW_MIN_COUNT\u003e --set drill.autoscale.maxCount=\u003cNEW_MAX_COUNT\u003e\nhelm upgrade drill1 drill/ --set drill.count=3 --set drill.autoscale.maxCount=6\n```\n### Autoscaling Drill Clusters\nThe size of the Drill cluster (number of Drill Pod replicas / number of drill-bits) can not only be manually scaled up or down as shown above, but can also be autoscaled to simplify cluster management. When enabled, with a higher CPU utilization, more drill-bits are added automatically and as the cluster load goes down, so do the number of drill-bits in the Drill Cluster. The drill-bits deemed excessive [gracefully shut down](https://drill.apache.org/docs/stopping-drill/#gracefully-shutting-down-the-drill-process), by going into quiescent mode to permit running queries to complete.\n\nEnable autoscaling by editing the autoscale section in `drill/values.yaml` file.\n\n### Package\nDrill Helm Charts can be packaged for distribution as follows:\n```\n$ helm package drill/\nSuccessfully packaged chart and saved it to: /Users/agirish/Projects/drill-helm-charts/drill-1.0.0.tgz\n```\n\n### Uninstall\nDrill Helm Charts can be uninstalled as follows: \n```\n# helm [uninstall|delete] \u003cHELM_INSTALL_RELEASE_NAME\u003e\nhelm delete drill1\nhelm delete drill2\n```\nNote that `LoadBalancer` and a few other Kubernetes resources may take a while to terminate. Before re-installing Drill Helm Charts, please make sure to wait until all objects from any previous installation (in the same namespace) have terminated.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagirish%2Fdrill-helm-charts","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fagirish%2Fdrill-helm-charts","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fagirish%2Fdrill-helm-charts/lists"}